This notebook was developed by Keneth Garcia. Source and license info are on GitHub.

Python API Quickstart¶

This notebook introduces the core Python API of SieveComP. We will walk through setting up the engine, executing a TOML configuration, and interacting with the resulting data using Pandas.

By the end of this quickstart, you will know how to load a configuration, run the quality-control pipeline, inspect the output DataFrames, and rapidly iterate on your rules without re-querying the database.

1. Setting up the connection to the database¶

SieveComP can connect to multiple databases using either a configuration file, an env file or environment variables. For this tutorial, we will use an env file named .env in our working directory with the following content:

env
# Credentials for SeisComP6 database
SERVER_SC6_USER=consulta
SERVER_SC6_PASSWORD=consulta
SERVER_SC6_HOST=10.100.100.xxx
SERVER_SC6_PORT=3306
SERVER_SC6_DATABASE=seiscomp

However, feel free to use your own credentials in any of the supported formats. The engine will automatically detect and load the credentials.

2. Setting the SQL query¶

SieveComP uses SQL queries to extract data from the database. These queries are stored in separate .sql files, which are referenced in the TOML configuration file. For this tutorial, we will use a very simple query that retrieves the last 20 events from the Event table in the SeisComP database. The query is stored in a file named ../../examples/data/sql/example.sql with the following content:

SELECT * FROM Event LIMIT 20;

3. The Configuration File¶

SieveComP is driven by human-readable TOML files. For this tutorial, assume we have a file named tutorial_config.toml in our working directory with the following content:

[[database]]
name = "connection1"
env_file = "/home/keneth/Documents/Academic_Projects/SieveComP/examples/data/credentials/.env"
env_prefix = "SERVER_SC6"

[[queries]]
name        = "query1"
sql_file    = "/home/keneth/Documents/Academic_Projects/SieveComP/examples/data/sql/example.sql"
description = "Query 1 — Get the last 20 events from the Event table"

[[checks]]
name = "Check events with a methodID"

  [[checks.conditions]]
  rule_type = "category"
  column = "methodID"
  mode = "in"
  values = ["Hypo71"]

[output]
columns = ["_oid", "time_value", "methodID"]

This file defines a database connection, a SQL query, a duplicate detection method, and a quality check to retrieve only events with a specific methodID. If you need help to create your own TOML configuration, please refer to the TOML Configuration Guide.

NOTE: You can modify these file to suit your needs, all these files will be stored at the examples/data directory of the SieveComP repository. You can also create your own files and point to them in the TOML configuration.

4. Initializing the Runner¶

The Runner class is the central orchestator. When you instantiate it, SieveComP reads your TOML file, rigorously validates your schema, and establishes the required database connections:

5. Executing the Pipeline¶

The run() method executes the entire quality-control pipeline. It fetches the events via SQL, runs the duplicate detection algorithms (Adjacent or SSWA), evaluates all spatial/logical checks, and generates the final merged output table.

6. Inspecting the Output¶

The run() method returns a RunResult data class. This object contains four distinct Pandas DataFrames, allowing you to inspect the data at various stages of the pipeline.

6.1. Total Events¶

result.total_events contains the raw, unmodified dataset pulled directly from your SeisComP database.

6.2. Duplicate & Checks¶

result.duplicates contains the events flagged as duplicates by the selected algorithm, while result.checks contains the results of all spatial/logical checks. They feature an appended Observations column detailing exactly which rules were broken.

6.3. Final Output¶

result.output is the compiled review table. If you defined an [output] section in your TOML, this DataFrame will strictly follow your column selections and format preferences.

7. Rapid Iteration (Using the Cache)¶

Designing the perfect quality-control rules often requires trial and error. If you decide to tweak a parameter in your TOML file (e.g., changing the category filter to a different value), you do not need to wait for the database query to run again.

Use update_from_cache() to instantly apply your new TOML rules to the data you already downloaded.