This notebook was developed by Keneth Garcia. Source and license info are on GitHub.

SieveComP: Advanced Usage¶

For those user that do not want to rely on the default TOML structure of SieveComP, this notebook will show how to use the library in a more advanced way, without requiring the user to create a TOML file. This is useful for those that want to use SieveComP in a more programmatic way, or for those that want to integrate SieveComP into their own codebase.

As a baseline of comparison, we will state some examples where I will show both the TOML and the programmatic way of using SieveComP. The examples stated here are real uses cases that I have encountered in my work, and I hope they will be useful for you as well.

One of SieveComP's most powerful features is its ability to bridge static configurations with dynamic Python applications. While your TOML files define standardized, reproducible SQL queries, building custom interfaces (like a CLI or GUI) often requires filtering data on the fly based on user input, such as date ranges.

Building Dynamic SeisComP Queries with the Runner Class¶

As we already show in the duplicates tutorial notebook, instead of hardcoding new SQL files for every possible filter combination, you can use a "query wrapping" technique. By loading the static SQL from your configuration and wrapping it inside a dynamic subquery, you can safely inject new parameters without breaking the original SQL structure.

This is really useful when developing custom applications on top of SieveComP, given that you often need to fetch raw catalog data for a specific time window before deciding whether to run the full quality-control engine. To achieve this, we can write a wrapper function that intercepts the SQL query defined in your TOML file, injects dynamic WHERE clauses for the start and end times, and executes it using the Runner's override parameters.

NOTE: In static cases where you don't need to create a lot of new SQL or TOML files, you can just use the Runner class with a full query including the time range to make this easier (see the Quickstart guide).

How the Wrapper Works:

• Config Extraction (Steps 1 & 3): We use the Runner's internal Configuration Manager (_cm) to extract the exact name of the time column and the raw SQL string from your existing TOML definitions. This ensures your script remains synchronized with your configuration file.

• Safe Parameterization (Step 2): We map the pandas Timestamps into standard SQL datetime strings and load them into a dictionary (sql_params). SieveComP's engine passes these to the database driver safely, preventing SQL injection vulnerabilities.

• The Subquery Trick (Step 4): Simply appending WHERE start_time = X to a complex SQL file full of JOIN statements will usually cause syntax errors. By placing the original query inside FROM (...) AS base_query, we create a clean, safe table representation that we can easily filter and sort.

• Engine Overrides (Step 5): The runner.run() method accepts a sql_text_override. By setting perform_duplicates=False and perform_checks=False, we tell the SieveComP engine to act purely as a data fetcher, returning the unflagged data instantly.

Usage Example:

Once defined, you can use this function to easily paginate through days or months of data (don't forget to set the connection parameters in your TOML file):

Beyond the TOML: Custom Workflows and Advanced Architecture¶

While SieveComP's TOML files excel at creating standardized, reproducible quality-control pipelines, building custom tools—such as an interactive Jupyter dashboard, a Streamlit web app, or a specialized data parser—demands greater flexibility. You often need to filter data on the fly based on user input, tweak checking parameters without editing text files, or simply process raw data at maximum speed.

To bridge the gap between static configurations and dynamic Python applications, SieveComP is built with a layered architecture. Depending on your specific use case, you can interact with the engine at three distinct depths:

1. The Dynamic Wrapper (High-Level): As we explained previously, the dynamic wrapper allows you to intercept static SQL queries from your configuration and wrap them in dynamic bounds to fetch specific data slices directly from the database.

2. In-Memory Rule Injection (Mid-Level): Bypassing the .toml file entirely by constructing programmatic condition dictionaries and pushing them directly into the Runner's active memory for instant re-evaluations.

3. Raw Vectorized Masking (Low-Level): Stripping away the configuration manager completely to apply SieveComP's lightning-fast, core NumPy masking functions directly to your Pandas DataFrames.

The following examples will break down how to implement the last two approaches, giving you complete programmatic control over how you query, evaluate, and filter your seismic catalogs.

1. In-Memory Rule Injection - High RMS Events¶

Let's suppose that you want to filter out events with a high RMS value, but you don't want to create a new TOML file for this. Instead, you can define the condition in memory and inject it into the Runner's active configuration.

Because SieveComP parses TOML files directly into Python dictionaries under the hood, creating a programmatic instance of a TOML configuration simply means constructing the equivalent Python dictionary and injecting it into the Runner class:

Once you have your programmatic dictionary, you can bypass the .toml file entirely by modifying the ConfigManager's data directly in memory.

2. Raw Vectorized Masking - Invalid Labeling¶

Let's suppose you want to filter out events that have invalid labeling, such as events with a label value of None or an empty string. Instead of creating a new TOML file or injecting rules into the Runner, you can directly apply vectorized masking using SieveComP's core functions. This approach is the fastest and most flexible, allowing you to manipulate your DataFrame directly.

Translating a categorical TOML check into a raw vectorized mask requires mapping the keys from the [[checks.conditions]] block:

[[checks]]
name = "Invalid label"

    [[checks.conditions]]
    rule_type = "category"
    column = "event_type"
    mode = "not_in"
    values = ["earthquake", "not locatable", "volcanic eruption", "explosion", "not existing", "outside of network interest"]

directly to the arguments of the available functions at sievecomp.checks.vectorized_masks. The following example shows how to implement this check programmatically:

Extended Examples and Custom Workflows¶

With the techniques outlined above, you can now build highly customized workflows that leverage SieveComP's powerful engine without being constrained by static TOML files. Whether you're developing a real-time monitoring dashboard, a batch processing pipeline, or an interactive data exploration tool, these programmatic approaches give you the flexibility to adapt SieveComP to your specific needs. This long LONG LONG section of the tutorial is meant to show you how to use SieveComP in a more advanced way, without requiring the user to create a TOML file. This is useful for those that want to use SieveComP in a more programmatic way, or for those that want to integrate SieveComP into their own codebase.

NOTE: The following examples are REAL use cases that I have encountered in my work, and I hope they will be useful for you as well. The examples are based on the seismic revision routine at RSNC, which is a real-world application of SieveComP. Take into account that, for the provided time range, all the events have been checked and modified if they satisfy the conditions of the checks. Therefore, the number of events that satisfy the conditions of the checks may be zero. However, the code is still valid and can be used to check for events that satisfy the conditions of the checks in other time ranges or datasets.

a. Locatable events: The Main Challenge of Single Comparisons¶

A check of the seismic revision routine at RSNC is to identify events that are locatable, but are labeled incorrectly as "not locatable". At RSNC, an event is locatable if it has at least 4 p phases and 2 s phases associated to it and are used to the localization model. This check is important because it can help to identify earthquakes that may have been misclassified and may require further investigation.

However, previous versions of the routine only checks a verification on the quality_associatedPhaseCount column to be greater than or equal to 8 (plus one due to an event with 6 phases in SeisComP3 will have a _quality_associatedPhaseCount_ of 7). Then, an event with for example 8 p phases and 0 s phases would be flagged as locatable, which is not correct.

The challenge here is that query the number of p and s phases associated to each event requires a join between the Origin and Arrival tables in the database, which can be time-consuming, especially for large datasets. Additionally, the check needs to be performed for each event individually, which can further increase the time taken to perform the check. Therefore, the strategy here is to use the columns quality_associatedPhaseCount, quality_usedPhaseCount, quality_usedStationCount and quality_associatedStationCount to create a vectorized check that can identify locatable events without the need for a join between the tables. This approach can significantly reduce the time taken to perform the check, while still providing accurate results.

After internal research at SGC, the locatable check is a composition of a column-to-column comparison and a numeric comparison. The conditions to check are:

In particular if an event with not locatable label have:

$$ quality\_usedStationCount \geq 4 \quad \text{and} \quad quality\_associatedPhaseCount \geq quality\_usedStationCount + 2$$

then it can be flagged as locatable, and therefore it should be reviewed to check if the label is correct or not. For SeisComP3 we can use the previous verification based on:

$$ quality\_associatedPhaseCount \geq 8$$

where the cutoff for SeisComP3 and SeisComP6 is 2026-03-17 00:00:00. We can implement this check by creating a boolean mask for the total condition, defined as:

$$ (event\_type = 'not locatable') \quad \text{and} \quad (quality\_usedStationCount \geq 4) \quad \text{and} \quad (quality\_associatedPhaseCount \geq quality\_usedStationCount + 2) \quad \text{if} \quad time\_value \geq 2026-03-17 \quad 00:00:00$$

$$\text{or}$$

$$ (event\_type = 'not locatable') \quad \text{and} \quad (quality\_associatedPhaseCount \geq 8) \quad \text{if} \quad time\_value < 2026-03-17 \quad 00:00:00$$

The next code snippet shows how to implement this check using the vectorized functions from SieveComP. The final result is a DataFrame with all the events that are locatable but are labeled as "not locatable". This check can be used to identify events that may have been misclassified and may require further investigation.

WARNING: Since 11-03-2025 17:00 HL the not locatable check was modified for events inside volcanic zones. Previously, all events inside these zones MUST have the not locatable label unless they have a comment with the word "DESTACADO". However, since that date, events inside these zones can be locatable and MUST have the SGC preferred origin.

b. Check if the event is inside/outside local zone and has wrong label¶

An example to apply polygonal checks is the case of events where the event type is "earthquake" or "volcanic eruption", but the event is located outside of the local zone defined by the polygon in the arealocal.txt file. This check can be implemented by creating a subset of the events that have event type "earthquake" or "volcanic eruption", and then applying the polygonal check to this subset to identify which events are outside of the local zone.

For this check we will consider two cases:

1. The event is outside the local zone (i.e., outside of the polygon defined in the arealocal.txt file) and event_type is NOT "outside of network interest" or "not locatable" or ''not existing''.
2. The event is inside the local zone (i.e., inside the polygon defined in the arealocal.txt file) and event_type is "outside of network interest".

c. Check if the event has not been processed by the user¶

To check if an event has not been processed by the user, we need to satisfy two conditions:

1. The creationInfo_author should be inside ["scanloc", "scautoloc_reg", "scanlocbay", "AI_picker"]
2. The event['type'] must be non "not existing"
3. The creationInfo_agencyID must be "SGC"

all joined with the 'and' condition. We can implement this check by creating three boolean masks for each condition, and then combining them using the combine_masks function:

d. Check if any event inside the Caribbean Plate and Pacific Plate zones has depth higher than 30 km¶

For this check, we need to satisfy three conditions:

1. The event is inside the Caribbean Plate or Pacific Plate zones, which are defined by the polygons in the caribe.txt and nazca.txt files.
2. The event has a depth higher than 30 km.
3. The event type is "earthquake" or "outside of network interest".

We can implement this check by creating a boolean mask for each condition, and then combining them using the combine_masks function:

e. Check if a 'DESTACADO' event has the correct label¶

At this section we need to check the following conditions:

1. The event is NOT, ["not locatable", "outside of network interest", "volcanic eruption", "explosion", "not existing"]
2. The event have magnitude higher than 4.0
3. The event have NOT a comment with the word "DESTACADO" (considering both byte string and regular string)

However, since 2026 those events with longitudes lower than -82.5 (adjusted for Colombian local area) have a higher limit for the magnitude, which is 5.0 instead of 4.0, due to the low capability of the local network to locate events in that area. Here we can use two different approaches:

1. Continue with the previous boolean mask for magnitude higher than 4.0, and add a mask for events with longitudes lower than -82.5 that have magnitude higher than 5.0, and then combine them with an 'or' condition.
2. Create a new boolean mask considering the longitude condition. Then join the 'DESTACADO' comment mask with the new magnitude mask, which considers the longitude condition, using an 'and' condition.

We will implement the second approach, as it is more efficient and easier to read:

$$ (longitude \leq -82.5 \quad \text{and} \quad magnitude > 5.0) \quad \text{or} \quad (longitude > -82.5 \quad \text{and} \quad magnitude > 4.0)$$

f. Verify if 'DESTACADO' events inside NonLinLoc zone use the correct earth model¶

To check this condition, we need to verify the following:

1. The event has a comment with the word "DESTACADO" (considering both byte string and regular string)
2. The event is inside the NonLinLoc zone, which is defined by the polygon in the NLL.txt file. However, it must not be inside the VMM zone, which is defined by the polygon in the VMM.txt file, as events inside the VMM zone should use a different earth model.
3. The event has the correct earth model specified by the 'earthModelID' column with value Poveda_et_al_2018.
4. The creationInfo_agencyID should be "SGC", as the NonLinLoc locations are only performed for events processed by SGC.

g. Check for events outside NLL zone with NLL earth model¶

This check is similar to the previous one, but in this case we need to check for events that are outside the NonLinLoc zone but have the Poveda_et_al_2018 earth model, which is only used for events inside the NonLinLoc zone. The conditions to check are:

1. The event is outside the NonLinLoc zone, which is defined by the polygon in the NLL.txt file.
2. The event has the Poveda_et_al_2018 earth model specified

h. Potentially locatable events¶

As explained earlier, the criteria for those events that are not locatable but need one p or one s phase to be locatable looks like:

If an event have 3 p and at least 2 s phases, it requires only one p to be locatable, or if an event have 4 p and at least 1 s phase, it requires only one s to be locatable:

$$ quality\_usedStationCount == 3 \quad \text{and} \quad quality\_associatedPhaseCount \geq quality\_usedStationCount + 2$$ $$ or $$ $$ quality\_usedStationCount \geq 4 \quad \text{and} \quad quality\_associatedPhaseCount \geq quality\_usedStationCount + 1$$

Using the same structure, for SeisComP3 we can use the previous established criteria:

$$ quality\_associatedPhaseCount \leq 7$$

i: Check correspondence between zones and magnitude labels¶

At the RSNC we consider two types of polygons:

1. Seven objects for selecting magnitude_type: called zona[1-5].txt, VMM.txt and ptogaitan.txt, where magnitude_type should be MLr_[1-5], MLr_vmm and MLr_PtoGtn respectively.
2. Four objects for selecting special earth models: VMM.txt should have only events with 'VMM' earth model, carma.txt with 'CARMA' earth model, ptogaitan.txt with 'Pto_Gaitan' earth model and cesar.txt with 'modelCesar2'.

The two polygon types are NOT equal, excepting for VMM and Pto_Gaitan zones: For example, it can exist events with CARMA model with MLr_3 or MLr_4 magnitudes, but there aren't events with VMM model with magnitudes different from MLr_vmm, or events with Pto_Gaitan model with magnitudes different from MLr_PtoGtn. Also, zona_vmm.txt is inside zona3.txt and zona_PtoGaitan.txt is inside zona5.txt, but the rest of the polygons are not nested.

Furthermore, the check must be applied in opposite cases: Check for any event outside the corresponding polygon with the corresponding magnitude type, and check for any event inside the corresponding polygon with a magnitude type different from the corresponding one. For example, for the zona3.txt polygon, we need to check for events outside this polygon with magnitude type 'MLr_3' (a mistake), and events inside this polygon with magnitude type different from 'MLr_3' (another mistake). This means that for each polygon we need to check for two different conditions, which can be implemented by creating two boolean masks for each polygon and magnitude type, and then combining them with an 'or' condition.

One last thing: This check must be applied for every event different from not existing and not locatable and if the event have a 'DESTACADO' comment, it can have a moment magnitud (Mw) fixed instead of the corresponding MLr magnitude, so those events with 'DESTACADO' comment should be considered from this check (MUST NOT be ignored due that they can have a fixed Mw instead of the corresponding MLr magnitude, but they also can have a MLr magnitude different from the corresponding one, which is also a mistake).

i.1: Check for events outside zona3 with magnitude type MLr_3 and events inside zona3 with magnitude type different from MLr_3¶

The condition follows:

$$ ((magnitude\_type = MLr_3 \quad \text{and} \quad event \quad \text{is outside} \quad zona3) \quad \text{or} \quad (magnitude\_type \neq MLr_3 \quad \text{and} \quad event \quad \text{is inside} \quad zona3)) \quad \text{and} \quad (event\_type \neq 'not existing' \quad \text{and} \quad event\_type \neq 'not locatable')$$

and should be applied for events with event_type different from 'not existing' and 'not locatable'. Moreover, we must consider the 'DESTACADO' condition also, where events with this comment can have a moment magnitude (Mw) instead of the corresponding MLr magnitude, even if they are inside the corresponding polygon. This means that for events with 'DESTACADO' comment, the condition should be:

$$ If \quad comment \quad contains \quad 'DESTACADO' \quad then \quad magnitude\_type \quad in \quad 'Mw'$$

Summaryzing:

$$\text{Valid\_Event\_Type} \quad \mathbf{AND} \quad (\text{mlr3\_mask} \quad \mathbf{XOR} \quad \text{inside\_zona3\_mask}) \quad \mathbf{AND} \quad \mathbf{NOT} \quad (D \quad \mathbf{AND} \quad M)$$

where D is the condition that the comment contains 'DESTACADO' and M is the condition that the magnitude type is in ['MLr_3', 'Mw'].

i.2. Check for events outside zona4 with magnitude type MLr_4 and events inside zona4 with magnitude type different from MLr_4¶

A similar approach can be followed for the zona4.txt polygon, where the magnitude type should be 'MLr_4'. Let be:

• I = Events inside zona4 polygon
• M = magnitude_type == 'MLr_4'
• V = valid event type, meaning event_type is neither 'not existing' nor 'not locatable'
• D = comment contains 'DESTACADO'
• A = magnitude_type == 'Mw'

Then the condition for zona4 is:

$$ V \quad \mathbf{AND} \quad (M \quad \mathbf{XOR} \quad I) \quad \mathbf{AND} \quad \mathbf{NOT} \quad (D \quad \mathbf{AND} \quad A)$$

where the first part of the condition is the same as for zona3, but changing the magnitude type to 'MLr_4', and the second part of the condition is also the same, where events with 'DESTACADO' comment can have a moment magnitude (Mw) instead of the corresponding MLr magnitude, even if they are inside the corresponding polygon.

i.3. Zone2¶

As we see in previous sections, the format for checking regular zones follows the same format:

$$ V \quad \mathbf{AND} \quad (M \quad \mathbf{XOR} \quad I) \quad \mathbf{AND} \quad \mathbf{NOT} \quad (D \quad \mathbf{AND} \quad A)$$

where V is the valid event type condition, M is the magnitude type condition, I is the inside polygon condition, D is the 'DESTACADO' comment condition and A is the magnitude type in 'Mw' condition. However, for the zone 2 we have an extra condition: The VMM polygon is inside the zona2 polygon, but events inside the VMM polygon should have magnitude type 'MLr_vmm' instead of 'MLr_2', so we need to add another condition for events inside the VMM polygon with magnitude type 'MLr_vmm'. As we only want to check events for zone2 discarding events inside VMM zone, we can create a mask for events inside zona2 but outside VMM, and then combine it with the magnitude type condition with a 'xor' condition, and follows the same structure as before:

$$ I_{2,outer} \quad = \quad I_{2} \quad \mathbf{AND} \quad \mathbf{NOT} \quad I_{VMM}$$

and finally:

$$ V \quad \mathbf{AND} \quad (M \quad \mathbf{XOR} \quad I_{2,outer}) \quad \mathbf{AND} \quad \mathbf{NOT} \quad (D \quad \mathbf{AND} \quad A)$$

i.4. Zone5¶

Zone 5 follows the same format as zone 2, but in this case the inner polygon is zona_PtoGaitan.txt and the magnitude type for events inside this inner polygon should be 'MLr_PtoGtn' instead of 'MLr_5'. So:

j. DESTACADO events with M<3 and outside interest network¶

It is almost impossible that an event with magnitude lower than 3 and outside network interest (defined in arealocal.txt) will be a DESTACADO event, as these events are usually not significant events in terms of impact and felt reports, and therefore they are usually not felt and do not cause damage. Let's implement a check to identify these events, which can be a mistake in the labeling of the event or in the magnitude estimation.