Proposal: Formalized categorization of QC metrics

[bittremieux]

QC metrics in the PSI-MS CV currently use the general relationship has_metric_category to encode multiple different kinds of information (e.g. workflow stage, measurement scope, dependency type). However, these categories are mixed together and applied inconsistently, making it difficult for both humans and software to interpret QC metric semantics in a structured way.

We propose to convert this into a formalized, ontology-driven taxonomy of QC metrics, using seven distinct classification dimensions. Each dimension captures one orthogonal aspect of a metric's semantic meaning (e.g. which workflow stage it refers to, what kind of property it measures, how its values should be interpreted, etc.).

By replacing the current catch-all relation has_metric_category with specific relationship types (defined as Typedefs at the top of the CV), we can explicitly encode this semantic information in a consistent, machine-readable, and reasoning-friendly way.

Current situation

At present, QC metrics are categorized with:

relationship: has_metric_category MS:4000009 ! ID free metric
relationship: has_metric_category MS:4000012 ! single run based metric
relationship: has_metric_category MS:4000018 ! XIC metric

All categories are encoded through the same has_metric_category property, even though they represent different kinds of information:

• one about the data dependency (ID free metric),
• one about the measurement scope (single run based metric),
• and one about the workflow stage (XIC metric).

This approach:

• makes it unclear which category types are (or aren't) specified for a given metric,
• is hard to maintain consistently,
• and prevents downstream tools or reasoning engines from automatically inferring relationships (e.g. "which metrics assess chromatography performance across runs?").

Proposed change

We propose to introduce seven distinct relationship types, one for each classification dimension, to replace has_metric_category.

Each relationship type will be defined as a Typedef in the ontology header, e.g.:

[Typedef]
id: has_measurement_scope
name: has measurement scope
def: "Links a QC metric to the level of data aggregation it summarizes (e.g. run level, batch level, study level)."
is_a: has_metric_category

This makes the ontology structure more expressive and explicitly states what each relationship means.

The seven classification dimensions

Each dimension encodes a different facet of a QC metric's meaning:

Classification Dimension	Definition	Example Subclasses
Workflow stage	The experimental or computational stage to which a QC metric applies.	Sample preparation, chromatography, ionization, MS1 acquisition, quantification, alignment.
Analytical dimension	The fundamental property being measured.	Mass accuracy, chromatographic performance, intensity stability, fragmentation efficiency, contamination, etc.
Information dependency type	The type of data or information required to compute the metric.	Raw acquisition data, identification results, quantification results, hybrid, reference data.
Measurement scope	The data aggregation level a metric summarizes.	Spectrum level, run level, batch level, study level.
Acquisition strategy	The acquisition mode or instrument configuration relevant to the metric.	DDA, DIA, targeted, imaging, ion mobility-coupled, acquisition-mode independent.
Quality interpretation type	The directionality or qualitative meaning of the metric's values with respect to quality.	Higher is better, lower is better, target range, categorical, trend.
Metric value type	The structural nature of the metric's output values.	Single value, tuple, table, matrix.

These seven axes are orthogonal; every QC metric can be classified along each of them, and each relation type conveys a unique piece of semantic information.

Example: Updated metric definition

Below is an example showing how a QC metric (MS:4000051 XIC-FWHM quantiles) would look under the proposed system.

Old definition (current style)

is_a: MS:4000004 ! n-tuple
relationship: has_metric_category MS:4000009 ! ID free metric
relationship: has_metric_category MS:4000012 ! single run based metric
relationship: has_metric_category MS:4000018 ! XIC metric

New definition (proposed style)

is_a: MS:4000001 ! QC metric

relationship: part_of_workflow_stage MS:XXXXXXX ! chromatography stage
relationship: measures_property MS:XXXXXXX ! chromatographic performance metric
relationship: depends_on_data_type MS:XXXXXXX ! raw acquisition data
relationship: has_measurement_scope MS:XXXXXXX ! run level
relationship: applies_to_acquisition_mode MS:XXXXXXX ! acquisition mode independent
relationship: has_quality_directionality MS:XXXXXXX ! lower is better
relationship: has_value_type MS:XXXXXXX ! tuple

Each line now encodes a specific semantic relationship that tools can reason over, index, or validate.

Benefits

• Clarity and consistency: Each relationship clearly states what kind of information it represents. Ontology editors and contributors can immediately see whether all relevant facets are defined for a given metric.
• Machine-readable semantics: Downstream QC tools (e.g. mzQC validators, dashboards, or repositories) can query or filter metrics by category:
  • "Show all chromatography-related metrics."
  • "Which metrics are computed at the run level?"
  • "Which metrics are ID-based?"
• Backward compatibility: Terms and identifiers remain unchanged, so existing tools and mzQC files continue to work.

---

Summary

This proposal does not change any existing QC metric terms or their identifiers, it only restructures how relationships are expressed in the CV to make semantic information more explicit and machine-readable.

In other words:

• The terms themselves remain the same, so no impact on downstream tools or data formats (e.g. mzQC files).
• Only the relationship layer (previously expressed through has_metric_category) is being refined into multiple, more specific relationships.
• The ontology will remain fully backward compatible, existing metrics and software will continue to function as before.

Implementing this will require some curation work to reannotate QC terms using the new relationship types. This will be done incrementally through a series of pull requests, starting with the addition of the new Typedef entries at the top of the CV. Once those are in place, individual metrics can gradually adopt the new structure.

With this issue we want to discuss this change first, to ensure full compatibility with existing conventions and downstream usage, before beginning implementation.

[bittremieux]

Related issue from the ELIXIR BioHackathon with full metric categorization across the seven dimensions as determined so far: MS-Quality-Hub/biohackathon2025/issues/4

[edeutsch]

I'm sure you've considered this, but why not just do this with multiple parents and is_a? Something like:

is_a: MS:XXXXXXX ! chromatography metric
is_a: MS:XXXXXXX ! chromatographic performance metric
is_a: MS:XXXXXXX ! raw data acquisition metric
is_a: MS:XXXXXXX ! run level metric
is_a: MS:XXXXXXX ! independent acquisition mode metric
is_a: MS:XXXXXXX ! lower is better metric
relationship: has_value_type MS:XXXXXXX ! tuple

This way you don't have to define all these relationships, which most software won't understand . On the other hand, with is_a, everyone (OLS, BioRegistry, etc.) can already handle these just fine. Asking the question "What are all the run level metrics" is easy by just getting "run level metric"s descendents.

It would be easy to write a reasoner that validates that each QC term is an descendent of each of the classification dimension terms. I left has_value_type since we already have that.

I don't feel strongly, but thought I'd probe to see why this wasn't a good solution.

[bittremieux]

Yes, that's a good point, and we considered it. One downside is that it has the same conflation of categories within a single encoding, which would then be is_a rather than has_metric_category. So we'd still lose the semantic distinction between the different categories.

In a related fashion, using is_a for all categories would imply:

"XIC-FWHM quantiles" is a kind of chromatography metric and also a kind of raw-data metric and also a kind of lower-is-better metric.

I.e. is_a implies "is a subtype of", but these are facets rather than types. So this multiple inheritance would conflate orthogonal properties (scope, input data, quality direction) with taxonomy (metric kind). By contrast, the different relationships preserve that these are categorical descriptors rather than hierarchical ancestors.

To a lesser extent, I also think that the DAG would become quite dense and complex when every metric would have half a dozen parents.

[cbielow]

maybe to add a small point in favour of separate types: when looking at a metric which carries only 5 of the 7 annotations, it's easier to discern which category is missing, e.g. during review.

What is still not entirely clear is if we should encourage all dimensions to be populated at all times, or if missing values imply that this dimension is not applicable. Example: if Acquisition strategy were not applicable because a metric is valid for any acquisition type, should we a) not mention it, or b) introduce an any value for that category? The latter would make the CV more verbose, but on the other hand it ensures that the metric cannot be forgotten (as opposed to not being mentioned because there is no 'any' value).
I have a slight preference for option b, but happy to discuss.

[edeutsch]

okay, well, I am still a bit disenchanted with this scheme, but I won't get in the way.

I would imagine that getting a student to write a simple reasoner that would ensure that all terms had each of the 7 categories would be a good/easy thing. With a bonus of standing up a little web app into which one could paste proposed term entries to ensure that they conform to the rules.

One possible inconsistency to have to check for is using a term that doesn't belong in the category. such as acquisition strategy lower is better.

[edeutsch]

One more thought I'll grumble about, how will one know that:

relationship: part_of_workflow_stage MS:XXXXXXX ! raw acquisition data

is not correct?

If a direct inheritance scheme with is_a relationships were used, the fallacy is not possible. But in this this relationship scheme, it would be possible to associate terms with the wrong classification dimension, and ordinary reasoners would not know that it is wrong.

Presumably you would still need to have parent terms for all these aspects and then have some bespoke code that would associate a classification dimension with a parent term. And target terms would have to be a descendent of the associated parent term (or the parent term itself to mean "any").

[bittremieux]

If a direct inheritance scheme with is_a relationships were used, the fallacy is not possible. But in this this relationship scheme, it would be possible to associate terms with the wrong classification dimension, and ordinary reasoners would not know that it is wrong.

I'm not sure I understand this issue. If everything is using an is_a, there indeed can't be a mix-up, but there's also no distinction between the different categories (without having to parse the CV tree). Instead, here the different categories are made explicit compared to the previous has_metric_category which also just grouped everything together in a single type of relationship.

I agree that there is no direct mechanism to enforce that the values are assigned to the correct category. Although we're now discussing whether we can build a quick validator for this, which shouldn't be too hard.

That said, I do agree that there is some benefit to using inheritance (with is_a) as opposed to the flat hierarchy used for the QC terms currently and which would be maintained with the initial proposal. Among the different categories, the "analytical dimension" basically defines what kind of a metric it is and forms a taxonomy of QC metrics types, so this should be specified using is_a.

In contrast, the other six categories describe attributes, rather than subtypes. For example:

• part_of_workflow_stage chromatography stage → contextual location in the workflow.
• depends_on_data_type raw acquisition data → required input.
• has_measurement_scope run level → aggregation scope.
• applies_to_acquisition_mode DDA → method applicability.
• has_quality_directionality lower is better → interpretive semantics.
• has_value_type tuple → structural format.

None of these mean that the metric is a kind of workflow stage or acquisition mode; they are properties, hence typed relationships are appropriate IMO.

[cbielow]

preliminary validation app to be used for validation before accepting new QC terms into the CV: https://mzqc-term-validator.lovable.app/
This is subject to change when we try to fit existing metrics into these categories (renaming/extension of possible values etc).

For now it seems a good idea to

• require all 7 dimensions (or 6, see above), which in turn requires to make all dimensions 'complete, i.e.
• introduce a catch-all where necessary (like any for Acquisition strategy)

[bittremieux]

I described this system in detail here.

The corresponding PR contains more updates to the mzQC documentation that would help users understand and apply these categorizations.

[mobiusklein]

I think the problem Eric is referring to is that neither OBO nor OWL have a built-in mechanism for saying whether a declared relationship is semantically valid. This is a design goal, because those tools are about declaiming truths and defining predicates that imply truths, which can be used to define constraints on other things. ShEx and SHACL are separate semantic tools for defining constraints on ontologies but that's more work maintain too.

Is there a way to build that validator into the CI workflow so we can detect if definitions start to drift over time?

[bittremieux]

Is there a way to build that validator into the CI workflow so we can detect if definitions start to drift over time?

Yes, that should certainly be possible. This was vibe-coded as a quick proof-of-concept, so needs to be made a bit more robust, runnable as a CLI tool, and updated to include accessions of terms permissible in the different categories once they've been added to the CV. But then it can be included during CI to check for terms in the QC namespace and verify that the categorization is applied correctly.

[mobiusklein]

My first pass through the rules didn't raise any concerns. I prefer this predicate-based design to ultra-factored semi-redundant hierarchies.

The only question that came to mind during my reading was whether a metric could have multiple values along one of the dimensions, e.g., could you have a metric that is both an ion mobility metric and an identification confidence metric, violating the exactly one is_a constraint? If one wished to torture the concept, it would be possible to produce something like "spectral library ion mobility deviation standard deviation" which would describe how far from a reference library of ion mobility annotated analytes which sounds like an ID confidence because it is related to an identification problem, but it is etiologically an ion mobility metric.

[bittremieux]

Yes, that's a relevant question, and one we were already discussing as well. For now we've decided to have a single value per dimension, and that's how @cbielow implemented the validator. If it's a bit ambiguous, like the example, then it can be assigned it to its dominant property.

We can relax this requirement of course and say that typically it should be a single value per dimension, but for some exceptions that might not be the case. I don't really have a strong opinion about this / data to inform that decision.