A Checklist / SOP for drafting new constraints

[brendanreardon]

While we have new constraints on the roadmap and have spent meetings discussing them, Daniel and I think that it would be good to get feedback on what the procedure / checklist / standard operating procedure for drafting new constraints should be. This is directly inspired by a Slack conversation with Larry.

I've organized a draft checklist and separated items into two categories: Conceptualization and Development. Once we have an initial version that we are happy with, I imagine creating at least a Discussion Template that includes these items to encourage usage of the checklist. The "Development" items could also be placed within an Issue Template.

The current version of the checklist is in the next post on this thread 👇, and I will edit it as the conversation evolves.

[brendanreardon]

Constraint concept checklist

Conceptualization questions:

• What property of genomic variation does this constraint concept address?
• Identify test set variants that are not representable with the current constraints. Articulate if the current constraints are able to partially represent these variants, and if this new constraint allows complete representation of them. If this constraint aims to address variants not represented within the test set document, please add them.
• Who are the cat-vrs implementers, specifically, who are in need of this constraint, or something like it?
• What are the open questions and design decisions that the group needs to reach consensus on with regard to this constraint concept?

Development checklist:

• Create a Discussion on the Cat-VRS repository with the above information to receive feedback on the constraint idea. Share the Discussion post with implementers who were identified as in need of this constraint concept.
• Create a new Issue and branch per the contributing guide for this constraint.
• Create examples for each of the the test set variants identified above. Each variant should have an example representing it without the constraint, as much as possible, and with the constraint (before / after examples).
• Draft a schema for the constraint. Compile from source and ensure that GitHub actions checks are compiling without error.
• Attend a cat-vrs meeting and discuss the constraint concept.
• Update the readthedocs to show the constraints' schema and examples.
• Open a new Pull Request to review the implementation of this constraint. Maintainers will review the pull request and will merge upon consensus.

This process is intended to be iterative, especially within the Discussion thread and at Cat-VRS meetings.

[brendanreardon]

Here is what the checklist may look like for the FeatureContextConstraint.

Constraint concept: FeatureContext

What property of genomic variation does this concept address? This proposed constraint, FeatureContext, aims to model both Genes and Protein markers within categorical variants. If created as a generalized constraint that can accommodate different subtypes of mappable concepts, it could also support variant consequences; e.g., Missense Mutations, Nonsense Mutations, etc.

Identify test set variants that are not representable with the current constraints. The following constraints are not currently representable, which FeatureContext will alleviate:

• PIK3CA Mutation
• KRAS Activating mutation, as well as listed Gene Function variants such as ERBB2 activating mutation, EZH2 activating mutation, BRCA2 mutation (LOF), PTEN Loss, and PTEN Wild type.
• Region-defined variants such as KRAS Mutation, SH2 DOMAIN MUTATION, TERT Promoter Mutation, and TP53 Conserved Domain Mutation.
• Sequence variants grouped by variant consequence; e.g., PTEN Nonsense.

However, this constraint may be applicable to any test set variant that is associated with a gene or protein marker.

Who are the cat-vrs implementers, specifically, who are in need of this constraint, or something like it? For sequence variants, currently, genomic knowledge bases for cancer are the primary implementers who are in need of this constraint as oftentimes knowledge is aggregated more generally than specific sequence variants. Examples include CIViC, MetaKB, and Molecular Oncology Almanac, (and NCH's VarCat?).

From my understanding, non-cancer resources are usually associating knowledge directly with either individual locations or nucleotide changes.

What are the open questions and design decisions that the group needs to reach consensus on with regard to this constraint concept? The following design decisions should be discussed for FeatureContextConstraint:

This constraint may introduce redundant information within a categorical variant's constraints used to model it. Is the community okay with this?

• For example, when modeling "BRAF p.V600E", the gene can be inferred from the underlying Allele and thus the FeatureContextConstraint would not provide any additional "filtering" for variant matches against the categorical variant.
• At the May 7th, 2025 cat-var meeting we discussed the acceptability of adding an additional property to the DefiningAllele and DefiningLocation constraints to reduce the number of overall constraints being passed. This seemed to be relatively acceptable to the group but we haven't reached consensus on this, I think.

Should this constraint be a highly specific constraint -- separate constraints for gene context, protein marker context, and variant consequence -- or generalizable?

[brendanreardon]

@DanielPuthawala noticed that the GKS Maturity Model documentation also broadly details requirements to draft a new product feature 👇

[brendanreardon]

This was discussed during the June 17th cat-vrs meeting and Daniel created the attached slides as a conversational aid.

Cat-VRS slides.pdf

The main outcome from the discussion was that the SOP should compliment the GKS Maturity Model, specifically the Product Feature Development Process. This will likely be discussed during an upcoming GKS Product Leads call.