1.0.0 Trial Use Review Ballot (November 2024)

[ahwagner]

Welcome to the Cat-VRS Nov '24 Trial Use review

Dear reviewer,

If you are able and willing to participate in the development of these standards, please list individual concerns as separate comments below. All parties are welcome.

Some guidance on creating a comment
Ideally, you should place a Heading tag on the first line of your comment with a summary heading of the specific issue or question (see first comment below as a model). There is no limit to how many comments you can create. It would also be helpful to first look for a similar or related comment and work within the reply thread of that comment if possible.

Thank you for your participation,
The GKS product leads and co-leads

Dates to Remember!

• The ballot review period will begin on Thursday, Nov 28, 2024 and close on Friday, Dec 6, 2024.
• There will be an opportunity for a live feedback and review of the process to be held during the Categorical Variant Meeting on Wednesday, Dec 4, 2024. All VA and Cat-VRS reviewers are welcome to attend.

Implementer Survey (<5mins)

• For all current, future or possible implementers we would like you to also answer this very short survey to indicate which of the Trial Use classes you are most interested in. This will help validate the number of implementers as well as help us keep you informed if future changes occur to your classes of interest. Here are the Implementer's Trial Use Review forms for the VA-Spec and Cat-VRS products, please participate, your contributions help advance the standards and make the process work.

The List of Classes under Consideration

This ballot is a GKS community trial use review for the following classes:

Cat-VRS Base classes

• Categorical Variant [docs] [schema] [example]
• Defining Allele Constraint [docs] [schema] [example]
• Defining Location Constraint [docs] [schema] [example]
• Copy Number Count Constraint [docs] [schema] [example]

Cat-VRS Recipes

• Canonical Allele [docs] [schema] [example]
• Protein Sequence Consequence [docs] [schema] [example]

Join the discussion on this page!

Changelog Since September Connect: 1.0.0-connect.2024-09.1...1.0.0-ballot.2024-11.1

---

This discussion was created from the release 1.0.0 Trial Use Review Ballot (November 2024).

[larrybabb]

CategoricalCnv and CopyNumberChangeConstraint SHOULD be added to the TU review...

Currently, the examples only consider the use of DefiningLocationConstraint and CopyNumberCountConstraint in the context of a CategoricalCnv recipe. If CategoricalCnv is NOT up for trial use consideration then those constraints should be removed. If it is then the CopyNumberChangeConstraint SHOULD also be added to the TU review since it is part of the CategoricalCnv recipe.

[DanielPuthawala]

@ahwagner Can you weigh in on this? Should the CategoricalCNV recipe be on the ballot?

[ahwagner]

IMO the way it is set up now is appropriate for where community implementations are at. Recipes reflect community conventions for certain types of categorical variants, e.g. canonical alleles (NCBI, ClinGen). AFAIK there is no community norm for a categorical CNV. However, there is clearly a need for copy number constraints in ClinVar and some of the VICC KBs.

[brendanreardon]

Adding in that moalmanac, and likely CIViC, is interested in using Categorical CNVs with the GeneContextConstraint instead of DefiningLocationConstraint. The GeneContextConstraint is something that we can resolve in the 1.1 release though, it was a bit sidelined in the relevant discussion thread.

Also echoing that it seems to me to be okay to hold off on adding CategoricalCNV and CopyNumberChangeConstraint until a later release. In the meantime instead, we should add an example(s) from NCBI/ClinGen of adding the CopyNumberCountConstraint to a CanonicalAllele or ProteinSequenceConsequence (I'm guessing that they are using this w.r.t. CanonicalAllele though). I'll take ownership of drafting up an example that meets this criteria.

[brendanreardon]

We moved these comments to #88, as it already related to CNV classes for the next trial use release. Please continue the conversation there! 🙇

[korikuzma]

Add Reference implementation to docs

I've created a Python reference implementation here: https://github.com/ga4gh/cat-vrs-python

I will be updating models soon.

[brendanreardon]

Thank you, @korikuzma 🙇! We created an issue to track documentation changes recommended from this ballot, #91.

[brendanreardon]

I moved this to its own Issue (#96), per instruction from Tuesday's meeting. Please continue the discussion there 🙇

[korikuzma]

Add examples

One thing I really like about the new docs for VRS, is that it now includes examples. I think it would be really beneficial to users to add examples for each class.

[afrubin]

Thank you for this. The examples are super useful for our project, especially when we are trying to engage with only a subset of the spec (Protein Sequence Consequence) at this stage.

On a related note, the recipe example file is super useful, but it would be great if the vrs.json file that is referenced in the constraints and members sections was also part of the repo for completeness.

[DanielPuthawala]

Like a fully-verbose example that drills down all the way to the GKS.core stuff?

[afrubin]

I think if one complete example was available it would help implementers, since it would be easier to see what CatVRS requires in one place. Having said that, it might be a lot of work if you don't have those files already so it might be out of scope for this TU ballot.

[brendanreardon]

Linking out to this Issue that Daniel created last week to track example updates! #87

[brendanreardon]

Kori's original comment and subsequent responses are now quoted in the relevant Issue: #87. Please continue to the conversation there 🙇

[Mrinal-Thomas-Epic]

Should relations be a mappable concept?

Applies to both DefiningAlleleConstraint and DefiningLocationConstraint:

• Do we expect different implementations to define their own types of relations? Wouldn't that defeat the purpose of cat-vrs (to be specific and descriptive about variant matching logic)?
• Do we expect implementations to use the other properties of MappableConcept (i.e., label, mappings, extensions)?

[DanielPuthawala]

I'm not sure we've really settled that question yet. AFAIK the current tack is that we can specify some relations, and adopters have those at a minimum, but could define their own custom relations in addition to the base vanilla ones? @ahwagner and @larrybabb how closely does this reflect your guys' current thinking?

[DanielPuthawala]

By way of update, there were three reasons for removing the explicit list of relations from the constraint to Mappable concept instead.

First, it's more DRY this way, and makes it easier to update relations if they are in one place in the spec instead of being distributed amongst the various constraints.

Second, it seems to make more sense to specify required relations in the recipes rather than the constraints, wherein any compatibly-typed relations are optional. Recipes, by contrast, can and do require some minimal relations, e.g. see the Canonical Allele recipe.

Third, the schema isn't the only place in the spec we can give guidance about which relations to include, and this is especially relevant given that the reference implementation and guidance to implementers are really where relation functions will have to be fleshed out in the first place. So perhaps guidance should live there?

I'd be happy to make an issue on this topic, @Mrinal-Thomas-Epic if we want revisit this question, just let me know!

[brendanreardon]

We moved this to its own GitHub issue for tracking purposes: #97 😄 Please continue the conversation there! 🙇

[rhdolin]

Categorical Variant class - minor wordsmithing

"Categorical variants exist solely within genomics knowledgebases" - suggest we broaden this, to something like "Categorical variants exist within genomics knowledgebases, as clinical trial eligibility criteria, and within clinical quality measures".

[brendanreardon]

Totally valid point. Thank you, Bob! We created an issue (#91) to track documentation updates per this ballot, including this thoughtful suggestion.

[rhdolin]

Categorical Variant class - attribute cardinality

1. Consider making categoricalVariant.label required (hard to know what the constraints are constraining otherwise...)
2. categoricalVariant.constraints minimum cardinality should be 1 (why else instantiate the class?)

[DanielPuthawala]

1. I'm not sure we want to do this, since label is usually more of a human-readibility thing, and I can imagine that lots of CatVars in, say, ClinVar, aren't generated with or are even concerned with being human-readible. and so there label would be extraneous. Aside form that, imo the properties of each constriant already record what is being constrained already, be it a location, allele, copy number, etc. That's not to say that you couldn't require label to be populated in a recipe specific to your use-case, though! I'm just not convinced we need it to be required across the board for all use cases.
2. That's not a bad idea. Will have to noodle on this!

[brendanreardon]

As a counter point to requiring minimum cardinality of 1, having the minimum constraints as 0 will allow implementers to pre-allocate and include variant classes that are not yet covered by either cat-vrs or vrs.

For example, if an implementer had a database that contained canonical alleles and other variant types that are covered by vrs/cat-vrs, but also things like... PD-L1 tumor proportion score, they could use the cat-vrs schema for their entire database, not attach any constraints to biomarkers like PD-L1 TPSs, and use extensions to represent the biomarkers as a placeholder.

[brendanreardon]

Hey @rhdolin, just following up on this. We're not going to move this to an Issue to continue the conversation for the time being, but happy to do so in the future if this comes up again.

[rhdolin]

Categorical variant constraints need Boolean operators

categoricalVariant.constraints needs the ability to insert boolean operators (e.g. [constraint A AND (constraint B OR constraint C)]).

[DanielPuthawala]

So the boolean logic is in the schemas, just for Recipes, and not for the base CategoricalVariant class at the moment. But I'm amenable to changing that if it makes sense.

One such thing I'm not sure how we'll handle at the moment are compound CatVars, like EGFR Exon 19 or 21 deletion or MET Amplification and TP53 Loss. Are those the sort of examples you had in mind?

Also, can you elaborate more on what it means to type CatVar constraints as expressions?

[rhdolin]

Thanks for this @DanielPuthawala. Still not quite following - where do you define where allOf/anyOf can go? That's kinda what I was getting at with a data type of expression, where the expression is a string that adheres to a defined syntax that formalizes where AND/OR/NOT/allOf/anyOf can go when combining constraints.

Yes, kinda along the lines of your EGFR and MET/TP53 examples. I was was also thinking, for instance, where a category can be satisfied in more than one way - e.g. where your criteria for amplification might be either an increased copy number or a duplication; or where your criteria for 'BRCA1 mutation' might be either a simple variant or a structural variant.

[DanielPuthawala]

Hey @rhdolin I'm still not quite following yet either. What language/system/type system is the expression data type from? Like in the SQL sense?

I think we should put our heads together in a call and brainstorm this. Currently, some of that, like your example regarding amplification can already be handled as a recipe. But it's also clear to me that some examples CAN'T be handled this way, like the MET/TP53 case, which seems like an irreducibly complex CatVar. This makes me think that perhaps the route forward (though maybe for release later down the road) is to create a higher-order type over CategoricalVariant, which could be defined as being, in contrast to CategoricalVariant which supports a single array of constraints', this higher-order typed class allows for a series of constraints` arrays, together with the boolean logic between them, so that you could say something like

Where:
 x, y, z :  CategoricalVariant
P : HOCategoricalVariant

As random made up example:
P = (x AND y) NOT z

meaning a complex CatVar that requires two separate sets of constraints to be satisfied, and also requires a third set to not be satisfied. And example of this would be something like:

MET Amplification and TP53 Loss, but not MET Gain-of-Function

[DanielPuthawala]

I'mm make an issue to where we can follow up on this and think

[brendanreardon]

@DanielPuthawala created an Issue (#92) for this a bit ago and I have quote-copied this exchange to there. Please continue the conversation in that Issue 🙇

[rhdolin]

Categorical variant - question about the example

I'm unclear why 'NM_004958.4(MTOR):c.5992_5993del (p.Met1998fs)' is used as an example of a categorical variant. This doesn't look to be a categorical variant.

[DanielPuthawala]

It's a canonical allele, so in addition to the specific contextual variant in the label, it also contains as members congruent variants on other builds etc. But more importantly, it's being represented categorically. Even a singleton set CatVar is still a CatVar.

[rhdolin]

@DanielPuthawala I think it actually makes it harder to describe what a 'categorical variant' is, if a simple variant can be a categorical variant. In the readthedocs, we talk about NC_000007.13:g.140453136A>T vs. BRAF V600E, noting that the latter is a categorical variant. But based on the cited example, so could be the former (e.g. by submitting my assayed variant to ClinVar).

[DanielPuthawala]

Well, you're not wrong, but I think we need to be able to do that in order to do assayed-to-categorical variant matching.

As for the example, in the readthedocs, its true that that's now a 100% accurate explanation, but I've yet to come up with abetter one that is as intuitive but captures the real distinction. VRS handles contextual variants, ie specific variants in the context of some specific assay, genome, etc. On the other hand, Cat-VRS handles variants treated as set-theoretic objects, abstracted from the context in which they are assayed, and instead used as labels for classes tied to knowledge, and for which I think it's ultimately irrelevant whether the set contains as it's member a single contextual variant or a much more expansive set of contextual variant, and with other CatVars as definable subsets.

[rhdolin]

Defining Location Constraint - question about the example

Example shows a location constraint with start and end integers, but should probably be start and end ranges.

[DanielPuthawala]

We're currently following VRS conventions, where locations typically contain start and end ranges, but for which unambiguous ranges can be replaced with an integer, so eg:

Location((4,4),(10,10)) = Location (4,10)

In the past I pushed for always using explicit ranges, and the group pushed back on the grounds that (1) this allows for shorter variant entries, (2) that it's not unduly ambiguous, and (3) that GKS already has the tools to support parsing integers as unambiguous ranges, so it wasn't an implementation issue.

[rhdolin]

Thanks @DanielPuthawala , I see your point. But for CNVs, isn't it often the case that, say, inner endpoints are known whereas outer endpoints are not? If I'm interpreting the example correctly, it seems to suggest that we know the precise endpoints for this CNV.

[DanielPuthawala]

Ah, no, it can handle those cases, too. (this is all inherited from VRS)
So let's say we have a range with endpoints A and B.
A and B are each ranges (so the full range is actually a tuple of ranges):
((A1, A2),(B1, B2))
Where A1, A2, B1, and B2 can either be integers or a null value (here I'll use "?" just for readibility)

So a really specific could be: ((4,4),(10,10))
Which as I said above by convention is evaluated to mean the same thing as (4,10)

Here is a range where the outermost possible endpoints are known, but the innermost endpoints are not known: ((4,?),(?,10))
any subsequence including or between 4 and 10 are possible in this range, including 4-10, 4-5, 6-8, 7-7, etc.

Here is a range where the innermost endpoints are known, but the outermost possible endpoints are not known: ((?,4),(10,?))
Any sequence that covers at least positions 4-10 are possible given this range, including 4-10, 2-10, 4-15, 3-20, etc.

And here's a range that is maximally vague: ((?,?),(?,?)), which admits any sequence, regardless of length or position.

Does that make sense to you?

[rhdolin]

Thanks @DanielPuthawala. Yes, makes sense. My question really was about the specific example - I haven't seen CNVs with precise endpoints, so I would have expected the example to have ranges.

[brendanreardon]

This was represented in Issue #87 and I have now quote copied this exchange as a comment on the Issue. Please continue the conversation there 🙇

[rhdolin]

Protein Sequence Consequence - need to have a structured representation of the protein sequence

Example should at lease include a valid pHGVS because as it stands, we only know that this example is for EGFR L858R via the label or alternativeLabels, both of which are text fields.

[afrubin]

I see the point, but we should definitely only support VRS for the DefiningAlleleConstraint for consistency. Keeping this object minimal is valuable so it's easy to pull out and validate, as adding extra fields that may be in some other format would complicate variant matching and validation. Adding it to mappings makes perfect sense to me.

[ahwagner]

I agree that we should only support VRS Alleles for the defining allele constraint, and will also point out that VRS Alleles have several things that can make downstream implementation easier, including a dedicated datatype (expressions) for other syntax representations that can be added to any variation object under the expressions slot.

For example, here is a VRS variant with a SPDI expression.

[afrubin]

It seems to me like we have multiple good options to solve this problem and store an HGVS representation:

• mappings in the CatVRS object
• expressions in the VRS object linked to the DefiningAlleleConstraint

Of these two I favor the latter, because the HGVS representation is a property of the variant not the category.

If we think about the Protein Sequence Consequence use case, the advantage is even more clear. We wouldn't want a mix of protein and nucleotide HGVS strings in the mappings, and downstream users are very likely to want the HGVS for the category members in addition to the DefiningAlleleConstraint variant.

Having said that, I see that in the example there are mappings to specific nucleotide variants (e.g. CA126713) as relatedMatch objects. Is this going to scale well? Is the intention for the implementation to pull the expressions out of the members to generate a set of relatedMatch mappings?

Are there use cases I'm not thinking of where we would want this in mappings instead, or is this an issue that could be solved by clearly documenting that the intended solution is to use the VRS expressions slot?

Maybe my question is really about the purpose of the mappings field in general, and its intended use.

[DanielPuthawala]

As you say, Alan, expressions within the VRS objects is the most precise place to stick the matched HGVS representation. There are several places where an HGVS representation can go in Cat-VRS, but it's not usually a very good fit, computationally, because whereas HGVS is representing contextual variants, Cat-VRS is representing categorical variants, ie sets of contextual variants. As @rhdolin has mentioned elsewhere, this distinction is muddied a bit because some CatVars are singleton sets: they are set objects with one and only one contextual variant member. This makes those CatVars synonymous with the VRS entries, but still distinct for our use-cases for search and matching.

So with that said, where could HGVS representations live in Cat-VRS? One common place will probably be in label, which can be whatever people want to call it, and will probably be especially useful for variant classes where the 'canonical' name in the literature happens to be an HGVS string. For singleton CatVars, mappings also works, as you can make an exact match, or for larger but for CatVars that are contextually well-clustered (like BRAF V600E, because exist in a relatively small range of genomic positions), a mapping to HGVS with a related match may be useful. (and also might be not be, I don't claim to know). And finally, within the expressions that appear with the VRS objects in members. This is, admittedly, the same as saying that HGVS just belongs in VRS, but does mean that Cat-VRS does have a place for such entities that can pull their HGVS expressions along with them. However, the members field is not required to be exhaustive, so the abilit yot match based on HGVS will depend on whether or not the corresponding VRS object happens to be in that field or not, which to my mind poses a similar problem for search by HGVS string matching as does having related matches in mappings.

But of course, that's not how we designed Cat-VRS to be enable search. Instead, the way to be able to find what your looking for is to compute the singleton CatVar representation of your HGVS variant in Cat-VRS, and then query your CatVars by constraints, which is precisely how recipes work.

So in sum, there's a couple places where HGVS expressions could lvie in Cat-VRS, but I think outside of singleton CatVars, the best place is in the members, but also that the use of HGVS for matching in Cat-VRS is probably limited at best.

[afrubin]

Thanks @DanielPuthawala for the detailed response. It's good to see that Cat-VRS has the flexibility to handle all of these cases, and I'm sure that the preferred ways of handling each situation will make their way into the documentation as they are discovered through use.