Scorpio: Many BA.4/5 misclassified as BA.2* because "Probable Omicron (BA.2-like)"

[corneliusroemer]

I've noticed that the proportion of BA.2* + S:452R, S:486V, S:493. in South Africa (and elsewhere) has been rising recently.

These should really be BA.4/5 but are misassigned, either by pangoLEARN or Scorpio.

It's as much as 25% of South African sequences.

https://cov-spectrum.org/explore/South%20Africa/AllSamples/Past3M/variants?aaMutations=S%3A452R%2CS%3A486V%2CS%3A493.&pangoLineage=BA.2*&

Example sequences:

hCoV-19/South Africa/CERI-KRISP-K038924/2022
hCoV-19/South Africa/CERI-KRISP-K039649/2022
hCoV-19/South Africa/NICD-N38280/2022
hCoV-19/South Africa/NICD-N38274/2022
hCoV-19/South Africa/CERI-KRISP-K039707/2022
hCoV-19/South Africa/CERI-KRISP-K039810/2022
hCoV-19/South Africa/CERI-KRISP-K037633/2022
hCoV-19/South Africa/NICD-N37127/2022
hCoV-19/South Africa/CERI-KRISP-K039699/2022
hCoV-19/South Africa/CERI-KRISP-K039677/2022
hCoV-19/South Africa/CERI-KRISP-K039662/2022
hCoV-19/South Africa/CERI-KRISP-K039711/2022
hCoV-19/South Africa/CERI-KRISP-K039682/2022
hCoV-19/South Africa/CERI-KRISP-K039818/2022
hCoV-19/South Africa/CERI-KRISP-K039828/2022
hCoV-19/South Africa/CERI-KRISP-K039750/2022
hCoV-19/South Africa/CERI-KRISP-K039634/2022
hCoV-19/South Africa/CERI-KRISP-K039655/2022
hCoV-19/South Africa/NICD-N38465/2022
hCoV-19/South Africa/CERI-KRISP-K039650/2022
hCoV-19/South Africa/NICD-N38543/2022
hCoV-19/South Africa/NICD-N38539/2022
hCoV-19/South Africa/NICD-N38537/2022
hCoV-19/South Africa/NICD-N38544/2022
hCoV-19/South Africa/NICD-N38538/2022
hCoV-19/South Africa/NICD-N38278/2022
hCoV-19/South Africa/NICD-N38282/2022
hCoV-19/South Africa/CERI-KRISP-K039391/2022
hCoV-19/South Africa/NICD-R03682/2022
hCoV-19/South Africa/CERI-KRISP-K039822/2022
hCoV-19/South Africa/CERI-KRISP-K039648/2022
hCoV-19/South Africa/CERI-KRISP-K039631/2022
hCoV-19/South Africa/CERI-KRISP-K039636/2022
hCoV-19/South Africa/CERI-KRISP-K039371/2022
hCoV-19/South Africa/CERI-KRISP-K039664/2022
hCoV-19/South Africa/CERI-KRISP-K039400/2022
hCoV-19/South Africa/CERI-KRISP-K040038/2022
hCoV-19/South Africa/CERI-KRISP-K040051/2022
hCoV-19/South Africa/CERI-KRISP-K040166/2022
hCoV-19/South Africa/CERI-KRISP-K040168/2022
hCoV-19/South Africa/CERI-KRISP-K040173/2022
hCoV-19/South Africa/CERI-KRISP-K040172/2022
hCoV-19/South Africa/CERI-KRISP-K040158/2022
hCoV-19/South Africa/CERI-KRISP-K040180/2022
hCoV-19/South Africa/CERI-KRISP-K040154/2022
hCoV-19/South Africa/NICD-N39845/2022
hCoV-19/South Africa/NICD-N39844/2022
hCoV-19/South Africa/NICD-N39840/2022
hCoV-19/South Africa/NICD-N39843/2022
hCoV-19/South Africa/NICD-N39835/2022
hCoV-19/South Africa/NICD-N39834/2022
hCoV-19/South Africa/CERI-KRISP-K040157/2022
hCoV-19/South Africa/CERI-KRISP-K040049/2022
hCoV-19/South Africa/CERI-KRISP-K040016/2022
hCoV-19/South Africa/CERI-KRISP-K040023/2022
hCoV-19/South Africa/CERI-KRISP-K040030/2022
hCoV-19/South Africa/CERI-KRISP-K040041/2022
hCoV-19/South Africa/CERI-KRISP-K040036/2022
hCoV-19/South Africa/CERI-KRISP-K040035/2022
hCoV-19/South Africa/NICD-N39013/2022
hCoV-19/South Africa/NICD-N39163/2022
hCoV-19/South Africa/NICD-N39154/2022
hCoV-19/South Africa/NICD-N39032/2022
hCoV-19/South Africa/NICD-N39079/2022
hCoV-19/South Africa/NICD-N39012/2022
hCoV-19/South Africa/NICD-N38935/2022
hCoV-19/South Africa/NICD-N39440/2022
hCoV-19/South Africa/NICD-N40226/2022
hCoV-19/South Africa/NICD-N40225/2022
hCoV-19/South Africa/NICD-N39264/2022
hCoV-19/South Africa/NICD-N40223/2022
hCoV-19/South Africa/NICD-N40218/2022
hCoV-19/South Africa/NICD-N40219/2022
hCoV-19/South Africa/CERI-KRISP-K039960/2022
hCoV-19/South Africa/CERI-KRISP-K038419/2022
hCoV-19/South Africa/CERI-KRISP-K037619/2022
hCoV-19/South Africa/CERI-KRISP-K038409/2022
hCoV-19/South Africa/NICD-N37124/2022

[corneliusroemer]

Nextclade places about a third in BA.4, a third in BA.5 and for another third it's not sure whether it's BA.4 or BA.5

Quality is a bit of an issue in these sequences, lots of reversions, so I guess that's what pangoLEARN struggles with

But given the large share in South Africa that's called BA.2 rather than BA.4/5 it's probably still worth looking into.

[corneliusroemer]

Ran the sequences through pangolin. Result is clear: Usher places correctly, but Scorpio overrides falsely.

We need something like Probable Omicron (BA.4/5-like) to solve this problem I think @aineniamh @AngieHinrichs @chrisruis

Or not let Probable Omicron (BA.2-like) override from BA.4/5 to BA.2

[dbtara]

@corneliusroemer what version of various components are you using, including Usher. Can uou post your commandline as well?

[willfischer]

We've observed this same phenomenon. Here's a near-whole-genome tree (raxml) showing both BA.4 and BA.5 clades with interspersed BA.2 calls. Inspection of the alignment confirms these as correctly placed sequences, but mis-labeled. Branches are colored by call: blue, BA.5; orange, BA.4, light green, BA.2. There are lots of green branches in the blue and orange clades, and even a few BA.3s (yellow).

BA4-and-BA5.lastz.for-phy.raxml.rooted.longnames.2022-05-01.rainbowtree.pdf

[corneliusroemer]

@dbtara

❯ pangolin gisaid_hcov-19_2022_05_02_14.fasta -t 10

❯ pangolin --all-versions
pangolin: 4.0.6
pangolin-data: 1.6
constellations: v0.1.9
scorpio: 0.3.17

But I first observed this problem on pango assignments that come from GISAID to covSpectrum. This seems to be a general problem, not restricted to the way I'm using pangolin.

Same thing happens with pangolin-data: 1.8

It's scorpio which is at the root of the problem here.

Same thing happens if I use pangoLEARN, most are correctly spotted as BA.4/5 but then Scorpio overrides.

[RajLABN]

Last week, I alerted GISAID CoVSurfer team about BA.4/5 assignment discrepancy.

Many of them do not have signature spike mutations (L452R & F486V).
@PeacockFlu spotted a dropout in the spike region.
Further analysis showed that this drop is observed in >10% of recently assigned BA4/5 sequences.

90% of them have N_P151S,NS7b_L11F,NSP1_K141del,NSP1_S142del,NSP1_F143del mutations (probable BA.4)

GISAID is considering to alert the submitters to conduct Sanger sequences or fill the gap in the missing Spike region with alternative primer sets.

[corneliusroemer]

This issue is still present becoming ever more severe. Around 50% of Swiss sequences are actually BA.4/5 but wrongly called BA.2 due to this Scorpio problem.

Now that BA.4/5 is becoming dominant in many countries this is raising questions. Can we not just deactivate Scorpio overwrite to BA.2?

@aineniamh

This query below shows actual BA.4/5 that are misclassified as BA.2 or other:

The query was written by @shay671, and I have verified that it is both sensitive and specific (for current BA.4/5).

https://cov-spectrum.org/explore/World/AllSamples/Past3M/variants/international-comparison?variantQuery=%28%21BA.4*%29+%26+%28%21BA.5*%29+%26+%28%28%5B1-of%3A+G27788T%2C+C28724T%2C+%5B1-of%3A+A686-%2CA687-%2CG688-%2CT689-%2CC690-%2CA691-%2CT692-%2CT693-%2CT694-%5D%5D%26%5B3-of%3A+G27788T%2C+C28724T%2C+%5B1-of%3AA686-%2CA687-%2CG688-%2CT689-%2CC690-%2CA691-%2CT692-%2CT693-%2CT694-%5D%2CG12160A%2CT22917G%2CT23018G%5D%26%21%5B2-of%3AC9866T%2CA23040G%5D%26%5B3-of%3AC26858T%2CA27259C%2CG27382C%2CA27383T%2CT27384C%5D%26%5B33-of%3AC241T%2CG4184A%2CA9424G%2CC14408T%2CC23525T%2CA24424T%2CC10198T%2CC17410T%2CA18163G%2CA22786C%2CG27382C%2CC3037T%2CC12880T%2CT22200G%2CA28271T%2CC10449A%2CA27383T%2CA23055G%2CA23063T%2CT22679C%2CC10029T%2CC22674T%2CG22992A%2CT27384C%2CC15714T%2CG22775A%2CC21618T%2CT24469A%2CC26060T%2CC26858T%2CC27807T%2CC28311T%2CA29510C%2CT670G%2CC2790T%2CC9344T%2CA20055G%2CC22686T%2CC22995A%2CA23013C%2CG23948T%2CG28882A%2CC4321T%2CC23604A%2CC25584T%2CG28881A%2CA22688G%2CA23403G%2CT23075C%2CC23854A%2CC26270T%2CG26709A%2CC9534T%2CG22578A%2CT22882G%2CC25000T%2CC26577G%2CG10447A%2CG22813T%2CT23599G%2CG28883C%2CC19955T%2CG21987A%2CA27259C%2C%5B1-of%3AT11288-%2CC11289-%2CT11290-%2CG11291-%2CG11292-%2CT11293-%2CT11294-%2CT11295-%2CT11296-%5D%2C%5B1-of%3AT21633-%2CA21634-%2CC21635-%2CC21636-%2CC21637-%2CC21638-%2CC21639-%2CT21640-%2CG21641-%2CT21765-%2CA21766-%2CC21767-%2CA21768-%2CT21769-%2CG21770-%5D%2C%5B1-of%3AG28362-%2CA28363-%2CG28364-%2CA28365-%2CA28366-%2CC28367-%2CG28368-%2CC28369-%2CA28370-%5D%5D%29+%7C+%28%5B1-of%3A+G26529A%2CC27889T%5D%26%5B3-of%3A+G12160A%2CT22917G%2CT23018G%2CG26529A%2CC27889T%5D%26%21%5B2-of%3A+C9866T%2CA23040G%2CC26858T%2CA27259C%2CG27382C%2CA27383T%2CT27384C%5D%26%5B30-of%3A+C10449A%2CG21987A%2CT22882G%2CC26270T%2CA28271T%2CG22578A%2CA23403G%2CC25000T%2CC12880T%2CC28311T%2CC15714T%2CC17410T%2CG22775A%2CA22786C%2CG4184A%2CA9424G%2CA20055G%2CC22995A%2CA23063T%2CG23948T%2CC22686T%2CG22992A%2CC23525T%2CG28881A%2CT670G%2CA24424T%2CC2790T%2CA23055G%2CC9534T%2CC19955T%2CT23075C%2CT23599G%2CC26577G%2CG28883C%2CA29510C%2CC23854A%2CT24469A%2CC26060T%2CG26709A%2CG10447A%2CT22200G%2CC3037T%2CC21618T%2CA22688G%2CG22813T%2CC27807T%2CC241T%2CC10198T%2CA18163G%2CC25584T%2CC4321T%2CC9344T%2CT22679C%2CA23013C%2CG28882A%2CC10029T%2CC14408T%2CC22674T%2CC23604A%2C%5B1-of%3A+T11288-%2CC11289-%2CT11290-%2CG11291-%2CG11292-%2CT11293-%2CT11294-%2CT11295-%2CT11296-%5D%2C%5B1-of%3A+T21633-%2CA21634-%2CC21635-%2CC21636-%2CC21637-%2CC21638-%2CC21639-%2CT21640-%2CG21641-%2CT21765-%2CA21766-%2CC21767-%2CA21768-%2CT21769-%2CG21770-%5D%2C%5B1-of%3AG28362-%2CA28363-%2CG28364-%2CA28365-%2CA28366-%2CC28367-%2CG28368-%2CC28369-%2CA28370-%5D%5D%29%29&

[AngieHinrichs]

pangolin --skip-scorpio (default UShER mode, without scorpio overriding results) can correctly identify those sequences as BA.4 despite the dropout, based on the other mutations in the sequences.

It has been over a month since the release of pangolin v4.0.6, and two months since the release of v4.0 with UShER as the default mode, with an optional assignment cache to speed up lineage assignment for pre-existing sequences, but GISAID is still using pangoLEARN for lineage assignments. GISAID may be unaware of the increased accuracy and stability of UShER mode, and although UShER is significantly slower than pangoLEARN, the assignment cache should help to make up for that. If GISAID hears from multiple community members (hint hint 😃 🙏) that they would prefer UShER assignments,and until scorpio/constellations are improved to better handle BA.4 and BA.5, --skip-scorpio, maybe they will update their methods.