How to rescue the data form the assemble step.

[fookloy]

Hi, here i am again for another question.

To better understand how Mixcr works, i am trying to separate every step to handle my data, and learn how this smart software think and do in these cases.

I've exported the Alignments.tsv using exportAlignments, then found there are about 4,792,748 reads which were successfully merged and aligned in align with

mixcr align --preset generic-amplicon --species human --rna --rigid-left-alignment-boundary --rigid-right-alignment-boundary C -OmergerParameters.minimalOverlap=10 -OmergerParameters.minimalIdentity=0.9 HB-HV_L1_1.fq.gz HB-HV_L1_2.fq.gz HB-HV_L1.vdjca &.

although there are several sequences that were not merged (in some sequences, i found "," between full R1 and R2), about 43,895, the proportion is low, it is ok to throw them away.

Then, i assembled them using

mixcr assemble --assemble-clonotypes-by "{CDR1Begin:FR4End}" alignments.vdjca clones_mindel.clns &

mixcr exportClones --chains IGH clones_mindel.clns clones_mindel_IGH.tsv & (IGH account for 99%)

After summing the total sequences in .tsv file → readcount / proportion = totalreads , i found that there are only 2,659,014 reads left in the .clns file, i understand some sequences will be thrown owing to different reasons like low quality or so., but for what reason there were almost a half of sequences were thrown away ? or what can i do to rescue them by adjusting the parameters?

Many thanks for your help!

[mizraelson]

Hi, can you export and share the reports for the align and assemble steps?

[fookloy]

Of course
here is the aligning report:

====================== report: align ======================
====================== report: align ======================
Total sequencing reads: 4927224
Successfully aligned reads: 4792885 (97.27%)
Coverage (percent of successfully aligned):
  VDJRegion: 97.83%
  CDR1_TO_FR4: 98.24%
  FR2_TO_FR4: 98.23%
  CDR2_TO_FR4: 98.36%
  FR3_TO_FR4: 99.28%
  CDR3: 99.5%
Alignment failed: no hits (not TCR/IG?): 33561 (0.68%)
Alignment failed: absence of V hits: 144 (0%)
Alignment failed: absence of J hits: 79992 (1.62%)
Alignment failed: no target with both V and J alignments: 20642 (0.42%)
Overlapped: 4828572 (98%)
Overlapped and aligned: 4748854 (96.38%)
Overlapped and not aligned: 79718 (1.62%)
Alignment-aided overlaps, percent of overlapped and aligned: 35510 (0.75%)
No CDR3 parts alignments, percent of successfully aligned: 225 (0%)
Partial aligned reads, percent of successfully aligned: 23596 (0.49%)
V gene chimeras: 4995 (0.1%)
J gene chimeras: 1 (0%)
Paired-end alignment conflicts eliminated: 32087 (0.65%)
Realigned with forced non-floating bound: 268324 (5.45%)
Realigned with forced non-floating right bound in left read: 24195 (0.49%)
Realigned with forced non-floating left bound in right read: 24195 (0.49%)
IGH chains: 4790351 (99.95%)
IGH non-functional: 75111 (1.57%)
IGK chains: 14 (0%)
IGK non-functional: 0 (0%)
IGL chains: 324 (0.01%)
IGL non-functional: 5 (1.54%)
TRD chains: 1632 (0.03%)
TRD non-functional: 94 (5.76%)
TRG chains: 564 (0.01%)
TRG non-functional: 194 (34.4%)
Trimming report:
  R1 reads trimmed left: 597 (0.01%)
  R1 reads trimmed right: 51 (0%)
  Average R1 nucleotides trimmed left: 2.062013011789194E-4
  Average R1 nucleotides trimmed right: 6.437702040743429E-4
  R2 reads trimmed left: 19 (0%)
  R2 reads trimmed right: 88 (0%)
  Average R2 nucleotides trimmed left: 4.140262346505862E-4
  Average R2 nucleotides trimmed right: 4.1280851043102565E-4

here is the assemble report:

===================== report: assemble =====================
===================== report: assemble =====================
Final clonotype count: 706767
Reads used in clonotypes, percent of total: 2659067 (53.97%)
Average number of reads per clonotype: 3.76
Reads dropped due to the lack of a clone sequence, percent of total: 68726 (1.39%)
Reads dropped due to a too short clonal sequence, percent of total: 0 (0%)
Reads dropped due to low quality, percent of total: 0 (0%)
Reads dropped due to failed mapping, percent of total: 2064956 (41.91%)
Reads dropped with low quality clones, percent of total: 0 (0%)
Aligned reads processed: 4724023
Reads used in clonotypes before clustering, percent of total: 2659067 (53.97%)
Number of reads used as a core, percent of used: 2465154 (92.71%)
Mapped low quality reads, percent of used: 193913 (7.29%)
Reads clustered in PCR error correction, percent of used: 0 (0%)
Reads pre-clustered due to the similar VJC-lists, percent of used: 394 (0.01%)
Clonotypes dropped as low quality: 0
Clonotypes eliminated by PCR error correction: 0
Clonotypes pre-clustered due to the similar VJC-lists: 276
Clones dropped in post filtering: 0 (0%)
Reads dropped in post filtering: 0.0 (0%)
Alignments filtered by tag prefix: 0 (0%)
IGH chains: 706715 (99.99%)
IGH non-functional: 9853 (1.39%)
IGL chains: 51 (0.01%)
IGL non-functional: 1 (1.96%)
TRG chains: 1 (0%)
TRG non-functional: 0 (0%)

Reads dropped due to failed mapping, percent of total: 2064956 (41.91%)

After the core clonotypes are built, MiXCR runs mapping procedure that processes records deferred on the previous step. Mapping is aimed at rescuing of quantitative information from low quality reads. For this, each deferred record is mapped onto already assembled clonotypes: if there is a fuzzy match, then this record will be aggregated by the corresponding clonotype; in case of several matched clonotypes, a single one will be randomly chosen with weights equal to clonotype counts. If no matches found, the record will be finally dropped.

Could these to-be dropped sequences stay, and keep all alignments information in .clns?

[mizraelson]

From the output Reads dropped due to failed mapping, percent of total: 2064956 (41.91%), it's clear that the primary issue lies in the data quality. Approximately 40% of the reads, characterized by the presence low-quality nucleotides, were set aside before assembly and subsequently failed to merge into any clones. This step is part of sequencing error correction.

If you wish to bypass this quality check, you can disable the mapping by using the following command:
mixcr assemble -ObadQualityThreshold=0 alignments.vdjca output.clns

This command instructs MiXCR to consider all reads as high-quality.

However, it's important to note that doing so might lead to a significant increase in artificial diversity within the data. The default setting for the badQualityThreshold parameter is 15. Adjusting this threshold to 0 could result in the inclusion of low-quality reads, potentially impacting the overall accuracy of the assembly.

[fookloy]

HI, new week is here, hope everything is going well with your days and works.

I ran the following command

java -Xmx500g -jar /mnt/C_NGS01/Fred/antibody/mixcr.jar assemble --assemble-clonotypes-by "{CDR1Begin:FR4End}" -ObadQualityThreshold=0 -OcloneClusteringParameters=null alignments.vdjca clones_mindel.clns &
turning the clustering and badqualitythreshold off for comparison using different parameters, but it shows that about 35% sequences were still dropped, i assume that the drop should be lower significantly, but just 6%.

===================== report: assemble =====================
Analysis time: 67.09m
Final clonotype count: 912141
Reads used in clonotypes, percent of total: 2980901 (60.5%)
Average number of reads per clonotype: 3.27
Reads dropped due to the lack of a clone sequence, percent of total: 68726 (1.39%)
Reads dropped due to a too short clonal sequence, percent of total: 0 (0%)
Reads dropped due to low quality, percent of total: 0 (0%)
Reads dropped due to failed mapping, percent of total: 0 (0%)
Reads dropped with low quality clones, percent of total: 1743122 (35.38%)
Aligned reads processed: 4724023
Reads used in clonotypes before clustering, percent of total: 2980901 (60.5%)
Number of reads used as a core, percent of used: 2980901 (100%)
Mapped low quality reads, percent of used: 0 (0%)
Reads clustered in PCR error correction, percent of used: 0 (0%)
Reads pre-clustered due to the similar VJC-lists, percent of used: 1310 (0.04%)
Clonotypes dropped as low quality: 1704752
Clonotypes eliminated by PCR error correction: 0
Clonotypes pre-clustered due to the similar VJC-lists: 1117
Clones dropped in post filtering: 0 (0%)
Reads dropped in post filtering: 0.0 (0%)
Alignments filtered by tag prefix: 0 (0%)
TRG chains: 1 (0%)
TRG non-functional: 0 (0%)
IGH chains: 912088 (99.99%)
IGH non-functional: 12960 (1.42%)
IGL chains: 52 (0.01%)
IGL non-functional: 1 (1.92%)

Maybe there are any other parameters concerning to this result?

[mizraelson]

So the alignments now pass the initial filtering, but the assembled clones do not pass the final quality assessment because many of them now consist of saved low-quality reads. To disable this check as well, you can add -OminimalQuality=0

[fookloy]

-OminimalQuality=0 seems the key parameter which causes the high dropped rate, and i could control the dropped rate by adjusting it in some cases.

===================== report: assemble =====================
===================== report: assemble =====================
Final clonotype count: 2616893
Reads used in clonotypes, percent of total: 4724023 (95.88%)
Average number of reads per clonotype: 1.81
Reads dropped due to the lack of a clone sequence, percent of total: 68726 (1.39%)
Reads dropped due to a too short clonal sequence, percent of total: 0 (0%)
Reads dropped due to low quality, percent of total: 0 (0%)
Reads dropped due to failed mapping, percent of total: 0 (0%)
Reads dropped with low quality clones, percent of total: 0 (0%)
Aligned reads processed: 4724023
Reads used in clonotypes before clustering, percent of total: 4724023 (95.88%)
Number of reads used as a core, percent of used: 4724023 (100%)
Mapped low quality reads, percent of used: 0 (0%)
Reads clustered in PCR error correction, percent of used: 0 (0%)
Reads pre-clustered due to the similar VJC-lists, percent of used: 1310 (0.03%)
Clonotypes dropped as low quality: 0
Clonotypes eliminated by PCR error correction: 0
Clonotypes pre-clustered due to the similar VJC-lists: 1117
Clones dropped in post filtering: 0 (0%)
Reads dropped in post filtering: 0.0 (0%)
Alignments filtered by tag prefix: 0 (0%)
IGH chains: 2616587 (99.99%)
IGH non-functional: 59250 (2.26%)
IGK chains: 5 (0%)
IGK non-functional: 0 (0%)
IGL chains: 299 (0.01%)
IGL non-functional: 5 (1.67%)
TRG chains: 2 (0%)
TRG non-functional: 0 (0%)

Sincerely thank you for your help!

[mizraelson]

Keep in mind, that now when all of the quality filters are off you have a lot of potential noise in the data and the sequences are not that reliable.