wf-human-variation produces TypeError: unsupported operand type(s) for /: 'NoneType' and 'NoneType' during data normalisation.

[vmkalbskopf]

While running the wf-human-variation pipeline, ont-spectre produced an error implying that either the vcf file and reference files were empty or that the script did not load them correctly. I have attached the relevant files (but only a small sample of the vcf file as it is sensitive human data). The reference is the standard GRCh38.p14 file, and I checked it manually myself.
Should this bug be filed on the wf-human-variation repo instead?

spectre::INFO> Spectre version: 0.2.2
spectre::INFO> Spectre enabled
spectre::INFO> Extraction of metadata is activated
spectre::INFO> Extracting metadata from resource: data/grch38_metadata.mdr
spectre::INFO> Using blacklist from resource: data/grch38_blacklist_0.3.bed
spectre::INFO> Spectre calculating for: work/6a/95c8185412eee91e54c8bc26ea21af/readstats and bin size: 1000
spectre::INFO> Data normalization and outlier removal (right tail)
spectre::INFO> Parsing VCF to AF freqs file
spectre::INFO> Thresholds for coverage, lower (Q10.0): None, upper(Q90.0): None
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/custflow/epi2meuser/conda/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/opt/custflow/epi2meuser/conda/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/opt/custflow/epi2meuser/conda/lib/python3.8/site-packages/spectre/main.py", line 120, in outside_spectre_worker
    worker.cnv_call()
  File "/opt/custflow/epi2meuser/conda/lib/python3.8/site-packages/spectre/spectreCNV.py", line 101, in cnv_call
    self.cnv_analysis.data_normalization()
  File "/opt/custflow/epi2meuser/conda/lib/python3.8/site-packages/spectre/analysis/analysis.py", line 172, in data_normalization
    self.lower_2n_threshold = coverage_lower_threshold / genome_median * self.ploidy
TypeError: unsupported operand type(s) for /: 'NoneType' and 'NoneType'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/custflow/epi2meuser/conda/bin/spectre", line 10, in <module>
    sys.exit(run_main())
  File "/opt/custflow/epi2meuser/conda/lib/python3.8/site-packages/spectre/main.py", line 506, in run_main
    spectre_run.spectre_exe()
  File "/opt/custflow/epi2meuser/conda/lib/python3.8/site-packages/spectre/main.py", line 274, in spectre_exe
    results = pool.map(outside_spectre_worker, tuple(spectre_instructions))
  File "/opt/custflow/epi2meuser/conda/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/custflow/epi2meuser/conda/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
TypeError: unsupported operand type(s) for /: 'NoneType' and 'NoneType'

B_22064.thresholds.bed.txt

800_lines.vcf.txt

B_22064.regions.filt.bed.txt

B_22064.mosdepth.global.dist.txt
B_22064.mosdepth.summary.txt

[snurk]

Dear @vmkalbskopf, thank you very much for the bug report!
While we are looking into it, can you please see if discussion in epi2me-labs/wf-human-variation#198 is relevant, I think this was the only time we saw this error message before.

[vmkalbskopf]

From what I can see in that bug report, spectre should not be used when the sequencing data was generated using adaptive sampling. However, I am using whole genome sequencing, so I don't think it is relevant. But that is an interesting coincidence...

I am going to try running QDNAseq in the meantime.

[richard-dnastack]

Was there any follow up with this? I'm also running into the same issue while using the demo data provided here:
https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-human-variation/hg002%2bmods.v1/wf-human-variation-demo.tar.gz

[jimmykhchiu]

I just hit the same issue. After tracing the source codes and examining some intermediate data, the reason for this in my case is the bin size input for mosdepth (--by) does not match with the bin size input (--bin-size) for spectre. Spectre creates an AF dataframe from the SNV input VCF, and another coverage dataframe from the mosdepth coverage file, then it left joins the coverage dataframe using the AF dataframe on chromosome and start position. This start position is a multiple of the bin-size. Initially I ran mosdepth using a target BED file for the --by parameter instead of an integer bin-size, so the left joint produced an empty coverage table due to the start position mismatch (e.g. 1,424,875 vs 1,424,000). The coverage thresholds are therefore of NoneType. I then set the bin size to 1000 for both mosdepth and spectre, and this error was gone.