bx-python error working with long chromosomes in plants

[brettChapman]

Hi

I've been getting the following error when working with plant genomes. I've also flagged the error with bx-python (bxlab/bx-python#67):

inner_distance.py \
      -i Root_1.markdup.sorted.bam \
      -r RGT_Planet.bed \
      -o Root_1 \
       \
      > stdout.txt
  head -n 2 stdout.txt > Root_1.inner_distance_mean.txt
  
  inner_distance.py --version | sed -e "s/inner_distance.py //g" > rseqc.version.txt

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: Skipping mount /usr/local/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
  Get exon regions from RGT_Planet.bed ...
  Traceback (most recent call last):
    File "/usr/local/bin/inner_distance.py", line 95, in <module>
      main()
    File "/usr/local/bin/inner_distance.py", line 87, in main
      obj.mRNA_inner_distance(outfile=options.output_prefix,low_bound=options.lower_bound_size,up_bound=options.upper_bound_size,step=options.step_size,refbed=options.ref_gene,sample_size=options.sampleSize, q_cut = options.map_qual)
    File "/usr/local/lib/python3.7/site-packages/qcmodule/SAM.py", line 3582, in mRNA_inner_distance
      exon_bitsets = binned_bitsets_from_list(ref_exons)
    File "/usr/local/lib/python3.7/site-packages/bx/bitset_builders.py", line 143, in binned_bitsets_from_list
      last_bitset.set_range( start, end - start )
    File "lib/bx/bitset.pyx", line 216, in bx.bitset.BinnedBitSet.set_range
    File "lib/bx/bitset.pyx", line 184, in bx.bitset.bb_check_range_count
    File "lib/bx/bitset.pyx", line 180, in bx.bitset.bb_check_index
  IndexError: 537013395 is larger than the size of this BitSet (536870912).

[pinin4fjords]

Thanks for reporting this issue! This is a known limitation in the underlying bx-python library that affects genomes with chromosomes exceeding ~537 million bases (commonly seen in plant genomes like wheat, barley, etc.).

Root Cause

The issue stems from a hard-coded BitSet size limit in bx-python (see bxlab/bx-python#67). Unfortunately, this upstream issue has remained unresolved since 2020, and there's no fix available in bx-python at this time.

Solution

Since we cannot fix the underlying library limitation, I've created PR #1624 to document this limitation and provide a clear workaround in our usage documentation.

Workaround: You can exclude the inner_distance module from RSeQC analysis by using:

--rseqc_modules 'bam_stat,infer_experiment,junction_annotation,junction_saturation,read_distribution,read_duplication'

This removes inner_distance from the default list while retaining all other quality control metrics. Note that the inner_distance metric is only relevant for paired-end data and provides information about fragment size distribution.

Once PR #1624 is merged, this workaround will be clearly documented in the usage guide for users working with large genomes.

I'm closing this issue as there's no actionable fix we can implement in the pipeline itself, but the documentation will help users encountering this limitation.