Skip to content

Using records iterator instead of read interface results in invalid memory access when reading a bam file and sending bam records over a thread #479

@uray-scalebio

Description

@uray-scalebio

When using the records iterator https://docs.rs/rust-htslib/latest/rust_htslib/bam/trait.Read.html#tymethod.records I get a memory access error. The error goes away when using the read interface https://docs.rs/rust-htslib/latest/rust_htslib/bam/trait.Read.html#tymethod.read.

The error is random and I cannot reproduce it with a small code snippet. Happy to share the entire codebase if need be. The relevant code for the threading and sending batches of records over the queue is below.

pub struct BamReaderThread {
    thread: std::thread::JoinHandle<()>,
    pub rx: mpsc::Receiver<Vec<Record>>,
}

impl BamReaderThread {
    pub fn new(mut bam_reader: Reader, read_processing_batch_size: usize, buffer_size: usize) -> Self {
        let (tx, rx) = mpsc::sync_channel(buffer_size);
        let mut batch = Vec::with_capacity(read_processing_batch_size);
        let thread = std::thread::spawn(move || {
            for record in bam_reader.records() {
                batch.push(record.unwrap());
                if batch.len() == read_processing_batch_size {
                    // take ensures that batch is cleared after sending, thus making it reusable
                    if tx.send(std::mem::take(&mut batch)).is_err() {
                        // Error in the receiver thread; shutting down
                        return;
                    }
                }
            }
            if !batch.is_empty() {
                // Can ignore error, since thread is done
                let _ = tx.send(batch);
            }
        });
        Self {
            thread,
            rx,
        }
    }

    pub fn finish(self) {
        drop(self.rx);
        self.thread.join().expect("Error closing BamReaderThread");
    }
}

Here is the rust backtrace

thread '<unnamed>' panicked at library/core/src/panicking.rs:218:5:
unsafe precondition(s) violated: hint::assert_unchecked must never be called when the condition is false

This indicates a bug in the program. This Undefined Behavior check is optional, and cannot be relied on for safety.
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/std/src/panicking.rs:697:5
   1: core::panicking::panic_nounwind_fmt::runtime
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/panicking.rs:117:22
   2: core::panicking::panic_nounwind_fmt
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/intrinsics/mod.rs:3241:9
   3: core::panicking::panic_nounwind
             at /rustc/17067e9ac6d7ecb70e50f92c1944e545188d2359/library/core/src/panicking.rs:218:5
   4: core::hint::assert_unchecked::precondition_check
             at /home/utsab/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ub_checks.rs:68:21
   5: core::hint::assert_unchecked
             at /home/utsab/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ub_checks.rs:75:17
   6: alloc::rc::RcInnerPtr::inc_strong
             at /home/utsab/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/rc.rs:3567:13
   7: <alloc::rc::Rc<T,A> as core::clone::Clone>::clone
             at /home/utsab/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/rc.rs:2337:13
   8: <rust_htslib::bam::Reader as rust_htslib::bam::Read>::read
             at /home/utsab/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/rust-htslib-0.50.0/src/bam/mod.rs:386:35
   9: <rust_htslib::bam::Records<R> as core::iter::traits::iterator::Iterator>::next
             at /home/utsab/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/rust-htslib-0.50.0/src/bam/mod.rs:1258:15
  10: seq_tag_finder::bam::BamReaderThread::new::{{closure}}
             at /data/seqTagFinder/debug/seqTagFinder/src/bam.rs:34:27

I am on rust-htslib 0.50

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions