encounter the memory allocation problem

[shuliliu]

Hi,

I currently try to run SViper to polish the SVs generated by cuteSV (have changed the vcf format) in both CLR pacBio reads and CCS PacBio reads. I successfully got the results in SVs from CLR PacBio reads, but failed (See below) after running for several minutes and got several polished SVs using SVs from CCS PacBio reads.

~/SViper/extern/seqan/include/seqan/basic/basic_exception.h:363 FAILED! (Uncaught exception of type std::bad_alloc: std::bad_alloc)

stack trace:
0 [0x41891e] ~/bin/SViper/build/sviper()
1 [0x407782] ~/bin/SViper/build/sviper()
2 [0x7ff21669e006] /soft/compiler/gcc/gcc-9.2.0/lib64/libstdc++.so.6(+0xad006)
3 [0x7ff21669d109] /soft/compiler/gcc/gcc-9.2.0/lib64/libstdc++.so.6(+0xac109)
4 [0x7ff21669da34] __gxx_personality_v0 + 0x264
5 [0x7ff215e31463] /soft/compiler/gcc/gcc-9.2.0/lib64/libgcc_s.so.1(+0x10463)
6 [0x7ff215e31cc6] _Unwind_Resume + 0x126
7 [0x40895c] ~/bin/SViper/build/sviper()
8 [0x413e44] ~/bin/SViper/build/sviper()
9 [0x7ff21604b4a2] GOMP_parallel + 0x42
10 [0x4091da] ~/bin/SViper/build/sviper()
11 [0x7ff2158626a3] __libc_start_main + 0xf3
12 [0x40976e] ~/bin/SViper/build/sviper()

/var/spool/slurmd/job634082/slurm_script: line 39: 2250216 Aborted (core dumped) sviper -s ${bam_sdir}/${bam_short} -l ${bam_ldir}/${bam_long} -r ${ref_dir}/hs37d5.fa -c ${wk_dir}/${out_dir}/reformat.${out_dir}_cuteSV.vcf -o ${wk_dir}/${out_dir}/${out_dir}.polished_variants -t 1 -x ${depth[index]} --output-polished-bam

Do you have any ideas where's the issue?

Thanks,

Shuli

[smehringer]

Hi @shuliliu,

thanks for your interest in SViper!
So at a first glance this looks like a memory issue as you already mentioned.
Multi-threading increases the memory consumption but I see that you already only use one thread.

What is your available main memory (RAM)?
What is your short read coverage?

If possible, try running your script on a machine with more main memory and see if it also crashes.
If the error persists it might also be a bug, but than I would need to debug SViper with your data. Is there a chance you can make them available to me?

Best,
Svenja

[shuliliu]

Hi Svenja ,

Thank you for your reply. I used 4 coverage short reads (for testing) and 400G memory, and didn't work.
Yesterday, I tried the long read bam file generated by pbmm2 instead of minimap2, and it worked. So the issue seems located in the long read bam file.

1. I checked these two bam file in detail and only saw two big differences:
   bam files for pbmm2:
   cigar string: X/= cigars instead of M
   bam files for minimap2:
   cigar string: M instead of X/= cigars.
   MD string: I added a MD string when running minimap2.

2. I also checked the bam file generated from CLR reads and CCS reads using (minimap2). and saw one difference:
   bam files from CLR reads: base quality is missing (Originally missing in fastq file); bam file from CCS reads: with base quality.

I don't know whether above would be the things for the issue. I'll send you the data later if you think it still need to check more.

Best,

Shuli

[smehringer]

Hi Shuli,

thanks for looking into it deeper. Yes your memory is sufficient big time, it looks like a bug.

To 1. I'm pretty sure that I handled X/= euqallly to M. And anyway I would suspect the error when handling X/=.
To 2. This seems more promising. I took a quick look into the code and I did not explicitly handle missing base qualities so I need to dig deeper to check if there is a problem.

It has been some time since I have touched the code though, so I would appreciate to get the data because debugging is easier that way. If you know the region where the error happens I am happy to only take a subregion of the data, too.

Best,
Svenja