rmap running time

[bdecato]

Trying to map 4 million reads, every single job of mine has been failing at its 15 hour walltime for this new dataset. The only thing that has changed is that I am using the new version of rmap. Did this substantially slow it down? Do other people use a longer walltime?

[andrewdavidsmith]

I think you have just done a walltime test ;-)

The only thing that was supposed to have changed is that now seek is used, rather than reading every line, in order to find the right reads.

However, there was some strange confusion where Liz suggested changing the meaning of the command line options...

Andrew

On Nov 22, 2014, at 5:36 PM, Benjamin Decato notifications@github.com wrote:

Trying to map 4 million reads, every single job of mine has been failing at its 15 hour walltime for this new dataset. The only thing that has changed is that I am using the new version of rmap. Did this substantially slow it down? Do other people use a longer walltime?

—
Reply to this email directly or view it on GitHub.

[pjuren]

I assume there were some regression tests to make sure nothing got screwed up in terms of output? Could run those on the new and old version and benchmark/profile runtime. Sometimes this kind of thing happens because an ampersand gets omitted somewhere by accident, or something similar.

[xjlizji]

I tested the output and the new one is same to the old one. But I haven't ran the speed testings - I'm going to do it soon. Actually the only change in rmap is the way to locate the first read - so the speed difference is thought to only happen where the reads are far away from the beginning of the file.
I didn't meet this issue with rmapbs-pe on my data set. It's also possible that the sequences themselves are difficult to be mapped. I will run old rmap on the fastq file from your data set which was mapped slowly.