Should I split time-series Input for inference

[townsag]

Good Afternoon All,

Context:
I have trained a CNN-LSTM-CTC model for the nanopore basecaller task. It maps the raw signal created by a strand of DNA moving through a nanopore to the labels representing the nucleotides in that DNA strand. I trained the model on equal-sized non-overlapping chunks of the raw signal. My intention was to trade the information lost by truncating the signal into chunks for faster training time, because the chunks of equal length could be put in a batch. Furthermore, the apply function would only have to be compiled for one bach and chunk size. I was able to evaluate each chunk because the label sequence was aligned to the raw signal, allowing me to pick out the sub-string of the label sequence associated with that chunk. The training batches were of shape (batch=64, length=2000, channel=1) with each element in the bach having a label sequence of ~150 characters.

Thank you for reading that other part, here is my question:
Now that the model is trained, I'm uncertain about the inference process. Should I split new raw signal examples into overlapping reads and use pairwise sequence alignment to assemble the complete nucleotide sequence? Alternatively, should I feed the entire raw signal into the apply function?

Some raw signal reads are around ~400,000 in length, corresponding to a nucleotide sequence output of 50,000 characters.
How do I measure what length of input signal will exceed the memory of my GPU?

If I chunk the input into equal sized pieces then the kernel will only have to be recompiled for the last chunk (because the raw signal wouldn't divide evenly by chunk size). So most of the computation will happen using the same compiled kernel. If I don't chunk the input then the kernel will have to be recompiled every time because none of the raw signal inputs are of the same length. However, I wouldn't have the overhead of reconstructing the complete nucleotide sequence from the pairwise alignments.

This task is very similar to automatic speech recognition. How do people handle long audio recordings?

Thank you for your time. This is my first time posting on this forum so I hope I got the format right and didn't break any rules. I am still learning jax/flax so I would be happy to hear any corrections in my thinking about jit compilation.