ThreadDataloader GIL

[hw-ju]

Hi! In the situation where we cache all deterministic transforms in GPU memory and then use ThreadDataloader to perform lightweight non-deterministic GPU transforms, does the separate thread used by ThreadDataloader release GIL so that it doesn't blocker other threads which are running training? Thanks!

[Nic-Ma]

Hi @ericspod ,

Could you please help share some comments here?

Thanks in advance.

[ericspod]

The ThreadDataLoader class is using Python threads which still need to have the GIL locked to execute. If all your transforms are pure Python then this will interfere with the main (training) thread, however typically transforms are using Numpy, Pytorch, or some other library using compiled code which releases the GIL. The purpose of this class is to take advantage of this and of the fact that training is mostly done in compiled code, despite the GIL it will in certain situations allow overlapping of training and batch queuing. If either training time or batch queue time a much longer than the other there will be less benefit however.

[hw-ju]

Thanks a lot @ericspod @Nic-Ma! Crystal clear explanation! :D