Dataloader for large annotation file

[jihwanp]

Hi I have a question about how to load large annotation file efficiently.
Im now using about 100GB annotation file and when I try to register all those annotations in CPU memory, it uses more than 100GB memory (about >200) and raise memory error:
numpy.core._exceptions.MemoryError: Unable to allocate 106. GiB for an array with shape (113830834151,) and data type uint8

I think there might be solution for efficiently handle large dataset but can not find it at detectron2.
Any ideas?
Thanks