Inference supports up to 12 images #17
Description
Thank you very much for your open-source work, the results are very good!
However, due to the size of the model itself, there are fewer images for one-time inference, and it is impossible to infer more than 12 images on single 4090, which will limit the application of this work in a wider range of scenarios. Is this due to the hard coding in the model that supports up to 12 images, even if my video memory still has some free space?
I noticed that there are some implementation methods for multi card inference in utilis/dist. py, but it doesn't seem to be complete. I hope the author can open source this part of the multi card parallel reasoning method, which will take this work to the next level!
Thank you again to the author for opening up such a great work!