multi-gpu inference

[ChunYuNoBugs]

Hi there! I really appreciate the fantastic work you do. I’d like to know how I can upgrade a single-GPU setup to support multiple GPUs. Do you have any recommendations?

[rex-29]

Hi there! I really appreciate the fantastic work you do. I’d like to know how I can upgrade a single-GPU setup to support multiple GPUs. Do you have any recommendations?

cache-dit support it now