Are there any plans to implement concurrent LoRa inference with multiple adapters (such as S-lora)?

[SamGalanakis]

Would be very useful and there doesn't seem to be a flexible implementation of this yet.

[BenjaminBossan]

Do you mean as suggested in #903? If yes, there are plans, hopefully we can tackle it soon. But note that SLoRA has a bunch of specialized optimizations that we cannot do in PEFT, as we want to support a very broad range of models and adapter types.

[SamGalanakis]

Ah my bad I missed that one. So that will allow parallel inference, any rough idea how pefromant it will be throughput and memory wise?

[BenjaminBossan]

parallel inference

If, by that, you mean different adapters in the same batch, then yes.

any rough idea how pefromant it will be throughput and memory wise

Sorry, we don't have any numbers yet, let's check once the PR is finalized.