How to train multiple LoRA adapters on the same base model concurrently.

[rajat95]

I want to train 2 LoRA models in conjunction on my dataset. I don't want gradients from one model impact the other. However, since the base model is the same I am confused if just setting adapter_name like below would achieve this purpose?

        modelA = get_peft_model(model, lora_config, adapter_name="A")
        modelB = get_peft_model(model, profile_lora_config, adapter_name="B")

And then in the model forward

        outputA = modelA.forward(inputsA)
        outputB = modelB.forward(inputsB)

[BenjaminBossan]

Hi, this is not the way. Could you try something like this instead?

model = get_peft_model(model, lora_config, adapter_name="A")
# add 2nd adapter, "A" is still the active adapter
model.add_adapter("B", profile_lora_config)
# now train adapter "A"
...
# switch to adapter "B" and train it
model.set_adapter("B")
...

[rajat95]

@BenjaminBossan Thanks for the reply. However adapter A and B are trained simultaneously since loss depends on both outputA and outputB. Is there a way to achieve that?

[BenjaminBossan]

You can activate multiple adapters at once:

model.base_model.set_adapter(["A", "B"])

This way, you can create outputs with the model that only include adapter A, B, or both. Note that this won't work if you use modules_to_save.