Projecting a QLoRa adapter to original model space

[alexflint]

Given a base model and a QLoRa adapter, is it possible to create a new model of the same size as the base model representing the weights with the QLoRa perturbation applied to it? During training QLoRa is extremely helpful, but during inference I would like to work with a single checkpoint representing the whole model, rather than with a checkpoint and an adapter. That way, I'll be able to use the huggingface inference API. It will also be nice to keep things as simple as possible.

So what I'm looking for is code to load a base model from a checkpoint, load a QLoRa adapter, output a new model checkpoint representing the weights of the base model with the QLoRa perturbation applied. Is this possible?

[BenjaminBossan]

Hey, could you check if merge_and_unload solves your problem?

[alexflint]

Oh that is exactly what I was looking for! Thank you!

[Nidhogg-lyz]

@BenjaminBossan
I'm facing the same problem. But merge_and_unload didn't work on my code. I use lora with config.bias='none', but after training and peft_model=peft_model.merge_and_unload(), the model.state_dict() has unexpected bias parameters. The base model set conv1.bias=False, so when loading the saved state_dict from peft_model, it raises an error(Unexpected key(s) in state_dict). Setting config.bias='none' still adds bias to the peft_model but only sets them untrainable, but when calling merge_and_unload, the bias parameters are added to the state_dict. Is there any way that I can cancel the bias of lora_layers? Freezing them doesn't mean they won't be added to the base model when calling merge_and_unload.

[BenjaminBossan]

@Nidhogg-lyz Thanks for reporting. Could you please provide a minimal script to reproduce the error?

[Nidhogg-lyz]

@BenjaminBossan
Here is a simple script in colab.

[BenjaminBossan]

Thanks for reporting this and providing a reproducer. I think this is indeed a bug, but it's not really related to the original discussion, so I created a separate issue for it: #1013.