fix llama issue

[yao-matrix]

similar like this PR #39646, fix the same issue found while enabling llama lora finetuning across multiple card.

@SunMarc , pls help review, thx very much.

[SunMarc]

nice

[SunMarc]

@bot /style

[github-actions]

Style fix runs successfully without any file modified.

[SunMarc]

You need to propagate the changes to other repos also using make fix-copies. Btw since we are touching llama model, can you share more details to the issue you had ?

[yao-matrix]

You need to propagate the changes to other repos also using make fix-copies. Btw since we are touching llama model, can you share more details to the issue you had ?

@SunMarc when I enable this multi-adapter lora inference in peft huggingface/peft#2711 on a 2-card env, the issue raised, it's because this line https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L66 is not a Module, so not hooked, same issue as before we discussed.

[SunMarc]

https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L66 is not a Module, so not hooked, same issue as before we discussed.

it is a nn.Module no ? class LlamaRMSNorm(nn.Module):. Can you share the traceback ?

[yao-matrix]

https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L66 is not a Module, so not hooked, same issue as before we discussed.

it is a nn.Module no ? class LlamaRMSNorm(nn.Module):. Can you share the traceback ?

I mean return self.weight * hidden_states.to(input_dtype) here * is not a nn.Module, so cannot hooked to place to same device automatically by accelerate, I'm trying to reproduce and get the log

[yao-matrix]

seems i lost the env to reproduce it, let's put it on hold here, and once i reproduced it, i'll put the log back

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: arcee, aria, bitnet, cohere, cohere2, deepseek_v2, deepseek_v3, diffllama, doge, dots1, ernie4_5, gemma, gemma2, glm, glm4, glm4_moe