[LoRA] make lora alpha and dropout configurable

[linoytsaban]

make lora_alpha and lora_dropout configurable for lora training,
if not provided, sets by default lora_alpha to the value of --rank to maintain current behaviour

edit: this PR will only make lora_dropout configurable until #11324 is merged to handle inference with lora_rank != lora_alpha

[linoytsaban]

@bot /style

[github-actions]

Style fixes have been applied. View the workflow run here.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[linoytsaban]

@bot /style

[github-actions]

Style fixes have been applied. View the workflow run here.

[sayakpaul]

Thanks for this effort!

LoRA dropout is okay to configure for now but using a different lora_alpha will have an adverse effect on inference without #11324. Let's revisit after #11324 is merged?

[linoytsaban]

good point! I thought we handled inference, will revert lora_alpha

[linoytsaban]

@bot /style

[github-actions]

Style fixes have been applied. View the workflow run here.

[linoytsaban]

@sayakpaul I think we can merge this?

[sayakpaul]

Thank you! Let's maybe add an empty commit to retrigger the CI before merging?