-
Notifications
You must be signed in to change notification settings - Fork 123
Open
Labels
kind:choreInternal improvementsInternal improvementsnote:upstreamThe issue must be tackled upstreamThe issue must be tackled upstream
Description
Related to elixir-nx/axon#549
LoRA targets QKV nodes. Right now the only way to do it is by inferring based on layer name. We can provide more information by giving specific metadata (such as op_name) for which layers are QKV
Metadata
Metadata
Assignees
Labels
kind:choreInternal improvementsInternal improvementsnote:upstreamThe issue must be tackled upstreamThe issue must be tackled upstream