Skip to content

Commit 87d90b4

Browse files
authored
feat: Adds embed_tokens, lm_head as trainable for vocab expansion in peft and enables tying of adapters.
1 parent 3a57f8a commit 87d90b4

File tree

4 files changed

+248
-24
lines changed

4 files changed

+248
-24
lines changed

tests/artifacts/language_models/maykeye-tinyllama-v0/config.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
"num_hidden_layers": 8,
1515
"pad_token_id": 0,
1616
"rms_norm_eps": 1e-06,
17-
"tie_word_embeddings": false,
17+
"tie_word_embeddings": true,
1818
"torch_dtype": "bfloat16",
1919
"transformers_version": "4.30.2",
2020
"use_cache": true,

0 commit comments

Comments
 (0)