Skip to content

Commit d601f22

Browse files
Update train_a_generative_llm.py
Lower embedding dim to reduce ram.
1 parent fb6d414 commit d601f22

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

train_a_generative_llm.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -195,7 +195,7 @@
195195
# embedding output dim must be an even number
196196
# Maximize EMBEDDING_N based on available RAM and CPU / GPU
197197

198-
EMBEDDING_N = 6 # trial.suggest_int('embedding_n',6, 9) # 12
198+
EMBEDDING_N = 5 # trial.suggest_int('embedding_n',6, 9) # 12
199199
EMBEDDING_DIM = int(EMBEDDING_N * 2)
200200

201201
# Size of the projection layer bet

0 commit comments

Comments
 (0)