Skip to content

Commit ce6e987

Browse files
committed
Add 1.5 config
1 parent 43020ef commit ce6e987

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"dim": 1536,
3+
"ffn_dim_multiplier": 1,
4+
"hidden_dim": 8960,
5+
"n_heads": 12,
6+
"n_kv_heads": 2,
7+
"n_layers": 28,
8+
"norm_eps": 1e-06,
9+
"rope_theta": 1000000.0,
10+
"use_scaled_rope": false,
11+
"vocab_size": 151936
12+
}

0 commit comments

Comments
 (0)