Skip to content

Commit 792c295

Browse files
authored
Revert model args default changes
1 parent e11ddec commit 792c295

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

examples/models/llama/model_args.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,12 @@
44

55
@dataclass
66
class ModelArgs:
7-
dim: int = 2048
7+
dim: int = 4096
88
n_layers: int = 4
99
n_heads: int = 32
1010
n_kv_heads: Optional[int] = None
1111
vocab_size: int = 512 # Arbitrary value, should be defined later by tokenizer.
12-
hidden_dim: Optional[int] = 8192
12+
hidden_dim: Optional[int] = None
1313
head_dim: Optional[int] = None # Optional customized head_dim
1414
multiple_of: int = 256 # make SwiGLU hidden layer size multiple of large power of 2
1515
ffn_dim_multiplier: Optional[float] = None

0 commit comments

Comments
 (0)