Skip to content

Commit 5d86da2

Browse files
phantomlei3yq33victor
authored andcommitted
fix: add the missing hidden act args in qwen3 model.
1 parent 1ae6ed8 commit 5d86da2

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

xllm/models/llm/qwen3.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ REGISTER_MODEL_ARGS(qwen3, [&] {
9494
LOAD_ARG_OR(dtype, "torch_dtype", "");
9595
LOAD_ARG_OR(vocab_size, "vocab_size", 152064);
9696
LOAD_ARG_OR(hidden_size, "hidden_size", 3584);
97+
LOAD_ARG_OR(hidden_act, "hidden_act", "silu");
9798
LOAD_ARG_OR(n_layers, "num_hidden_layers", 28);
9899
LOAD_ARG_OR(n_heads, "num_attention_heads", 28);
99100
LOAD_ARG(n_kv_heads, "num_key_value_heads");

0 commit comments

Comments
 (0)