Skip to content

Commit 9730b40

Browse files
Update src/llama-model.cpp
Co-authored-by: Xuan-Son Nguyen <[email protected]>
1 parent 68fa44b commit 9730b40

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-model.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13327,7 +13327,7 @@ struct llm_build_arcee : public llm_graph_context {
1332713327
cb(ffn_inp, "ffn_inp", il);
1332813328

1332913329
// feed-forward network
13330-
// ARCEE uses relu^2 instead of swiglu
13330+
// ARCEE uses relu^2 instead of silu
1333113331
cur = build_norm(ffn_inp,
1333213332
model.layers[il].ffn_norm, NULL,
1333313333
LLM_NORM_RMS, il);

0 commit comments

Comments
 (0)