Skip to content

Commit 6997fad

Browse files
committed
Update rope comment
1 parent a5f19bb commit 6997fad

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-model.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12216,7 +12216,7 @@ struct llm_build_olmo2 : public llm_graph_context {
1221612216
const bool is_swa = hparams.is_swa(il);
1221712217

1221812218
if (is_swa) {
12219-
// For sliding window layers, Olmo3 does not use rope scaling.
12219+
// For sliding window layers, Olmo3 use regular rope with no yarn rope scaling.
1222012220
// This is achieved here by setting freq_scale and attn_factor to 1.
1222112221
// We also set ext_factor to 0 to avoid a few unnecessary computations.
1222212222
Qcur = ggml_rope_ext(

0 commit comments

Comments
 (0)