Skip to content

Commit f913179

Browse files
committed
refactor rotary embeddings
1 parent 06472bb commit f913179

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

keras_hub/src/models/smollm3/smollm3_layers.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -600,7 +600,7 @@ def call(
600600
batch_size = ops.shape(x)[0]
601601
seq_len = ops.shape(x)[1]
602602
positions = ops.arange(seq_len, dtype="float32")
603-
positions + ops.cast(start_index, dtype="float32")
603+
positions = positions + ops.cast(start_index, dtype="float32")
604604

605605
inv_freq_expanded = ops.broadcast_to(
606606
inv_freq_expanded, (batch_size, ops.shape(self.inv_freq)[0], 1)

0 commit comments

Comments
 (0)