Skip to content

Commit bc5565b

Browse files
authored
Fix a typo in the formula of RoPE
1 parent b18a822 commit bc5565b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

labml_nn/transformers/rope/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ class RotaryPositionalEmbeddings(nn.Module):
7878
x^{(1)}_m x^{(1)}_n \cos (m - n) \theta +
7979
x^{(1)}_m x^{(2)}_n \sin(m - n) \theta &+ \\
8080
- x^{(2)}_m x^{(1)}_n \sin (m - n) \theta +
81-
x^{(2)}_m x^{(1)}_n \cos (m - n) \theta &= \\
81+
x^{(2)}_m x^{(2)}_n \cos (m - n) \theta &= \\
8282
8383
\big(x^{(1)}_m \cos (m - n)\theta - x^{(2)}_m \sin (m - n) \theta\big) x^{(1)}_n &+ \\
8484
\big(x^{(2)}_m \cos (m - n)m\theta + x^{(1)}_m \sin (m - n) \theta\big) x^{(2)}_n &= \\

0 commit comments

Comments
 (0)