We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent b18a822 commit bc5565bCopy full SHA for bc5565b
labml_nn/transformers/rope/__init__.py
@@ -78,7 +78,7 @@ class RotaryPositionalEmbeddings(nn.Module):
78
x^{(1)}_m x^{(1)}_n \cos (m - n) \theta +
79
x^{(1)}_m x^{(2)}_n \sin(m - n) \theta &+ \\
80
- x^{(2)}_m x^{(1)}_n \sin (m - n) \theta +
81
- x^{(2)}_m x^{(1)}_n \cos (m - n) \theta &= \\
+ x^{(2)}_m x^{(2)}_n \cos (m - n) \theta &= \\
82
83
\big(x^{(1)}_m \cos (m - n)\theta - x^{(2)}_m \sin (m - n) \theta\big) x^{(1)}_n &+ \\
84
\big(x^{(2)}_m \cos (m - n)m\theta + x^{(1)}_m \sin (m - n) \theta\big) x^{(2)}_n &= \\
0 commit comments