Skip to content

Commit 44d4487

Browse files
ai-edge-botcopybara-github
authored andcommitted
Replace in-place element-wise multiplication with normal one.
- Inference must not update tensors in-place. - Verified that conversion tests of gemma2 with enable_hlfb=False PiperOrigin-RevId: 715980473
1 parent c1e5ce7 commit 44d4487

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

ai_edge_torch/generative/layers/scaled_dot_product_attention.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ def scaled_dot_product_attention(
6565
scale=scale,
6666
)
6767
else:
68-
q.mul_(scale)
68+
q = q * scale
6969
scores = q @ k.transpose(-1, -2)
7070
scores = scores / softcap
7171
scores = torch.tanh(scores)
@@ -130,7 +130,7 @@ def scaled_dot_product_attention_with_hlfb(
130130
scale=scale,
131131
)
132132
else:
133-
q.mul_(scale)
133+
q = q * scale
134134
scores = q @ k.transpose(-1, -2)
135135
scores = scores / softcap
136136
scores = torch.tanh(scores)

0 commit comments

Comments
 (0)