Skip to content

Commit 90a7011

Browse files
committed
fix imagebind_LLM bug
1 parent fe7ec53 commit 90a7011

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

imagebind_LLM/llama/llama.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -187,7 +187,7 @@ def forward(self, x: torch.Tensor, start_pos: int, freqs_cis: torch.Tensor, mask
187187
if mask is not None:
188188
scores = scores + mask # (bs, n_local_heads, slen, cache_len + slen)
189189

190-
scores = F.softmax(scores.float(), dim=-1).type_as(xq)
190+
scores = F.softmax(scores.float(), dim=-1).type_as(xq)
191191
output = torch.matmul(scores, values) # (bs, n_local_heads, slen, head_dim)
192192

193193
if adapter is not None:

0 commit comments

Comments
 (0)