Skip to content

Commit a8ebf7c

Browse files
committed
Update on "Update eager runner to support AttentionSink"
This PR updates the eager runner to support AttentionSink. It also fixes issues in the `chat_completion` function to properly handle the position id. Differential Revision: [D66076486](https://our.internmc.facebook.com/intern/diff/D66076486/) [ghstack-poisoned]
2 parents 6481210 + 8cf2b28 commit a8ebf7c

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

examples/models/llama/source_transformation/attention_sink.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -266,7 +266,7 @@ def _replace_attention(
266266
for _, child_module in module._modules.items():
267267
if len(list(child_module.children())) > 0: # pyre-ignore [16]
268268
_replace_attention(
269-
module=child_module, # pyre-ignore [6]
269+
module=child_module, # pyre-ignore [6]
270270
rope_with_attention_sink=rope_with_attention_sink,
271271
sink_size=sink_size,
272272
window_size=window_size,

0 commit comments

Comments
 (0)