Qualcomm AI Engine Direct - fix sliding attention update bug (#14426)

pytorchbot · haowhsu-quic · web-flow · commit 7f201c2bac57 · 2025-09-23T12:13:21.000-07:00
Co-authored-by: haowhsu-quic &lt;111341466+haowhsu-quic@users.noreply.github.com&gt;
diff --git a/examples/qualcomm/oss_scripts/llama/runner/kv_manager.cpp b/examples/qualcomm/oss_scripts/llama/runner/kv_manager.cpp
@@ -242,9 +242,8 @@ void KVManager<T>::update_attention_mask(
         std::fill_n(
             cur_ptr, std::abs(n_past + ar_len) - avalible_cache_len, neg_val);
       }
-
-      cur_ptr += metadata_.context_len;
     }
+    cur_ptr += metadata_.context_len;
   }
 }
 

Original file line number	Diff line number	Diff line change
`@@ -242,9 +242,8 @@ void KVManager<T>::update_attention_mask(`
`242`	`242`	`std::fill_n(`
`243`	`243`	`cur_ptr, std::abs(n_past + ar_len) - avalible_cache_len, neg_val);`
`244`	`244`	`}`
`245`		`-`
`246`		`- cur_ptr += metadata_.context_len;`
`247`	`245`	`}`
	`246`	`+ cur_ptr += metadata_.context_len;`
`248`	`247`	`}`
`249`	`248`	`}`
`250`	`249`