Skip to content

Commit 552a5ce

Browse files
committed
fix: Fix the bad merge of kv-cache-recurrent for mamba2 changes
Branch: GraniteFour Signed-off-by: Gabe Goodhart <[email protected]>
1 parent 48ebcec commit 552a5ce

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

src/llama-kv-cache-recurrent.cpp

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -597,6 +597,18 @@ bool llama_kv_cache_recurrent::find_slot(const llama_ubatch & ubatch) {
597597
}
598598
}
599599

600+
// Find first to-be-cleared cell
601+
rs_z = -1;
602+
for (int i = min; i <= max; ++i) {
603+
if (rs_z < 0 && cells[i].src == -1) {
604+
rs_z = i;
605+
}
606+
// Stage the source ids for all used cells to allow correct seq_* behavior
607+
// and still make these values available when setting the inputs
608+
cells[i].src0 = cells[i].src;
609+
cells[i].src = i;
610+
}
611+
600612
// allow getting the range of used cells, from head to head + n
601613
head = min;
602614
n = max - min + 1;

0 commit comments

Comments
 (0)