[BUG]: Qwen3 VL + HandleRunOutOfContext()

[aropb]

Description

The error occurs when filling in the context.

Tried the options InferenceParams:
TokensKeep: -1, 0, value

Code:
...
InteractiveExecutor executor = new(context, logger);
...
await foreach (string text in session.ChatAsync(new ChatHistory.Message(AuthorRole.User, query), false, inferenceParams, cancellationToken))
{
}

Critical error:

D:\a\LLamaSharp\LLamaSharp\src\llama-kv-cache.cpp:398: GGML_ASSERT(hparams.n_pos_per_embd() == 1 && "seq_add() is only supported for n_pos_per_embd() == 1") failed

Environment & Configuration

• Operating system: Windows 11
• .NET runtime version: NET 10.0.3
• LLamaSharp version: 0.26.0
• CUDA version (if you are using cuda backend): 12.9
• CPU & GPU device: GPU RTX 5090

[aropb]

More details, the error occurs on the model (VL models):
Qwen3-VL-30B-A3B-Instruct-Q6_K_L.gguf
Qwen3-VL-30B-A3B-Instruct-Q4_K_M.gguf <- official Qwen3 VL
Qwen3-VL-8B-Instruct-F16.gguf <- official Qwen3 VL

But this one doesn't have it (text models):
Qwen3-30B-A3B-Instruct-2507-Q6_K_L.gguf

Mtmd is not used!

LLamaSharp Binaries:
https://github.com/SciSharp/LLamaSharpBinaries/releases/tag/506bb6e01009058f355

[aropb]

the error occurs here (class StatefulExecutorBase):

Context.NativeHandle.MemorySequenceAdd(LLamaSeqId.Zero, tokensToKeep + n_discard, _pastTokensCount, -n_discard);

Execution data:

        Context.NativeHandle.MemorySequenceRemove(LLamaSeqId.Zero, 3488, 5584);
        Context.NativeHandle.MemorySequenceAdd(LLamaSeqId.Zero, 5584, 7680, -2096); 


    protected virtual void HandleRunOutOfContext(int tokensToKeep)
    {
        // if we run out of context:
        // - take the tokensToKeep first tokens from the original prompt (via n_past)
        // - take half of the last (n_ctx - tokensToKeep) tokens and recompute the logits in batches
        var n_left = _pastTokensCount - tokensToKeep;
        var n_discard = n_left / 2;

        Context.NativeHandle.MemorySequenceRemove(LLamaSeqId.Zero, tokensToKeep, tokensToKeep + n_discard);
        Context.NativeHandle.MemorySequenceAdd(LLamaSeqId.Zero, tokensToKeep + n_discard, _pastTokensCount, -n_discard); // <- here

        _pastTokensCount -= n_discard;
    }

llama.cpp:
ggml-org/llama.cpp#19116

https://github.com/ggml-org/llama.cpp/blob/a0c91e8f9f69c11bbdb1111af20537e206f0866f/src/llama-kv-cache.cpp#L403

but what solution can there be for us?

[aropb]

@martindevans, I suggest this solution, apparently this is the only reliable option with a very small slowdown.

    protected virtual async Task HandleRunOutOfContext(int tokensToKeep)
    {
        var n_left = _pastTokensCount - tokensToKeep;
        var n_discard = n_left / 2;

        _session_tokens.RemoveRange(tokensToKeep - 1, n_discard);

        LLamaBatch batch = new();
        Context.NativeHandle.MemoryClear();

        (var result, _, _pastTokensCount) = await Context.DecodeAsync(_session_tokens, LLamaSeqId.Zero, batch, 0);
        if (result != DecodeResult.Ok) { throw new LLamaDecodeError(result); }
    }