Optimized KV Cache (sliding window) bug fix #925
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
talentJay-ux
commented
Dec 15, 2025
talentJay-ux
commented
- Fix bug because of KV cache and GPT's ptr pointer doesn't get reset when window_size > context_length
- Fix bug because of KV cache and GPT's ptr pointer doesn't get reset
- Fix unit test KV Cache import issue for gpt_with_kv_cache_optimized
talentJay-ux
force-pushed
the
kv_cache/fix
branch
from
4c0cc77 to
b964990
Compare
1. Fix bug because of KV cache and GPT's ptr pointer doesn't get reset when window_size > context_length 2. Fix bug because of KV cache and GPT's ptr pointer doesn't get reset 3. Fix KV Cache import issue for gpt_with_kv_cache_optimized
talentJay-ux
force-pushed
the
kv_cache/fix
branch
from
b964990 to
ee18587
Compare
Owner
|
Thanks for the PR, and thanks also for the unit test, that's very helpful! |
rasbt
approved these changes
Dec 16, 2025
Owner
rasbt
left a comment
rasbt
left a comment
There was a problem hiding this comment.
This is a very carefully crafted PR. Thank you for this!
SethHWeidman
pushed a commit
to SethHWeidman/LLMs-from-scratch
that referenced
this pull request
Dec 21, 2025
1. Fix bug because of KV cache and GPT's ptr pointer doesn't get reset when window_size > context_length 2. Fix bug because of KV cache and GPT's ptr pointer doesn't get reset 3. Fix KV Cache import issue for gpt_with_kv_cache_optimized
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.