feat: Jan manages Context Allocation dynamically#7609
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds automatic context window growth + seamless continuation when a generation is truncated (finish_reason "length") or fails due to context-size limits, so users see an uninterrupted assistant message while the model reloads and resumes.
Changes:
- Detects truncated responses and triggers an automatic context-size increase + regeneration, carrying the partial assistant output forward as a “prefill”.
- Extends the custom transport/hook API with
setContinueFromContentand injects the partial content into the next UI stream as the first text delta. - Updates message rendering to support a continuation placeholder state (hide actions, adjust animation, show “Growing the Mind...” shimmer).
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| web-app/src/routes/threads/$threadId.tsx | Continuation orchestration: detect truncation/context errors, increase ctx_len, show placeholder + loading UI, hide actions during continuation |
| web-app/src/lib/custom-chat-transport.ts | Implements continuation “prefill” and stream wrapper that prepends partial text as the first text-delta |
| web-app/src/hooks/use-chat.ts | Exposes setContinueFromContent to callers via the useChat wrapper |
| web-app/src/containers/RenderMarkdown.tsx | Adds isAnimating control for streaming markdown animation |
| web-app/src/containers/MessageItem.tsx | Plumbs isAnimating/hideActions to message rendering and action visibility |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
4 tasks
urmauur
approved these changes
Mar 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe Your Changes
When a response is cut off due to hitting the context size limit (finish_reason: "length") OR request payload exceeds the context size window (context shift disabled),
Jan now automatically increases the context size by 50% (minimum 32768), reloads the model,
and seamlessly continues generation — prepending the partial response as a prefill so the
model resumes exactly where it stopped rather than regenerating from scratch.
next request
demo.mov
Self Checklist