-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Show the reserved output tokens in the context window bar #1458
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
cte
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wow - looks great!
5d47934 to
c70d1ce
Compare
c70d1ce to
de47cab
Compare
de47cab to
1246e99
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Several inline style objects (e.g., for the overlay divs) are repeated. Consider extracting these common styles into constants or CSS classes to improve maintainability.
1246e99 to
a5f8f01
Compare
a5f8f01 to
671064a
Compare
Context
One thing that's not clear in the context window progress bar is the amount of space that's reserved for the output tokens of the model. I've become particularly nervous about this since we let people extend their max tokens out to 128k tokens with the new thinking models. So, I think we should update the progress bar to give a visual hint about how much the output tokens are limiting their context.
Implementation
This PR computes the amount and percent of the context window that's reserved for output and displays it to the right of the input context.
Screenshots
Important
Add reserved output tokens display to context window progress bar with supporting logic and tests.
TaskHeader.tsx.calculateTokenDistributionandgetMaxTokensForModelfrommodel-utils.ts.calculateTokenDistributioncalculates token distribution percentages and sizes.getMaxTokensForModeldetermines max tokens based on model and API config.ContextWindowProgress.test.tsxfor rendering tests.ContextWindowProgressLogic.test.tsfor logic tests.getMaxTokensForModel.test.tsxfor utility function tests.This description was created by
for 671064a. It will automatically update as commits are pushed.