-
Notifications
You must be signed in to change notification settings - Fork 2.6k
feat: exclude GPT-5 models from 20% context window output token cap #6963
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- GPT-5 models now bypass the 20% context window cap and use their full configured max output tokens (e.g., 128k) - Added logic to detect GPT-5 models by checking if modelId contains "gpt-5" (case-insensitive) - Added comprehensive test coverage for GPT-5 exclusion logic - Non-GPT-5 models continue to be capped at 20% of context window as before
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewing my own code because apparently I trust no one, not even myself.
| // Exception: GPT-5 models should use their exact configured max output tokens | ||
| if (model.maxTokens) { | ||
| // Check if this is a GPT-5 model (case-insensitive) | ||
| const isGpt5Model = modelId.toLowerCase().includes("gpt-5") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The pattern matching here could be more precise. Currently, modelId.toLowerCase().includes("gpt-5") would match unintended model IDs like "not-gpt-5-compatible" or "legacy-gpt-500".
Could we consider a more specific pattern? Perhaps:
| const isGpt5Model = modelId.toLowerCase().includes("gpt-5") | |
| // Check if this is a GPT-5 model (case-insensitive, more precise matching) | |
| const isGpt5Model = /^(openai\/)?gpt-5($|-|\.)/.test(modelId.toLowerCase()) |
This would match "gpt-5", "gpt-5-turbo", "openai/gpt-5-preview" but not "not-gpt-5" or "gpt-500".
| } | ||
|
|
||
| // If model has explicit maxTokens, clamp it to 20% of the context window | ||
| // Exception: GPT-5 models should use their exact configured max output tokens |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This GPT-5 exception is a significant behavior change that should be documented in the function's JSDoc. Future maintainers might wonder why GPT-5 models get special treatment. Could we add a note to the function documentation explaining this exception?
| } | ||
|
|
||
| // Test various GPT-5 model IDs | ||
| const gpt5ModelIds = ["gpt-5", "gpt-5-turbo", "GPT-5", "openai/gpt-5-preview", "gpt-5-32k", "GPT-5-TURBO"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great test coverage! Though it might be worth adding edge case tests to ensure the pattern matching doesn't have false positives. For example, testing that "not-gpt-5", "gpt-500", or "legacy-gpt-5-incompatible" don't incorrectly bypass the cap.
daniel-lxs
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This PR addresses a Slack request to exclude models containing "gpt-5" from the logic that caps the max output tokens to 20% of the context window.
Changes
getModelMaxOutputTokens()insrc/shared/api.tsto detect GPT-5 models (case-insensitive check for "gpt-5" in modelId)Testing
Impact
This change allows GPT-5 models to utilize their full 128k max output token capacity as configured, improving their ability to generate longer responses without artificial limitations.
Important
getModelMaxOutputTokensnow allows GPT-5 models to bypass the 20% context window cap, using their full configured max tokens, with tests verifying this behavior.getModelMaxOutputTokensinapi.tsnow excludes GPT-5 models from the 20% context window cap, allowing them to use their full configured max tokens.api.spec.tsto verify GPT-5 models bypass the cap and use full max tokens.This description was created by
for 133a953. You can customize this summary. It will automatically update as commits are pushed.