fix: Fix the issue where some model providers do not yet support max_completion_tokens, causing the inability to set the maximum number of response tokens.
#7592
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dosubot
bot
added
size:S
hannesrudolph
added
the
Issue/PR - Triage
hannesrudolph
added
PR - Needs Preliminary Review
and removed
Issue/PR - Triage
Contributor
Author
|
I've implemented this fix specifically in the Moonshot provider implementation, so I'm closing this PR as it may have a broader impact. |
github-project-automation
bot
moved this from PR [Needs Prelim Review]
to Done
in Roo Code Roadmap
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related GitHub Issue
Closes: #6936
Description
The new version of the OpenAI API uses
max_completion_tokensinstead of the previousmax_tokens. However, other providers such as Qwen, Moonshot, and DeepSeek still use the oldmax_tokens. Therefore, I've modified the request to include both parameters simultaneously. Additionally, according to the OpenAI API documentation, the O-series models are incompatible withmax_tokens. Hence, I've added a condition to include onlymax_completion_tokenswhen the model is an O-series model.Test Procedure
Using MoonPalace, you can see that both parameters are included in the request body:
"max_tokens": -1, "max_completion_tokens": -1.Pre-Submission Checklist
Get in Touch
Discord username: haha.w
Important
Fix handling of
max_tokensandmax_completion_tokensfor different model providers inopenai.ts, ensuring compatibility with O-series models.addMaxTokensIfNeeded()inopenai.tsto include bothmax_tokensandmax_completion_tokensfor compatibility with different model providers.max_completion_tokensfor O-series models usingisO3FamilyModel().isO3FamilyModel()to check if a model is part of the O-series inopenai.ts.openai.tsto reflect changes in token handling logic.This description was created by
for 3f1b4cb. You can customize this summary. It will automatically update as commits are pushed.