Skip to content

Comments

Add max_completion_tokens to gateway ChatCompletionRequest#817

Merged
njbrake merged 1 commit intomozilla-ai:mainfrom
sarmientoF:feat/gateway-max-completion-tokens
Feb 20, 2026
Merged

Add max_completion_tokens to gateway ChatCompletionRequest#817
njbrake merged 1 commit intomozilla-ai:mainfrom
sarmientoF:feat/gateway-max-completion-tokens

Conversation

@sarmientoF
Copy link
Contributor

@sarmientoF sarmientoF commented Feb 15, 2026

Summary

  • Adds max_completion_tokens: int | None = None to the ChatCompletionRequest Pydantic model in the gateway's chat route
  • Newer OpenAI models (o-series, gpt-5-nano, etc.) use max_completion_tokens instead of the legacy max_tokens parameter
  • The SDK's acompletion() already supports this kwarg, but the gateway schema was silently stripping it from incoming requests since it wasn't declared on the model

Details

OpenAI's newer reasoning models (o1, o3-mini, gpt-5-nano, etc.) reject requests that use max_tokens and require max_completion_tokens instead. Clients sending this field to the gateway get it dropped by Pydantic validation, causing either unbounded generation or provider errors.

This one-line addition to ChatCompletionRequest allows the field to pass through model_dump() into the acompletion(**completion_kwargs) call, which already handles it correctly.

Test plan

  • Verify existing max_tokens behavior is unchanged
  • Send a request with max_completion_tokens set and confirm it reaches the provider
  • Confirm schema validation still rejects unknown fields

Newer OpenAI models (o-series, gpt-5-nano, etc.) require
max_completion_tokens instead of max_tokens. The underlying
acompletion() call already supports it, but the gateway's
Pydantic schema was stripping the field from incoming requests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@njbrake njbrake merged commit 2008c49 into mozilla-ai:main Feb 20, 2026
8 of 10 checks passed
@codecov
Copy link

codecov bot commented Feb 20, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
see 44 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants