fix: OpenAI-compatible providers inconsistently handle max_tokens vs max_completion_tokens

[peteski22]

Problem

OpenAI deprecated max_tokens in favour of max_completion_tokens for newer models (o-series, gpt-5). Some OpenAI-compatible providers have adopted this change, some support both, and some still only accept max_tokens.

PR #865 addressed this for OpenaiProvider only, but the same issue affects other providers that inherit from BaseOpenAIProvider.

Current state

BaseOpenAIProvider._convert_completion_params passes through whichever field the user provides without remapping. This means:

• Users passing max_tokens get 400 errors on OpenAI o-series/gpt-5 models
• PR fix(openai): map max_tokens to max_completion_tokens #865 fixed this for OpenaiProvider but not for other providers like Azure OpenAI, which also rejects max_tokens for o-series models

Provider landscape

The OpenAI spec now uses max_completion_tokens. Providers that claim OpenAI compatibility are at various stages of adoption:

• vLLM deprecated max_tokens in favor of max_completion_tokens (vllm-project/vllm#9837)
• Azure OpenAI supports both, but o-series models require max_completion_tokens (docs)
• llama.cpp added max_completion_tokens support recently (ggml-org/llama.cpp#19831)
• DeepSeek does not support max_completion_tokens (docs)
• Mistral does not support max_completion_tokens (docs)
• Other providers would need their API docs checked individually

Proposed approach

The base provider should follow the current OpenAI spec — remap max_tokens → max_completion_tokens in BaseOpenAIProvider._convert_completion_params. Providers that deviate from the spec should override the method to handle their own requirements (e.g. remapping back to max_tokens, stripping unsupported fields like user or reasoning_effort).

This is the approach taken in the Go implementation: mozilla-ai/any-llm-go#57. The base compatible provider sends max_completion_tokens (per spec), and non-compatible providers supply a request transform hook that can adjust any fields before the request is sent.

Related

• OpenAI provider should map max_tokens to max_completion_tokens for newer models #862 — original issue
• fix(openai): remap max_tokens from kwargs and warn on conflict #864 — proposed base provider fix (closed)
• fix(openai): map max_tokens to max_completion_tokens #865 — OpenAI-only fix (merged)
• mozilla-ai/any-llm-go#57 — Go implementation with per-provider request transform hook

[njbrake]

@peteski22 thank you for the thorough research: you're right, your original approach proposed in #864 was the right one, and my patch in #865 was unnecessarily conservative, sorry about that. Let's get your original proposal merged in, or I'd be happy to pick it up if you'd rather.

[peteski22]

@peteski22 thank you for the thorough research: you're right, your original approach proposed in #864 was the right one, and my patch in #865 was unnecessarily conservative, sorry about that. Let's get your original proposal merged in, or I'd be happy to pick it up if you'd rather.

Hey @njbrake thanks for the reply, I appreciate the conversation on this.

TBH it works both ways, as my PR (#864) would have still made things 'uncomfortable' for the providers that claim OpenAI compatibility but aren't using the latest version of the spec.

So either way we'd have some follow up to do to fix one set of providers or the other 😄

If you want to handle the changes that's fine with me 👍🏼