Configurable max_tokens/max_completion_tokens key

[sjmonson]

Summary

Makes the max_tokens request key configurable through an environment variable per endpoint type. Defaults to max_tokens for legacy completions and max_completion_tokens for chat/completions

Details

• Add the GUIDELLM__OPENAI__MAX_OUTPUT_KEY config option which is a dict mapping from route name -> output tokens key. Default is {"text_completions": "max_tokens", "chat_completions": "max_completion_tokens"}

Test Plan

Related Issues

• Closes rm max_completion_tokens #395
• Closes Guidellm adds unexpected field to requests #269
• Related Possible invalid request formatting for `max_completion_tokens` #210

---

• "I certify that all code in this PR is my own, except as noted below."

Use of AI

• Includes AI-assisted code completion
• Includes code generated by an AI application
• Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

[Copilot]

Pull Request Overview

This PR implements configurable request keys for output token limits in OpenAI API calls. Instead of hardcoding both max_tokens and max_completion_tokens in all requests, the system now uses the appropriate key based on endpoint type through a new environment variable configuration.

• Adds GUIDELLM__OPENAI__MAX_OUTPUT_KEY configuration mapping endpoint types to their respective output token keys
• Updates payload generation to use the configured key instead of setting both keys
• Fixes test assertions to match the new single-key approach

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
src/guidellm/config.py	Adds new max_output_key configuration with defaults for text and chat completions
src/guidellm/backend/openai.py	Updates payload generation to use configurable key and adds type definitions
tests/unit/conftest.py	Removes duplicate token limit assertions and fixes mock response generation

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[jaredoconnell]

You may want to wait for Mark's review, but looks good to me.