Skip to content

Conversation

markurtz
Copy link
Collaborator

@markurtz markurtz commented May 7, 2025

No description provided.

…similar to the extra query params that were added

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/guidellm/actions/runs/14893341746/artifacts/3081835535.
They will be retained for up to 30 days.

@markurtz markurtz self-assigned this May 7, 2025
@markurtz markurtz requested a review from Copilot May 7, 2025 21:13
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request adds support for passing extra body parameters to the OpenAI pathway, mirroring the existing extra query parameters approach.

  • Updates the SyntheticDatasetCreator signature for improved formatting in synthetic.py.
  • Introduces an extra_body parameter in OpenAI client initialization and integrates it into text and chat completions payload building in backend/openai.py.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/guidellm/dataset/synthetic.py Adjusted method signature formatting for clarity in parameter ordering.
src/guidellm/backend/openai.py Added extra_body support by extending initialization and payload methods.
Comments suppressed due to low confidence (1)

src/guidellm/backend/openai.py:410

  • [nitpick] Consider refactoring the condition by iterating over a list of endpoint types to improve maintainability as additional types may be supported in the future.
if (CHAT_COMPLETIONS in self.extra_body or MODELS in self.extra_body or TEXT_COMPLETIONS in self.extra_body):

@MeganEFlynn MeganEFlynn self-requested a review May 7, 2025 21:30
Copy link

@MeganEFlynn MeganEFlynn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for setting temperature and max tokens!

@markurtz markurtz merged commit fa9f6e2 into main May 7, 2025
10 of 11 checks passed
@markurtz markurtz deleted the openai_extra_body_params branch May 7, 2025 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants