Skip to content

Conversation

jaredoconnell
Copy link
Collaborator

Summary

This PR handles errors that occur when there are no successful requests. There will obviously still be an error, but it will be one that the user can get useful information from, rather than one that is the inner workings breaking.

Details

  • Adds default value for an inner data type to allow it to work in this edge case.
  • Adds an error check that creates a runtime error with an explanation for the failure. The error message can be changed if you would like the wording changed.
  • Fixes a type literal mismatch.

Test Plan

  • Run GuideLLM against a mock server in a way that results in all requests failing. Like setting the max token value way too small.

  • "I certify that all code in this PR is my own, except as noted below."

Use of AI

  • Includes AI-assisted code completion
  • Includes code generated by an AI application
  • Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

Copy link
Collaborator

@sjmonson sjmonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammar nit, otherwise LGTM

Copy link
Collaborator

@sjmonson sjmonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above

@jaredoconnell jaredoconnell force-pushed the features/refactor/edge-case-errs branch from 612526d to f8810e8 Compare September 29, 2025 16:11
@jaredoconnell jaredoconnell merged commit 46a2c1e into vllm-project:features/refactor/base Sep 29, 2025
10 of 17 checks passed
@markurtz markurtz added this to the v0.4.0 milestone Oct 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants