Skip to content

Commit 46a2c1e

Browse files
[GuideLLM Refactor] Edge case errors (#376)
## Summary This PR handles errors that occur when there are no successful requests. There will obviously still be an error, but it will be one that the user can get useful information from, rather than one that is the inner workings breaking. ## Details - Adds default value for an inner data type to allow it to work in this edge case. - Adds an error check that creates a runtime error with an explanation for the failure. The error message can be changed if you would like the wording changed. - Fixes a type literal mismatch. ## Test Plan - Run GuideLLM against a mock server in a way that results in all requests failing. Like setting the max token value way too small. --- - [x] "I certify that all code in this PR is my own, except as noted below." ## Use of AI - [ ] Includes AI-assisted code completion - [ ] Includes code generated by an AI application - [ ] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes `## WRITTEN BY AI ##`) --------- Signed-off-by: Jared O'Connell <[email protected]>
1 parent da02ee8 commit 46a2c1e

File tree

2 files changed

+4
-3
lines changed

2 files changed

+4
-3
lines changed

src/guidellm/benchmark/aggregator.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -441,7 +441,7 @@ def __call__(
441441

442442
def compile(
443443
self, state: AggregatorState, scheduler_state: SchedulerState
444-
) -> dict[Literal["scheduler_stats"], BenchmarkSchedulerStats]:
444+
) -> dict[Literal["run_stats"], BenchmarkSchedulerStats]:
445445
"""
446446
Compile scheduler timing metrics into benchmark statistics.
447447
@@ -473,7 +473,7 @@ def compile(
473473
key="worker_resolve_time", type_="avg", default=0.0
474474
),
475475
worker_resolve_end_delay_avg=state.get_metric(
476-
key="worker_resolve_end_delay", type_="avg"
476+
key="worker_resolve_end_delay", type_="avg", default=0.0
477477
),
478478
finalized_delay_avg=state.get_metric(
479479
key="finalized_delay", type_="avg", default=0.0

src/guidellm/benchmark/profile.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -680,6 +680,8 @@ def next_strategy(
680680
self.throughput_rate = (
681681
prev_benchmark.metrics.requests_per_second.successful.mean
682682
)
683+
if self.synchronous_rate <= 0 and self.throughput_rate <= 0:
684+
raise RuntimeError("Invalid rates in sweep; aborting. Were there any successful requests?")
683685
self.measured_rates = list(
684686
np.linspace(
685687
self.synchronous_rate,
@@ -698,7 +700,6 @@ def next_strategy(
698700
if strat.type_ == self.strategy_type
699701
]
700702
)
701-
702703
if self.strategy_type == "constant":
703704
return AsyncConstantStrategy(
704705
rate=self.measured_rates[next_rate_index],

0 commit comments

Comments
 (0)