Skip to content

Stop multi-rate benchmarks after first failure threshold #604

@ushaket

Description

@ushaket

Problem Statement

When running multi-rate benchmark profiles (constant, poisson, concurrent, and the async phase of sweep), benchmarking continues at higher rates even after a lower rate has already hit a failure constraint (for example over-saturation or excessive errors).

This wastes benchmark time, adds low-signal data at clearly unsustainable rates, and can place unnecessary load on the target system.

Proposed Solution

  • Treat scheduler constraints with request_processing=stop_all as terminal for rate escalation.
  • Sort configured rates/streams ascending for deterministic escalation.
  • For multi-rate profiles, stop generating higher-rate strategies after the first terminal failure.
  • Keep normal completion constraints (stop_local, e.g. max duration/max requests) non-terminal so progression continues.
  • In sweep, always run synchronous and throughput, then apply early-exit logic only during the async-rate phase.

Alternatives Considered

No response

Usage Examples

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions