Skip to content

report TTFT, TPOT and other important metrics as part of IGW framework #2163

@nirrozenbaum

Description

@nirrozenbaum

as of today, some of the critical metrics for performance measurements are NOT reported by IGW framework.
while checking the code I did see that some of those metrics are reported by the slo predictor.

slo predictor is a plugin and the framework should not rely on it to have these metrics (e.g., one can run IGW without it).

we should add these metrics to the framework (while slo predictor can keep reporting the predicted ttft, predicted tpot, etc, e.g., predicted_xxx).

Metadata

Metadata

Assignees

Labels

triage/acceptedIndicates an issue or PR is ready to be actively worked on.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions