Required Metrics for Workload Variant Autoscaler (WVA) Integration with llm-d-inference-sim

[vishakha-ramani]

We (the WVA team from llm-d-incubation/workload-variant-autoscaler) plan to integrate the llm-d-inference-sim for our e2e testing infrastructure. The WVA relies on the following metrics to function correctly:

1. vllm:request_success_total
2. vllm:request_prompt_tokens
3. vllm:request_generation_tokens
4. vllm:time_to_first_token_seconds
5. vllm:time_per_output_token_seconds

While PR #202 introduces the first three of these metrics, the support for the crucial TTFT and TPOT metrics (vllm:time_to_first_token_seconds and vllm:time_per_output_token_seconds) is still missing.

We request the llm-d-inference-sim team to consider adding these metrics as the next immediate step.

cc @WheelyMcBones, @mamy-CS

[mamy-CS]

From @mayabar: A new release v0.5.2 was created, https://github.com/llm-d/llm-d-inference-sim/releases/tag/v0.5.2
It contains all metrics required for e2e test.

[mayabar]

@vishakha-ramani can we close this issue?

[vishakha-ramani]

Yes, and thank you for all your help @mayabar .

[mayabar]

Fixed by #202 and #216