@huydhn is attempting to deploy a commit to the Meta Open Source Team on Vercel.

A member of the Team first needs to authorize it.

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Thanks for the pr!

Is it possible to add in/out to the name? E.g., "meta-llama/Llama-4-Scout-17B-16E-Instruct / tp4 / qps_10 / 200 / 200" -> "meta-llama/Llama-4-Scout-17B-16E-Instruct / tp4 / qps_10 / input-200 / output-200":

Also, curious what is the setting here?

Thanks for the pr!

Is it possible to add in/out to the name? E.g., "meta-llama/Llama-4-Scout-17B-16E-Instruct / tp4 / qps_10 / 200 / 200" -> "meta-llama/Llama-4-Scout-17B-16E-Instruct / tp4 / qps_10 / input-200 / output-200":

Oh, they're already part of the name, you can see them there in the format / 200 / 200. But let me make them clearer with their prefix, i.e. / input_200 / output_200, otherwise, it's unclear which is which

Also, curious what is the setting here?

For latency and thoughput benchmarks, input and output length aren't set directly, but from what I see input_len is 32 and output_len is 128, probably the default values somewhere, I could show them in the chart

One last note here is that input and output lengths are set in latency and serving benchmarks, but not throughput. I try to look into the default that it uses, but there is none for the ShareGPT dataset https://github.com/vllm-project/vllm/blob/main/vllm/benchmarks/datasets.py#L402. Few other datasets like Random or Sonnet define input and output lengths, but not ShareGPT. I will keep these fields empty for throughput benchmark for now.