Skip to content

Commit bcc20e4

Browse files
authored
Correctly populate metric values for each commit (#6991)
This table shows the raw data from each commit / workflow / job that the dashboard uses to draw the chart. However, it screwed up when the same metric appears multiple times on one commit. For example, there are four `median_itl_ms` metrics on vLLM dashboard for each commit, one for each qps `[1, 4, 16, inf]`, while there is only one latency metric on the commit. On the [dashboard](https://hud.pytorch.org/benchmark/llms?startTime=Fri%2C%2001%20Aug%202025%2023%3A34%3A26%20GMT&stopTime=Fri%2C%2008%20Aug%202025%2023%3A34%3A26%20GMT&granularity=hour&lBranch=main&lCommit=b2c8ce57c68db0764a49d66f048b8a7a5cef9d13&rBranch=main&rCommit=808a7b69df479b6b3a16181711cac7ca28a9b941&repoName=vllm-project%2Fvllm&benchmarkName=&modelName=meta-llama%2FLlama-4-Scout-17B-16E-Instruct&backendName=All%20Backends&modeName=All%20Modes&dtypeName=All%20DType&deviceName=cuda%20(NVIDIA%20H100%2080GB%20HBM3)&archName=All%20Platforms) showed 2 latency values on the top 2 rows. This was wrong because these 2 values were from 2 commits. The rest of the column was empty: <img width="2453" height="146" alt="Screenshot 2025-08-08 at 17 16 32" src="https://github.com/user-attachments/assets/6ce30050-0fd4-4995-b903-c155cdb0d2d0" /> The [correct table](https://torchci-git-fork-huydhn-fix-data-details-table-fbopensource.vercel.app/benchmark/llms?startTime=Fri%2C%2001%20Aug%202025%2023%3A34%3A26%20GMT&stopTime=Fri%2C%2008%20Aug%202025%2023%3A34%3A26%20GMT&granularity=hour&lBranch=main&lCommit=b2c8ce57c68db0764a49d66f048b8a7a5cef9d13&rBranch=main&rCommit=808a7b69df479b6b3a16181711cac7ca28a9b941&repoName=vllm-project%2Fvllm&benchmarkName=&modelName=meta-llama%2FLlama-4-Scout-17B-16E-Instruct&backendName=All%20Backends&modeName=All%20Modes&dtypeName=All%20DType&deviceName=cuda%20(NVIDIA%20H100%2080GB%20HBM3)&archName=All%20Platforms) should show the correct values for each commit: <img width="2326" height="147" alt="Screenshot 2025-08-08 at 17 35 17" src="https://github.com/user-attachments/assets/e1c98e4c-9dc9-4b8c-9246-3d51c5bed2f3" /> ### Testing [Preview with the correct value for each commit](https://torchci-git-fork-huydhn-fix-data-details-table-fbopensource.vercel.app/benchmark/llms?startTime=Fri%2C%2001%20Aug%202025%2023%3A34%3A26%20GMT&stopTime=Fri%2C%2008%20Aug%202025%2023%3A34%3A26%20GMT&granularity=hour&lBranch=main&lCommit=b2c8ce57c68db0764a49d66f048b8a7a5cef9d13&rBranch=main&rCommit=808a7b69df479b6b3a16181711cac7ca28a9b941&repoName=vllm-project%2Fvllm&benchmarkName=&modelName=meta-llama%2FLlama-4-Scout-17B-16E-Instruct&backendName=All%20Backends&modeName=All%20Modes&dtypeName=All%20DType&deviceName=cuda%20(NVIDIA%20H100%2080GB%20HBM3)&archName=All%20Platforms) Signed-off-by: Huy Do <[email protected]>
1 parent c61852d commit bcc20e4

File tree

1 file changed

+29
-6
lines changed

1 file changed

+29
-6
lines changed

torchci/components/benchmark/llms/components/LLMsGraphPanel.tsx

Lines changed: 29 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -360,7 +360,17 @@ const MetricTable = ({
360360
metricNames.forEach((metric) => {
361361
if (chartData[metric]?.length) {
362362
const label = METRIC_DISPLAY_SHORT_HEADERS[metric] ?? metric;
363-
row[label] = chartData[metric][index]?.actual ?? "";
363+
// Find the matching record for this metric based on workflow_id and other identifying properties
364+
const matchingRecord = chartData[metric].find(
365+
(record: any) =>
366+
record.workflow_id === entry.workflow_id &&
367+
record.job_id === entry.job_id &&
368+
record.model === entry.model &&
369+
record.device === entry.device &&
370+
record.dtype === entry.dtype
371+
);
372+
row[label] =
373+
chartData[metric][index]?.actual ?? matchingRecord?.actual ?? "";
364374
}
365375
});
366376
return row;
@@ -437,11 +447,24 @@ const MetricTable = ({
437447
</TableCell>
438448
{metricNames
439449
.filter((metric) => chartData[metric]?.length)
440-
.map((metric) => (
441-
<TableCell key={`${metric}-${index}`} sx={{ py: 0.25 }}>
442-
{chartData[metric][index]?.actual ?? ""}
443-
</TableCell>
444-
))}
450+
.map((metric) => {
451+
// Find the matching record for this metric based on workflow_id and other identifying properties
452+
const matchingRecord = chartData[metric].find(
453+
(record: any) =>
454+
record.workflow_id === entry.workflow_id &&
455+
record.job_id === entry.job_id &&
456+
record.model === entry.model &&
457+
record.device === entry.device &&
458+
record.dtype === entry.dtype
459+
);
460+
return (
461+
<TableCell key={`${metric}-${index}`} sx={{ py: 0.25 }}>
462+
{chartData[metric][index]?.actual ??
463+
matchingRecord?.actual ??
464+
""}
465+
</TableCell>
466+
);
467+
})}
445468
</TableRow>
446469
);
447470
})}

0 commit comments

Comments
 (0)