Skip to content

Fix Triton GEMM + PostOp (add matrix) kernel benchmark int8 failure BMG #5595

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dev-tomek wants to merge 6 commits into
base: main
Choose a base branch
from
+3 −1
Open

Fix Triton GEMM + PostOp (add matrix) kernel benchmark int8 failure BMG #5595

Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
fca27a5
empty cache between each run to avoid OOM
dev-tomek Dec 2, 2025
19e276e
add cache flush to flex att bs=16
dev-tomek Dec 3, 2025
facac0f
Revert "add cache flush to flex att bs=16"
dev-tomek Dec 4, 2025
279573d
Merge branch 'main' into tkuczynski/fix_benchmarks_bmg
dev-tomek Dec 4, 2025
923c0a7
move cache clearance to common logic
dev-tomek Dec 5, 2025
82babfb
Merge branch 'main' into tkuczynski/fix_benchmarks_bmg
dev-tomek Dec 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions benchmarks/triton_kernels_benchmark/gemm_postop_addmatrix_benchmark.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,7 @@ def benchmark(B, M, N, K, dtype, provider):
# Maximum across onednn=600, triton=1000
# For onednn and triton: Some configs increase performance with warmup as a step function, but some
# slowly decrease with saturation. Performance is best at 150-200ms range, but we want stable, not just best
torch.xpu.empty_cache()
Copy link
Contributor

@etiotto etiotto Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about the other benchmarks we run, shouldn't we do the same ?

Copy link
Contributor

@whitneywhtsang whitneywhtsang Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do it in a common place, maybe get_empty_cache_for_benchmark?

Copy link
Contributor Author

@dev-tomek dev-tomek Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed the implementation so it is common for all of the benchmarks

do_bench = benchmark_suite.get_do_bench(n_warmup=1000, n_repeat=10, quantiles=[0.5, 0.0, 1.0])
res_dtype = torch.float32 if dtype.is_floating_point else torch.int32
if dtype.is_floating_point:
Expand Down
Loading