Benchmarks subset

[kwasd]

Related to #2522

[kwasd]

Test run with skipped GEMM and Softmax https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/11788616521/job/32836016215

[pbchekin]

contains(inputs.skip_benchmarks, 'gemm_benchmark.py') returns true if the skip list contains gemm_benchmark.py_default or gemm_benchmark.py_advanced. Is it expected?

[kwasd]

I don't think this is true if the input is a JSON list, see the test run with ['softmax_kernel.py','gemm_benchmark.py']: https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/11788616521/job/32836016215

[pbchekin]

How contains knows this is json and not a string? The input type iof skip_benchmarks is string and there is no fromJSON...

[kwasd]

How contains knows this is json and not a string? The input type iof skip_benchmarks is string and there is no fromJSON...

I don't know, let's add this for clarity.

[kwasd]

It is now fixed: https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/11803428969/job/32881355590
//upd: not sure, there's some errors...

[pbchekin]

Is it simpler to add input checkboxes for each benchmark?

[kwasd]

Is it simpler to add input checkboxes for each benchmark?

I've tried, but number of inputs is limited to 10

Invalid workflow file: .github/workflows/triton-benchmarks.yml#L1
you may only define up to 10 `inputs` for a `workflow_dispatch` event

[pbchekin]

Looks good! Let's merge and see how daily run work.

[kwasd]

Thanks for your contribution!