[Benchmarks] Add core benchmarks preset

Add 'Core' Compute Benchmarks preset with SubmitKernel scenarios with
the following parameters:
- time measurement
- all runtimes
- out of order/in order queue
- with/without measuring completion time
- with/without using events.

Author: PatKamin

.github/workflows/sycl-ur-perf-benchmarking.yml

@@ -24,6 +24,7 @@ on:
           - Full
           - SYCL
           - Minimal
+          - Core
           - Normal
           - Test
           - Gromacs

devops/scripts/benchmarks/CONTRIB.md

@@ -209,7 +209,7 @@ Descriptions should:
     * If adding to an existing category, modify the corresponding `Suite` class (e.g., `benches/compute.py`) to instantiate and return your new benchmark in its `benchmarks()` method.
     * If creating a new category, create a new `Suite` class inheriting from `benches.base.Suite`. Implement `name()` and `benchmarks()`. Add necessary `setup()` if the suite requires shared setup. Add group metadata via `additional_metadata()` if needed.
 3. **Register Suite:** Import and add your new `Suite` instance to the `suites` list in `main.py`.
-4. **Add to Presets:** If adding a new suite, add its `name()` to the relevant lists in `presets.py` (e.g., "Full", "Normal") so it runs with those presets. Update `README.md` and benchmarking workflow to include the new suite in presets' description/choices.
+4. **Add to Presets:** If adding a new suite, add its `name()` to the relevant lists in `presets.py` (e.g., "Full", "Normal") so it runs with those presets. Update `README.md` and benchmarking workflow to include the new suite in presets' description/choices. Don't forget to create a new Suite object in `main.py`.
 
 ## Recommendations
 

devops/scripts/benchmarks/README.md

@@ -115,6 +115,7 @@ The available benchmarks options are:
 * `Full` (BenchDNN, Compute, Gromacs, llama, SYCL, Velocity and UMF benchmarks)
 * `SYCL` (Compute, llama, SYCL, Velocity)
 * `Minimal` (Compute)
+* `Core` (Compute: SubmitKernel)
 * `Normal` (BenchDNN, Compute, Gromacs, llama, Velocity)
 * `Gromacs` (Gromacs)
 * `OneDNN` (BenchDNN)

devops/scripts/benchmarks/benches/compute.py

@@ -353,6 +353,41 @@ def createRrBench(variant_name: str, **kwargs):
         return benches
 
 
+class ComputeBenchCoreSuite(ComputeBench):
+    """
+    A suite for core compute benchmarks scenarios for quick runs.
+    """
+
+    def name(self) -> str:
+        return "Compute Benchmarks Core"
+
+    def benchmarks(self) -> list[Benchmark]:
+        core_benches = []
+        submit_kernel_params = product(
+            list(RUNTIMES),
+            [0, 1],  # in_order_queue
+            [0, 1],  # measure_completion
+            [0, 1],  # use_events
+        )
+        for (
+            runtime,
+            in_order_queue,
+            measure_completion,
+            use_events,
+        ) in submit_kernel_params:
+            core_benches.append(
+                SubmitKernel(
+                    self,
+                    runtime,
+                    in_order_queue,
+                    measure_completion,
+                    use_events,
+                    KernelExecTime=1,
+                )
+            )
+        return core_benches
+
+
 class ComputeBenchmark(Benchmark):
     def __init__(
         self,

devops/scripts/benchmarks/main.py

@@ -269,6 +269,7 @@ def main(directory, additional_env_vars, compare_names, filter, execution_stats)
 
     suites = [
         ComputeBench(),
+        ComputeBenchCoreSuite(),
         VelocityBench(),
         SyclBench(),
         LlamaCppBench(),

devops/scripts/benchmarks/presets.py

@@ -26,6 +26,9 @@
     "Minimal": [
         "Compute Benchmarks",
     ],
+    "Core": [
+        "Compute Benchmarks Core",
+    ],
     "Normal": [
         "BenchDNN",
         "Compute Benchmarks",