ci: Add benchmark ci (#328)

* add benchmark ci

* install twice do to switch branch

* explicit bench mark pytest install

* Drop benchmark ci for now

* CI in separate commit

* run on every pull request

* Update .github/workflows/benchmark.yml

Co-authored-by: Copilot <[email protected]>

* clean

* clean a bit

* markdown

* Clean output

* fail on 10% drop and compare

* safe benchmark cross checkout

* benchmark compare with int

* What happens on new function/rename?

* Add a quick warmup run

* 5%

---------

Co-authored-by: Copilot <[email protected]>

Author: APN-Pucky

.github/workflows/benchmark.yml

@@ -0,0 +1,64 @@
+name: Benchmark Comparison
+
+on:
+  pull_request:
+
+jobs:
+  benchmark:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout PR branch
+        uses: actions/checkout@v5
+        with:
+          fetch-depth: 0  # important to fetch full history for git compare
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.13"
+
+
+      - uses: astral-sh/setup-uv@v7
+
+      - name: Install base
+        run: |
+          git checkout ${{ github.event.pull_request.base.sha }}
+          uv pip install --system '.[test]'
+
+      # Run benchmarks on base
+      - name: Run benchmarks on base
+        run: |
+          pytest benchmarks --benchmark-only --benchmark-warmup='on' --benchmark-warmup-iterations=1 --benchmark-min-rounds=10 --benchmark-save=base --no-cov --disable-warnings
+
+      - name: Install PR
+        run: |
+          git checkout -
+          uv pip install --system '.[test]'
+
+      # Run benchmarks on MR
+      - name: Run benchmarks on MR
+        run: |
+          pytest benchmarks --benchmark-only --benchmark-warmup='on' --benchmark-warmup-iterations=1 --benchmark-min-rounds=10 --benchmark-save=mr --benchmark-compare=0001_base --benchmark-compare-fail=min:5% --no-cov --disable-warnings
+
+      # Compare results
+      - name: Run benchmarks on PR and compare
+        id: compare
+        run: |
+          set +e
+          result=$(pytest-benchmark compare)
+          echo "$result"
+          echo "result<<EOF" >> $GITHUB_OUTPUT
+          echo "$result" >> $GITHUB_OUTPUT
+          echo "EOF" >> $GITHUB_OUTPUT
+
+      # Post a comment to the PR with results
+      - name: Comment on PR
+        uses: marocchino/sticky-pull-request-comment@v2
+        with:
+          header: benchmark
+          message: |
+            ## 🏎️ Benchmark Comparison
+            ```
+            ${{ steps.compare.outputs.result }}
+            ```

benchmarks/test_awkward_bench.py

@@ -26,7 +26,7 @@
 ]
 
 
-def test_fromfile_and_to_awkward_benchmark_collective(benchmark):
+def test_fromfile_and_to_awkward(benchmark):
     """Benchmark LHEFile.fromfile and to_awkward conversion across all test files."""
 
     def fromfile_and_to_awkward_all_files(filepaths):