[Hackathon] Benchmark GHA by mnajdova · Pull Request #21222 · mui/mui-x

mnajdova · 2026-02-04T16:25:11Z

Based on top of #21220

…nardobelchior/mui-x into bernardobelchior/perf-benchmark-tool

… events

mui-bot · 2026-02-04T16:26:54Z

Deploy preview: https://deploy-preview-21222--material-ui-x.netlify.app/

Bundle size report

Bundle	Parsed size	Gzip size
@mui/x-data-grid	0B^(0.00%)	0B^(0.00%)
@mui/x-data-grid-pro	0B^(0.00%)	0B^(0.00%)
@mui/x-data-grid-premium	0B^(0.00%)	0B^(0.00%)
@mui/x-charts	0B^(0.00%)	0B^(0.00%)
@mui/x-charts-pro	0B^(0.00%)	0B^(0.00%)
@mui/x-charts-premium	0B^(0.00%)	0B^(0.00%)
@mui/x-date-pickers	0B^(0.00%)	0B^(0.00%)
@mui/x-date-pickers-pro	0B^(0.00%)	0B^(0.00%)
@mui/x-tree-view	0B^(0.00%)	0B^(0.00%)
@mui/x-tree-view-pro	0B^(0.00%)	0B^(0.00%)

Details of bundle changes

Generated by 🚫 dangerJS against c41ee72

… events

…n/gh-action-master-performance

This reverts commit 6ad6eae.

github-actions · 2026-02-09T06:21:12Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Copilot

Pull request overview

This pull request adds a comprehensive benchmark tool and GitHub Actions workflows for performance testing of MUI X components. The implementation includes a Next.js-based benchmark application using Playwright for automated testing, along with CI/CD workflows to track and compare performance metrics over time.

Changes:

Added benchmark tool infrastructure with Next.js app, Playwright tests, and React Profiler integration
Implemented GitHub Actions workflows for running benchmarks on PRs and storing baseline data
Created CI scripts for comparing benchmark results and generating performance reports

Reviewed changes

Copilot reviewed 37 out of 38 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
`.github/workflows/benchmark-comparison.yml`	Workflow for comparing PR benchmarks against master baseline
`.github/workflows/benchmark-baseline.yml`	Workflow for storing baseline benchmark data on master branch
`test/benchmark-tool/utils/*`	Utility functions for profiling, reporting, and page navigation
`test/benchmark-tool/ci-scripts/*`	Scripts for fetching, comparing, and reporting benchmark data
`test/benchmark-tool/app/*/`	Benchmark test pages and tests for various components
`test/benchmark-tool/*.{json,ts}`	Configuration files for Next.js, Playwright, and TypeScript
`pnpm-lock.yaml`	Dependency updates including Playwright version bump

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-14T11:45:39Z

.github/workflows/benchmark-baseline.yml

+  pull_request_target:
+    types:
+      - opened
+      - synchronize
+      - reopened


The workflow uses pull_request_target which runs in the context of the base repository with write permissions. This is a security risk because it could allow untrusted code from a PR to access secrets and write to the repository. Consider using only pull_request or implement proper security measures.

Suggested change

pull_request_target:

types:

- opened

- synchronize

- reopened

This worries me. I think we should move to the pull_request + workflow_run pattern. pull_request executes bench and stores artifact, low permissions. Workflow_run gathers artifacts and posts PR comment.

Shouldn't it be pull_request_target if we want it to have write access to a branch in the main repo? Else it will only have access to the users branch, which is not how our contribution model works

Why does it need write access to a branch in the main repo? It should execute the benchmark and stores the results as an artefact.

I got confused, this is for master, then it shouldn't need pull_request nor pull_request_target as it should run on push to master only.

I'm inclined to store master's results in an orphan branch on this repo

This comment wouldn't work when using pull_request only

It's only for pull_requests, master branch is handled by the push action

push (only on master): run benchmarks, store in orphan branch

pull_request (only onPRs`): run benchmark, store in artefact

workflow_run (when PR workflow finishes): download artefact + master result for merge-base, compare, post PR comment

Ah ok, got it

Copilot · 2026-02-14T11:45:40Z

test/benchmark-tool/ci-scripts/fetchMasterMetrics.ts

+    const contentsResponse = await fetch(
+      'https://api.github.com/repos/mnajdova/performance-benchmark-data/contents/mui-x',


The URL is hardcoded to a personal GitHub repository 'mnajdova/performance-benchmark-data'. For production use, this should be configurable via environment variables or point to an official organization repository to avoid dependency on a personal account.

To avoid external repos and keeping the process self-contained, I'm inclined to store master's results in an orphan branch on this repo. PR results could live as GHA artefacts with 90 day retention.

.github/workflows/benchmark-comparison.yml

Copilot · 2026-02-14T11:45:40Z

test/benchmark-tool/utils/Profiler.tsx

+
+function onRender(id: string, phase: 'mount' | 'update' | 'nested-update', actualDuration: number) {
+  const startTime = performance.now() - actualDuration;
+  (window as any)[CAPTURE_RENDER_FN]?.({


Using (window as any) bypasses TypeScript's type safety. Consider defining a proper type for the window object with the custom property, or at least check if the function exists before calling it to avoid potential runtime errors if the function isn't exposed yet.

test/benchmark-tool/playwright.config.ts

Copilot · 2026-02-14T11:45:41Z

test/benchmark-tool/utils/goToPage.ts

+const pageState = new WeakMap<Page, PageState>();
+
+export function getRouteFromFilename(filename: string): string {
+  return path.dirname(filename).split('/app').pop()!;


The non-null assertion operator (!) is used on line 15 with .pop()! which assumes that the split will always produce at least one element. If the filename doesn't contain '/app', this will return undefined and cause a runtime error. Consider adding validation or a more robust path extraction method.

Copilot · 2026-02-14T11:45:42Z

test/benchmark-tool/next.config.ts

+  // Enable React profiling in production builds
+  reactProductionProfiling: true,
+  productionBrowserSourceMaps: true,
+  experimental: {


The configuration disables Turbopack minification with turbopackMinify: false. While this may be intentional for benchmarking purposes to get consistent results, consider adding a comment explaining why minification is disabled, as it's unusual for production configurations.

Suggested change

experimental: {

experimental: {

// Disable Turbopack minification to keep benchmark results consistent and easier to analyze.

Copilot · 2026-02-14T11:45:42Z

.github/workflows/benchmark-comparison.yml

+  pull_request_target:
+    types:
+      - opened
+      - synchronize
+      - reopened


The workflow uses pull_request_target which runs in the context of the base repository with write permissions. This is a security risk because it could allow untrusted code from a PR to access secrets and write to the repository. The workflow should either use only pull_request (which runs in a restricted context) or implement proper security measures to prevent malicious PRs from exploiting the write permissions.

Suggested change

pull_request_target:

types:

- opened

- synchronize

- reopened

Copilot · 2026-02-14T11:45:42Z

test/benchmark-tool/utils/reporter.ts

+      console.warn(
+        `High coefficient of variation (${(coefficientOfVariation * 100).toFixed(1)}%) for event "${getEventKey(event)}". ` +
+          `Mean: ${meanDuration.toFixed(2)}ms, StdDev: ${stdDev.toFixed(2)}ms. Results may be unreliable.`,
+      );


The warning uses console.warn which will be visible in the output, but on line 19 console.log is disabled via eslint comment while console.warn is not. For consistency, either allow both or suppress both console methods depending on the intended behavior.

Copilot · 2026-02-14T11:45:42Z

.github/workflows/benchmark-baseline.yml

+          # Extract just the repository name (after the /)
+          REPO_DIR=$(echo "$REPO_NAME" | cut -d'/' -f2)


The comment on line 91 says "Extract just the repository name (after the /)" but then uses it to create a directory. This assumes the repository name doesn't contain characters that could cause issues in directory names. Consider sanitizing the repository name to ensure it's safe for use in file paths.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Jan Potoms <2109932+Janpot@users.noreply.github.com>

Janpot · 2026-02-17T09:16:56Z

test/benchmark-tool/ci-scripts/fetchMasterMetrics.ts

+
+    const latestFolder = folders[0];
+
+    console.warn(`Using latest benchmark folder: ${latestFolder}`);


Using latest benchmark from master is incorrect. it's a moving target. You may be comparing with different commits at different points in time. Instead you should calculate the merge-base with the master branch and use that commit for comparison.

Therefore you must store data by commit. I'm changing data layout to

# store results by commit benchmarks/commits/{SHA}.json # append only, store the same results in jsonl, batched per month benchmarks/monthly/{YYYY-MM}.jsonl

JCQuintas · 2026-02-17T21:43:28Z

Just a note that we deliberately cut corners, so this is not really in a review state

Janpot · 2026-02-18T06:15:58Z

I was asked to take this to the finish line. Just documenting the changes I'm making.

JCQuintas · 2026-02-18T09:16:23Z

I was asked to take this to the finish line. Just documenting the changes I'm making.

Nvm then. From what you wrote it seemed you were just reviewing it.

github-actions · 2026-03-20T15:21:41Z

This pull request has been inactive for 30 days. Please remove the stale label or leave a comment to keep it open. Otherwise, it will be closed in 15 days.

bernardobelchior and others added 14 commits February 4, 2026 15:49

Hackathon - Benchmark Tool

ad563e8

New commit for Connor

76d17eb

Start structure

d51148b

Start structure

d483215

Merge branch 'bernardobelchior/perf-benchmark-tool' of github.com:ber…

391017a

…nardobelchior/mui-x into bernardobelchior/perf-benchmark-tool

hackaton benchmark gha

5bc4ecd

Fix version

64fe375

Convert to Next.js

299050b

fix master issue

1c889db

Refactor benchmark test to use dynamic route for navigation

562bd81

Try to fix head

6208458

fix pr

8d3c448

Implement Profiler component and update benchmark tests for rendering…

e1d445f

… events

initial commit gh action

71bc4b3

mnajdova added scope: code-infra Involves the code-infra product (https://www.notion.so/mui-org/5562c14178aa42af97bc1fa5114000cd). type: new feature Expand the scope of the product to solve a new problem. labels Feb 4, 2026

mnajdova and others added 13 commits February 4, 2026 17:30

git commit --allow-empty

4bfd629

try to fix

3cc8c30

d6d2640

try fix

c9297d0

split data in multiple directories per repo

6f67078

Hackathon - Benchmark Tool

465b9ec

New commit for Connor

10f6fb2

Start structure

81046f3

Convert to Next.js

90a9d25

Refactor benchmark test to use dynamic route for navigation

a0383b9

Implement Profiler component and update benchmark tests for rendering…

0c3219c

… events

Save the report to the benchmarks/ dir

a6af89f

Update pnpm-lock

a3ed64d

JCQuintas and others added 16 commits February 5, 2026 15:50

delete

9cec224

fix metadata

d48d2b1

Add tracing

3374939

fix issue

11ae51e

Merge branch 'master' into bernardobelchior/perf-benchmark-tool

3ccd5db

Merge commit '3ccd5dbec3a0d39d96a9bdc4b8d57f0ca48fe4ef' into hackatho…

9764324

…n/gh-action-master-performance

fix pnpm script

46fe52b

fix markdown

2816303

fix output generation

6ad6eae

fix silent

86c818f

Revert "fix output generation"

8871c26

This reverts commit 6ad6eae.

2f0739f

improve comparison units

01d4672

cf6b945

respect threshold

6ab0995

display threshold

64c5a9f

github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged. label Feb 9, 2026

Janpot requested a review from Copilot February 14, 2026 11:41

Copilot started reviewing on behalf of Janpot February 14, 2026 11:42 View session

Copilot AI reviewed Feb 14, 2026

View reviewed changes

Janpot changed the title ~~[Hackaton] Benchmark GHA~~ [Hackathon] Benchmark GHA Feb 16, 2026

Update test/benchmark-tool/playwright.config.ts

c41ee72

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Jan Potoms <2109932+Janpot@users.noreply.github.com>

Janpot mentioned this pull request Feb 17, 2026

[code-infra] Benchmark GHA revisited #21349

Open

Janpot reviewed Feb 17, 2026

View reviewed changes

github-actions bot added the stale Inactive for 7 days (issues) or 30 days (PRs); closed after 5 or 15 more days if no update. label Mar 20, 2026

		const contentsResponse = await fetch(
		'https://api.github.com/repos/mnajdova/performance-benchmark-data/contents/mui-x',

	experimental: {
	experimental: {
	// Disable Turbopack minification to keep benchmark results consistent and easier to analyze.

		# Extract just the repository name (after the /)
		REPO_DIR=$(echo "$REPO_NAME" \| cut -d'/' -f2)


		const latestFolder = folders[0];

		console.warn(`Using latest benchmark folder: ${latestFolder}`);

Uh oh!

Conversation

mnajdova commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mui-bot commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bundle size report

Uh oh!

github-actions bot commented Feb 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Janpot Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

JCQuintas Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

Janpot Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

JCQuintas Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

Janpot Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

JCQuintas Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Janpot Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Janpot Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JCQuintas commented Feb 17, 2026

Uh oh!

Janpot commented Feb 18, 2026

Uh oh!

JCQuintas commented Feb 18, 2026

Uh oh!

github-actions bot commented Mar 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

mnajdova commented Feb 4, 2026 •

edited

Loading

mui-bot commented Feb 4, 2026 •

edited

Loading

Janpot Feb 17, 2026 •

edited

Loading