[Browser] Memory profiler to find tests having issues

[Eldraed]

Hello,

I am currently experiencing multiple issues with browser testing. We are using Vitest to run our Storybook interaction tests thanks to the addon they made to use Vitest's browser capability.

Sadly, after adding a lot of new tests, our CI is struggling and failing on WebKit mostly, and sometimes on Firefox.
When I look at our resource charts, we see that it goes to 100% easily, and I'm wondering if that's the cause:

We are running with:

• isolate: false
• fileParallelism: false

This is to be faster, as we have a lot of tests and isolation is so slow... (even more on WebKit).

The issues I think we have are:

• Potential side effects related to overlay testing
• WebKit going too fast between tests (weird when you know that Chromium is faster overall)
• Potential memory overflow (unable to run tests anymore, finishing with Caused by: Error: [birpc] rpc is closed, cannot call "createTesters")

To identify some tests that might be using too much memory, I wanted to do some heap profiling using --heap-prof via execArgv, but from what I understand, it's stopped by this condition for browsers:

vitest/packages/vitest/src/node/pool.ts

Lines 111 to 114 in 372e86f

	if (pool === 'browser') {
	browserSpecs.push(...specs)
	continue
	}

I also found a reporter for Playwright that could help with memory/test analysis for the browser (https://github.com/cenfun/monocart-reporter?tab=readme-ov-file#monocart-reporter), but as it's a Playwright reporter, we are not able to use it. 😕

I've done some search on the vitest/playwright repositories to find eventual solutions, but hard to find one that would work for both.

Do you have any suggestions on how we could check memory usage or profile the browser tests to better find tests that are having issues?

Thanks in advance! 🙂

[tt-a1i]

Memory profiling for browser tests is tricky. Here are some approaches:

1. Chrome DevTools Protocol (CDP) for Chromium

Enable remote debugging and connect to take heap snapshots.

2. Performance observer in tests

Track memory before/after each test using performance.memory (Chromium only) to identify tests that allocate significantly more memory.

3. For the birpc is closed error:

This usually means the browser crashed. Try:

• isolate: false
• fileParallelism: false
• poolOptions: { threads: { maxThreads: 1 } }
• Increased timeouts (30000ms)

4. Binary search for problematic tests:

Run subsets to narrow down which tests cause issues.

5. CI memory monitoring:

Add a background job to log browser process memory usage over time.

The Storybook test addon context makes standard profiling harder - consider a feature request on the Storybook repo for better memory debugging support.

[Eldraed]

Hey, thanks for the answer, sadly it concerns mainly Webkit, Chromium working fine and fast 😕
It's even harder with Webkit... I tried a lot of different stuff from your list, including the profiler in headed mode locally just to try to better understand what is going on, but the profiler is not capable to handle it and anyway it would have been on the Vitest UI + render, not the best.

So 1./2. are not our target, regarding 3., It works better for sure with fileParallelism at false, but it's way slower. I might still consider it with some job parallelisation instead of workers.

4. Might not help as less tests will go green easily due to less memory usage. It only happens since we added a lot more tests lately... We were already having a lot of memory usage, but now it's too much 😅

5. I have already what I shown from CircleCI, I should check if there is more than that for better details 🤔 But to be able to locate a specific "spike", I need to have also the timestamp of the tests and within Vitest, I didn't find the possibility to get the timestamp in the reporter for each test run. Did I miss it ?

Thanks anyway for the answer, it makes me think at least that I tried already a lot of stuff to find the issue 🙏

[tt-a1i]

Hey, thanks for the answer, sadly it concerns mainly Webkit, Chromium working fine and fast 😕 It's even harder with Webkit... I tried a lot of different stuff from your list, including the profiler in headed mode locally just to try to better understand what is going on, but the profiler is not capable to handle it and anyway it would have been on the Vitest UI + render, not the best.

So 1./2. are not our target, regarding 3., It works better for sure with fileParallelism at false, but it's way slower. I might still consider it with some job parallelisation instead of workers.

4. Might not help as less tests will go green easily due to less memory usage. It only happens since we added a lot more tests lately... We were already having a lot of memory usage, but now it's too much 😅
5. I have already what I shown from CircleCI, I should check if there is more than that for better details 🤔 But to be able to locate a specific "spike", I need to have also the timestamp of the tests and within Vitest, I didn't find the possibility to get the timestamp in the reporter for each test run. Did I miss it ?

Thanks anyway for the answer, it makes me think at least that I tried already a lot of stuff to find the issue 🙏

for webkit specifically, profiling is rough - no devtools access like chrome.

what might work:

1. add timestamps with a setup file and match against your CI resource graph:

beforeEach((ctx) => {
console.log([${Date.now()}] START ${ctx.task.name});
});
afterEach((ctx) => {
console.log([${Date.now()}] END ${ctx.task.name});
});

2. if you can access playwright context through the storybook addon, try enabling tracing - but not sure if that's exposed.

3. honestly fastest path is probably binary search - comment out half your stories, see if it still crashes, narrow down.

the birpc error means the browser process died, so you're looking for a story that either leaks memory or loads something huge.

[Eldraed]

Thanks for the tip on the beforeEach/afterEach, I didn't even thought about this simple thing 😂

I can add the tracing through Vitest directly, but when using tracing, it goes slower (as it records the page) and doesn't trigger the issues 😕

I can re-try binary search, but I need better profiling to find the tests that might be responsible of that and it's currently hard to have something good enough.

I hope that we will be able to do something similar to the --heap-prof on browsers later !

[tt-a1i]

Thanks for the tip on the beforeEach/afterEach, I didn't even thought about this simple thing 😂

I can add the tracing through Vitest directly, but when using tracing, it goes slower (as it records the page) and doesn't trigger the issues 😕

I can re-try binary search, but I need better profiling to find the tests that might be responsible of that and it's currently hard to have something good enough.

I hope that we will be able to do something similar to the --heap-prof on browsers later !

haha yeah the slowdown hiding the bug is classic 😅 hope the timestamp logs + circleci graphs help you spot the spike. good luck!