Upgrading to newer Chromium Version causes non-stable tests

[jo-hoff]

I've encountered issues while attempting to use Loki.js for visual regression testing with a newer Chromium version. I opted for the Docker image yukinying/chrome-headless-browser-stable:132.0.6834.110 due to the aging default image (yukinying/chrome-headless-browser-stable:118.0.5993.117).

To accommodate the new Docker image, I had to set the remoteDebuggerPort to 9222 in target-chrome-docker, as the updated image doesn't support random port assignment. This limitation stems from the removal of the remote-debugging-address flag in Chromium. Consequently, socat is employed to redirect all traffic from port 9222 to 9223, which is the specified remote-debugging-port in the new image.

Upon implementing the newer version, I've observed instability in test results. Specifically, I executed the commands one after the other:

1. loki update --verboseRenderer --reactUri file:./storybook-static --requireReference
2. loki test --verboseRenderer --reactUri file:./storybook-static --requireReference

Numerous tests fail due to slight discrepancies in element positioning. Within browser/src/get-selector-box-size.js, the getBoundingClientRect function often returns slightly different values (typically a variance of 1-2 pixels) for identical elements, leading to frequent test failures.

We are currently utilizing Storybook version 8.3.0

Has anyone experienced similar issues, can reproduce this problem, or has suggestions for resolution? Your insights would be greatly appreciated.

[Anxhul10]

yes facing similar issues , when tests are run locally it passes but when i use these test with github actions test fails , maybe due to different docker enviroment in virtual machine

[Anxhul10]

yes facing similar issues , when tests are run locally it passes but when i use these test with github actions test fails , maybe due to different docker enviroment in virtual machine

I think this is solved when i use chrome.docker in package.json test locally and on CI run loki test on docker action and temporarily chage chrome.docker to chrome.app .

also on CI pass --chromeConcurrency=2 or maybe 1 .