Debugging cypress open performance

[amenk]

EDIT Issue opened: #26691

I am not posting this as a bug, because I don't have a minimum reproducible example yet.

We recently upgraded from 9.6.0 to 12 and migrated the specs using the wizard. In 9.x we did not notice this slow cypress open performance.

In a fresh Cypress project with 12.11.0 which I opened once, did not add any specs and closed it, and then I execute
DEBUG=cypress:* CYPRESS_BASE_URL=https://example.com yarn cypress:open --e2e --browser chrome it takes less ~5 seconds for the "Create your first spec" be visible.

In my migrated project with around 20 specs it takes 25 seconds (sometimes even ~ 45 seconds). This is really annoying as we are constantly switching the BASE URL and restart often.

We have some plugins installed (percy, deploysentinel) and a special fixture loader in the config, but this does not seem the one which takes long.

In the debug log I see the following:

  cypress:lifecycle:EventRegistrar plugin event registered? { eventName: 'before:browser:launch', isRegistered: false } +2s
  cypress:server:appdata path: /home/user/.config/Cypress/cy/production/extensions +6s
  cypress:server:util:process_profiler current & mean memory and CPU usage by process group:
  cypress:server:util:process_profiler ┌─────────┬───────────────────┬──────────────┬──────────────────────────────────┬────────────┬────────────────┬──────────┬──────────────┬─────────────┐
  cypress:server:util:process_profiler │ (index) │       group       │ processCount │               pids               │ cpuPercent │ meanCpuPercent │ memRssMb │ meanMemRssMb │ maxMemRssMb │
  cypress:server:util:process_profiler ├─────────┼───────────────────┼──────────────┼──────────────────────────────────┼────────────┼────────────────┼──────────┼──────────────┼─────────────┤
  cypress:server:util:process_profiler │    0    │     'cypress'     │      1       │             '217762'             │   20.51    │     10.25      │  664.41  │    496.02    │   664.41    │
  cypress:server:util:process_profiler │    1    │ 'electron-shared' │      4       │ '217764, 217926, 217765, 218397' │     0      │       0        │  282.2   │    190.43    │    282.2    │
  cypress:server:util:process_profiler │    2    │     'plugin'      │      1       │             '218217'             │     0      │       0        │  152.06  │    152.06    │   152.06    │
  cypress:server:util:process_profiler │    3    │    'launchpad'    │      1       │             '218434'             │     0      │       0        │  108.95  │    108.95    │   108.95    │
  cypress:server:util:process_profiler │    4    │      'other'      │      2       │         '218890, 218891'         │     0      │       0        │   2.48   │     2.47     │    2.48     │
  cypress:server:util:process_profiler │    5    │      'TOTAL'      │      9       │               '-'                │   20.51    │     10.26      │ 1210.11  │    819.43    │   1210.11   │
  cypress:server:util:process_profiler └─────────┴───────────────────┴──────────────┴──────────────────────────────────┴────────────┴────────────────┴──────────┴──────────────┴─────────────┘ +12s

With DEBUG=* there are lots and lots such lines during the time when nothing seems to happen:

cypress-verbose:packherd:trace Loaded "/home/user/.cache/Cypress/12.11.0/Cypress/resources/app/packages/data-context/node_modules/chokidar/lib/constants.js" (cache | exports) -> moduleCache: 776, exportsCache: 4048, loaded: 773 +0ms

Could that be the reason?
What does this mean?

What should I look for in our specs / plugins?

EDIT: I tried to make a minimal example by deleting most of the specs and config. Finally I replaced cypress/ and cypress.* by the working (fast) set of files from my fresh project to the project causing problems. Still the start up takes long.
So the reason must be somewhere else in the project.

Is cypress maybe scanning the full project for specs, not only cypress/ ? The project has quite a lot of other files (Magento)... 🤔

EDIT2: it seems to add watchers for each and every folder in the Magento project

Our project has ~ 176K files in 43K folders most of them are not related to Cypress:

What I did is to start cypress, pick the process ID of the CPU consuming Cypress process and called

$ sudo strace -e trace=desc -p 338942

The spits out tons of inotify adds:

inotify_add_watch(42, "/home/user/company/projects/example.com/var/view_preprocessed/frontools/pub/static/frontend/Foo/bar/Magento_Email/email", IN_MODIFY|IN_ATTRIB|IN_MOVED_FROM|IN_MOVED_TO|IN_CREATE|IN_DELETE|IN_DELETE_SELF|IN_MOVE_SELF) = 38987

Even if I use

 --config watchForFileChanges=false

there are still lots of statx calls on all my folders.

[lmiller1990]

cypress-verbose:packherd:trace Loaded "/home/user/.cache/Cypress/12.11.0/Cypress/resources/app/packages/data-context/node_modules/chokidar/lib/constants.js" (cache | exports) -> moduleCache: 776, exportsCache: 4048, loaded: 773 +0ms

This is to do with something called v8 snapshots - we speed things up by sticking a bunch of commonly used modules in memory, so each require call does not have to read a file from disc. This wouldn't be the bottleneck.

In my migrated project with around 20 specs it takes 25 seconds (sometimes even ~ 45 seconds).

Something is definitely not right here, the Cypress repo itself has projects with 100s of specs and opens nice and quick - globbing for specs should be very fast. Can you share your migrated project's cypress.config and any additional info you can provide?

[amenk]

My current config is "empty":

const { defineConfig } = require("cypress");

module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      // implement node event listeners here
    },
  },
});

Because the startup is nearly as slow if I replace the full cypress/ folder, cypress* files, package.json and so node_modules with the one from a freshly initialized cypress. So the cause must be somewhere else in the project. is it somehow scanning the full project for javascript files or so? So saw there is a fileServer config which is per default the project root, but I did not manage to change this config.

Can any other files outside cypress/ influence the startup?

[amenk]

I updated my question, Cypress seems to scan / watch the full project it is embedded in, wondering how this can be avoided.

[amenk]

I opened and issue: #26691

[amenk]

As a workaround I can put cypress and the node modules in a subfolder:

mkdir cypress-project
mv cypress* cypress-project/
cp package.json yarn.lock cypress-project/
cd cypress-project/
yarn

Then it opens fast ...

But we'd rather like to avoid that.

UPDATED workaround

mkdir cypress-project
cd cypress-project
ln -s ../cypress
ln -s ../cypress.config.ts
ln -s ../cypress.env.json
ln -s ../node_modules/
ln -s ../package.json

Now start cypress from this folder and it opens super-fast.

[lmiller1990]

Neat trick! We should fix this properly, though. I'll post in your issue.