npm run build keeps lots of jest-worker child process and eating up all memories after some builds

[hkmsadek]

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
Platform: linux
Arch: x64
Version: #58-Ubuntu SMP Thu Oct 13 08:03:55 UTC 2022
Binaries:
Node: 16.17.0
npm: 8.15.0
Yarn: 1.22.19
pnpm: N/A
Relevant packages:
next: 13.0.3-canary.1
eslint-config-next: N/A
react: 18.2.0
react-dom: 18.2.0

What browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

Describe the Bug

We have been running some npm run builds commands as we are publishing updates 5/6 times a day, we noticed that
at, npm run build leaves lot of jest-worker child process instances like the below image.

It's eating all the memory and getting locked.

Expected Behavior

It shouldn't eat lot of memories and child instances should get deleted after build?

Link to reproduction

it's a production app not sure how to add

To Reproduce

just running npm run build multiple times in a large projects will leave this issue

[hkmsadek]

[icyJoseph]

Do you run tests as part of your build process? Do you observe the same by just running npx next build? Also your screenshots are not visible.

[mohas]

We also have this issue, we do not have any test runs in our deployment runs

[icyJoseph]

AFAIK, then this might be something related to the actual Workers used by Next.js to build pages in parallel. So this has nothing to do with tests, this is just a product made by the jest team to run work in parallel.

What next.js version are you running? Are you using any special next.config.js experimentals?

[icyJoseph]

With:

const nextConfig = {
  reactStrictMode: true,
  swcMinify: true,
  experimental: {
    workerThreads: true,
    cpus: 4,
    sharedPool: true,
  },
};

And building 1000 pages on build I still can't get this issue, my CPU and RAM go back down to regular levels after build and there's no jest-worker processes running.

I am in MacOS though... could you try to provide a minimal reproduction demo?

[mohas]

we are using [email protected], btw what does sharedPool do in this config?

[icyJoseph]

I still can't trigger this 😞 Wonder if it is OS related. I don't know about these experimental flags, I just started to play with them, given that they affect workers at build time, at least from inspecting the code it seems that way.

[mohas]

Ok thank you, I will try some more to see what may trigger these, will post more if I see something

[IronDdtRIiGh]

I run it s subserver issntsi.u.c.e.m- abcdefghighlmnopqrstuvwxy.! Apptsdinue. e. u*^+

…

On Wed, Nov 9, 2022, 5:36 PM Marsha Elledge ***@***.***> wrote: On Wed, Nov 9, 2022, 4:28 PM Mohas ***@***.***> wrote: > Of course, we will work something out when I get to the office tomorrow > > — > Reply to this email directly, view it on GitHub > <#42668 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/A4BZRNKYXWUGQ5XN72R62S3WHQJPVANCNFSM6AAAAAAR3FUUTY> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> >

[brandonfake69]

Running into this issue as well. Crashes my server. Tried adding the following to my docker-compose file with no luck:

  NODE_OPTIONS: --max-old-space-size=512
  NEXT_TELEMETRY_DISABLED: 1

[brandonfake69]

Also tried adding some optimizations to next.config.ts (also didn't work):

const nextConfig: NextConfig = {
  // Reduce memory usage during build
  experimental: {
    workerThreads: false,
    cpus: 1,
    optimizePackageImports: ["@google-cloud/kms", "twilio"],
  },

  // Disable source maps in production to save memory
  productionBrowserSourceMaps: false,

  // Reduce bundle size
  swcMinify: true,

  // Limit webpack memory usage
  webpack: (config, { isServer }) => {
    if (!isServer) {
      config.optimization = {
        ...config.optimization,
        splitChunks: {
          chunks: "all",
          cacheGroups: {
            vendor: {
              test: /[\\/]node_modules[\\/]/,
              name: "vendors",
              chunks: "all",
            },
          },
        },
      };
    }

    // Limit memory usage
    config.optimization = {
      ...config.optimization,
      moduleIds: "deterministic",
    };

    return config;
  },
};