extractSourceMap takes a huge amount of RAM on boot

[NikhilVerma]

I run a series of workers on my Temporal pod bootup. They sometimes take up over 2GB of RAM simply from parsing the source maps. I've confirmed this using clinic.js.

sdk-typescript/packages/worker/src/worker.ts

Line 1868 in 5473ebd

const [actualCode, sourceMapJson] = extractSourceMap(code);

I am curious if there is a way to optionally allow turning off source maps? I can do so via the builder

	const { code } = await bundleWorkflowCode({
		workflowsPath: worker.workflowsPath,
		// Remove source maps
		webpackConfigHook: config => {
			config.devtool = false;
			return config;
		}
	});

But then the extractSourceMap throws, and if I patch it I see further failures down the line in addWorkflowBundle > SourceMapConsumer

I managed to patch it (in a dirty way)

    return [code, JSON.stringify({
            version: 3,
            file: '',
            sources: [],
            mappings: '',
    })];
    // throw new Error("Can't extract inlined source map from the provided Workflow Bundle");

Which results in significantly lower bootup + IDLE ram usage

Before

After

[SamuelMS]

Have you considered using workflowBundle over workflowsPath for production use? e.g. compile the bundle in CI/CD ahead of time with Temporal's helper, then use that bundle with workflowBundle: { codePath: '/app/worker/dist/workflows.cjs' }.

import { bundleWorkflowCode } from '@temporalio/worker';
import { writeFile } from 'fs/promises';
import path from 'path';
import { fileURLToPath } from 'url';

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

async function bundle() {
  const { code } = await bundleWorkflowCode({
    workflowsPath: path.resolve(__dirname, './src/tasks/workflows.ts'),
  });
  const codePath = path.resolve(__dirname, './dist/workflows.cjs');

  await writeFile(codePath, code);
  console.log(`Bundle written to ${codePath}`);
}

bundle().catch((error) => {
  console.error('Bundle compilation failed:', error);
  process.exit(1);
});

Frankly, I'm not sure if this'll help your RAM usage with sourcemaps, but we haven't noticed any spikes in RAM with the above approach (for a fairly massive app, at that).

[NikhilVerma]

@SamuelMS Actually I've tested this exclusively on the workflow bundle (because the source maps parsing happens on the bundle as well)

https://github.com/temporalio/sdk-typescript/blob/5473ebd3442d00624fb0ed20fa82f8d5bd78e8fd/packages/worker/src/worker.ts#L729C1-L736C8

You would primarily see this issue if you run multiple workers in the same machine, this the default strategy before we can start having worker specific pods. My workaround has been to load workers synchronously with a global.gc call in between them, it has kept the memory pressure under 1.8GB for now which works for us for the time being.

[SamuelMS]

@SamuelMS Actually I've tested this exclusively on the workflow bundle (because the source maps parsing happens on the bundle as well)

https://github.com/temporalio/sdk-typescript/blob/5473ebd3442d00624fb0ed20fa82f8d5bd78e8fd/packages/worker/src/worker.ts#L729C1-L736C8

You would primarily see this issue if you run multiple workers in the same machine, this the default strategy before we can start having worker specific pods. My workaround has been to load workers synchronously with a global.gc call in between them, it has kept the memory pressure under 1.8GB for now which works for us for the time being.

We run multiple workers on our nodes, but use PM2 in cluster mode to manage them: https://pm2.keymetrics.io/docs/usage/cluster-mode/