Add Node.js support for `--import` flag by raphael-theriault-swi · Pull Request #3416 · open-telemetry/opentelemetry-operator

raphael-theriault-swi · 2024-11-01T18:01:19Z

Description:

This adds a new useImport option for Node.js autoinstrumentation that injects and --import flag to NODE_OPTIONS instead of the default --require flag. The behaviour with the provided image is the same as what --require does with the added feature of also registering hooks to instrument ESM code.

This also opens the door for custom images to run asynchronous code in their instrumentation code, which isn't possible with --require.

Link to tracking Issue(s):

Resolves: Support --import flag for Node.js instrumentation #3414

Testing:

Added both unit tests and a new e2e test that exercise the feature.

Documentation:

The new option is documented and was added to the generated docs.

.chloggen/node-import.yaml

config/manager/kustomization.yaml

swiatekm

This looks good to me overall, but I'd feel more confident if one of @open-telemetry/javascript-maintainers could have a look at the JS specific parts.

bundle/community/manifests/opentelemetry-operator.clusterserviceversion.yaml

trentm · 2024-11-06T22:18:42Z

apis/v1alpha1/instrumentation_types.go

+	// UseImport overrides the default injected --require flag with an --import flag that supports ESM.
+	// Requires Node.js 18 or later.
+	// +optional
+	UseImport bool `json:"useImport,omitempty"`


One subtlety that might impact the choice of the config var name.
The feature for the user is whether a module import hook is added (via module.register()) to support instrumentating ESM code.

That is actually independent of whether --import something.mjs or --require something.js is used to run the OTel setup when Node.js is starting up. It is definitely possible to call module.register("@opentelemetry/instrumentation/hooks.mjs"); from the CommonJS autoinstrumentation.js (loaded via --require).

So I wonder if a config name something like instrumentEsm or something like that would be clearer.

There are a few subtleties discussed at open-telemetry/opentelemetry-js#4933, but I don't think you need to read and grok all that. Eventually we'd like it so that there is just the one obvious way to setup OTel -- and that would be via --import some-esm-file-that-calls-module.register.mjs. However, for now we need to make the ESM support opt-in because:

it requires newer versions of node

it is possible the user explicitly does not want ESM instrumentation enabled because it is still new and can have overhead and bugs

I think instrumentEsm here would be a bit of a misnomer, given that functionality could be added without an import flag, and a custom image could also require the use of an import flag for top level await support (which is actually our usecase for this addition) without providing ESM instrumentation

a custom image could also require the use of an import flag for top level await support (which is actually our usecase for this addition)

Ah, I misunderstood the use case then. Using import in the var name makes sense then. The docs for this feature should mention the two things then: (1) --import ... is used to load "autoinstrumentation.mjs" (so custom versions can use ESM and top-level await), (2) the default "autoinstrumentation.mjs" differs from "autoinstrumentation.js" in that it registers the hook for ESM instrumentation.

trentm · 2024-11-06T22:20:25Z

.chloggen/node-import.yaml

+# Use pipe (|) for multiline entries.
+subtext: |
+  The new UseImport configuration overrides the default injected --require flag with an --import flag that supports ESM.
+  Requires Node.js 18 or later.


The important Node.js feature here is whether module.register() is supported and that version range is ^18.19.0 || ^20.6.0 || >=22. It would be good to be specific here, so some user of an earlier minor version of v18 or v20 doesn't expect this to work.

trentm · 2024-11-06T22:30:27Z

autoinstrumentation/nodejs/src/autoinstrumentation.mts

+import { register } from "node:module";
+register("@opentelemetry/instrumentation/hooks.mjs");


The current code will crash if used on a verison of node that doesn't yet have module.register:

% cat foo.mjs import { register } from "node:module"; console.log('reg', register); % /Users/trentm/.nvm/versions/node/v18.0.0/bin/node foo.mjs file:///Users/trentm/.../foo.mjs:1 import { register } from "node:module"; ^^^^^^^^ SyntaxError: The requested module 'node:module' does not provide an export named 'register' at ModuleJob._instantiate (node:internal/modules/esm/module_job:128:21) at async ModuleJob.run (node:internal/modules/esm/module_job:194:5) at async Promise.all (index 0) at async ESMLoader.import (node:internal/modules/esm/loader:409:24) at async loadESM (node:internal/process/esm_loader:85:5) at async handleMainPromise (node:internal/modules/run_main:61:12) Node.js v18.0.0

An option might be to gracefully warn (untested code):

Suggested change

import { register } from "node:module";

register("@opentelemetry/instrumentation/hooks.mjs");

import { diag } from '@opentelemetry/api';

import * as module from 'node:module';

if (typeof module.register === 'function') {

module.register('@opentelemetry/instrumentation/hooks.mjs');

} else {

diag.warn(`OpenTelemetry Operator auto-instrumentation could not instrument ESM code: Node.js ${process.version} does not support 'module.register()'`);

}

Does the SDK setup a logger for the diag API by default ? If not I'm thinking this should probably be a console.warn

The NodeSDK sets a DiagConsoleLogger if the OTEL_LOG_LEVEL env var is set. (Per open-telemetry/opentelemetry-js#3693 it does not set a diag logger at all if the env var isn't set at all. So by default logging is effectively off.) node -r @opentelemetry/auto-instrumentations-node/register ... does turn it on by default (https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/metapackages/auto-instrumentations-node/src/register.ts#L23-L26).

Using the diag logger is the intended mechanism for logging by the OTel JS packages, but that doesn't necessarily have to mean that the OTel Operator uses it. I don't have a strong opinion whether the diag logger or console.* is used here.

raphael-theriault-swi · 2024-11-08T20:24:18Z

@trentm Thoughts on the updated wording ?

trentm

I think the JS parts of this look good now. (I haven't reviewed the test code at all.)

swiatekm · 2024-11-21T11:24:04Z

@raphael-theriault-swi could you regenerate the manifests? You have some conflicts there.

swiatekm · 2024-11-21T16:42:42Z

@raphael-theriault-swi looks like you have a slightly malformed changelog entry. Can you have a look? It's complaining about tabs vs spaces.

raphael-theriault-swi · 2024-11-21T17:04:57Z

This should do it

swiatekm

@raphael-theriault-swi it looks like the new e2e test you added is consistently failing. Can you have a look?

My suspicion is that it's because we don't actually build the autoinstrumentation images for e2e tests, and your new test depends on the image having the mjs file.

raphael-theriault-swi · 2024-11-22T16:32:12Z

Yeah that was also my assumption, I'm not sure what could be done on my side to fix that as I'm far from familiar with the test machinery.

pavolloffay · 2024-11-22T16:34:56Z

autoinstrumentation/nodejs/src/autoinstrumentation.mts

@@ -0,0 +1,10 @@
+import "./autoinstrumentation.js";


@raphael-theriault-swi how does the --import flag with with the upstream opentelemetry/auto-instrumentations-node ? We are trying to minimize the nodejs code and rely only on that package.

Related ticket: #3465

@trentm could you please take a look here? cc) @JamieDanielson

@pavolloffay Are you asking if node --import @opentelemetry/auto-instrumentations-node/register ... works? If so, I think yes it should work fine because node --import something ... works with something being either ESM or CommonJS code. I haven't tried it, though.

However, whether to move the operator to using @opentelemetry/auto-instrumentations-node instead of the bootstrap code in "autoinstrumentation.js" doesn't need to be part of this PR, right?

I would also add that the instrumentation.js file needs to exist. Adding a --require @opentelemetry/auto-instrumentations-node/register option alone will not work, since this requires the package to be present in the node_modules search path of the application itself, which would require the application to have the package installed and defeat the entire purpose of the operator autoinstrumentation.

The contents of the file could be replaced by require("@opentelemetry/auto-instrumentations-node/register") instead, which would not conflict with this PR whatsoever.

However, whether to move the operator to using @opentelemetry/auto-instrumentations-node instead of the bootstrap code in "autoinstrumentation.js" doesn't need to be part of this PR, right?

The issue is that, if we keep adding new features here it will be harder to migrate them to auto-instrumentations-node (or other node packages)

Like I mentioned above switching to auto-instrumentations-node would be not require any change to the code added by this PR now or in the future.

@pavolloffay Are there any changes you would like to see with regards to the switch auto-instrumentations-node ? I can make that part of the PR, but again either way it wouldn't really change any files this PR touches.

trentm · 2024-11-22T18:12:36Z

docs/api.md

+        <td><b>useImport</b></td>
+        <td>boolean</td>
+        <td>
+          UseImport overrides the default injected --require flag with an --import flag.


I think this documentation should be expanded a bit. It is relevant that useImport: true also changes to using the autoinstrumentation.mjs file (rather than autoinstrumentation.js). Anyone providing a custom image for nodejs injection should be aware of that.

It's documented in the Dockerfile which is where the related info about autoinstrumentation.js was, I don't feel strongly about it either way but I wasn't sure whether documenting the internal filenames made sense in the user-facing documentation.

raphael-theriault-swi · 2025-02-18T14:23:00Z

@swiatekm @pavolloffay Would this be more likely to get merged if I also transition the rest of the image to using auto-instrumentations-node ?

e2digital-eurostar · 2025-06-18T22:54:08Z

Hello, interested in seeing this merged! Is any help required to get it over the line?

swiatekm · 2025-06-24T10:04:11Z

Hello, interested in seeing this merged! Is any help required to get it over the line?

I think the change has been approved conceptually, but in the meantime the nodejs instrumentation package has moved to using auto-instrumentations-node. So @raphael-theriault-swi would need to rebase this PR on main.

On a more positive note, we now run E2E tests on instrumentation images built from PRs, so if this breaks anything obviously, we'll know.

raphael-theriault-swi · 2025-06-25T16:11:38Z

I'll rebase when I have some free time, thanks for the ping !

raphael-theriault-swi · 2025-07-08T16:21:09Z

@swiatekm Rebased on latest main, biggest difference being that the default image no longer instruments ESM code with useImport: true since that would require some custom JS code and all of the existing one has been removed. I can add that feature back if needed.

github-actions · 2025-07-08T16:40:08Z

E2E Test Results

34 files ±0 225 suites +4 2h 21m 51s ⏱️ + 18m 40s
89 tests +1 88 ✅ ±0 0 💤 ±0 1 ❌ +1
229 runs +4 227 ✅ +2 0 💤 ±0 2 ❌ +2

For more details on these failures, see this check.

Results for commit 1fea4ce. ± Comparison against base commit 3d66a4c.

♻️ This comment has been updated with latest results.

swiatekm · 2025-07-08T17:13:35Z

@pavolloffay can you have a final look before we merge?

raphael-theriault-swi · 2025-07-14T16:07:07Z

quick bump @pavolloffay

e2digital-eurostar · 2025-10-23T16:46:24Z

Once again requesting a merge for this... 🥲

swiatekm · 2025-10-29T10:48:40Z

@raphael-theriault-swi looks like the E2E tests you added are failing, can you have a look? Once that's done, we can merge. Thank you for your patience!

raphael-theriault-swi requested a review from a team as a code owner November 1, 2024 18:01

iblancasa reviewed Nov 4, 2024

View reviewed changes

.chloggen/node-import.yaml Outdated Show resolved Hide resolved

swiatekm reviewed Nov 5, 2024

View reviewed changes

config/manager/kustomization.yaml Outdated Show resolved Hide resolved

swiatekm reviewed Nov 6, 2024

View reviewed changes

bundle/community/manifests/opentelemetry-operator.clusterserviceversion.yaml Outdated Show resolved Hide resolved

trentm reviewed Nov 6, 2024

View reviewed changes

trentm mentioned this pull request Nov 6, 2024

use module.register(...) in recommended bootstrap code for ESM support open-telemetry/opentelemetry-js#4933

Open

trentm approved these changes Nov 8, 2024

View reviewed changes

trentm mentioned this pull request Nov 8, 2024

add support for OTel Operator useImport: true option elastic/elastic-otel-node#432

Open

raphael-theriault-swi requested a review from swiatekm November 11, 2024 17:33

swiatekm requested a review from iblancasa November 21, 2024 17:48

swiatekm approved these changes Nov 21, 2024

View reviewed changes

swiatekm self-requested a review November 22, 2024 14:54

swiatekm reviewed Nov 22, 2024

View reviewed changes

pavolloffay requested changes Nov 22, 2024

View reviewed changes

trentm reviewed Nov 22, 2024

View reviewed changes

raphael-theriault-swi mentioned this pull request Jan 29, 2025

REQUEST: New membership for raphael-theriault-swi open-telemetry/community#2541

Closed

6 tasks

pavolloffay approved these changes Mar 27, 2025

View reviewed changes

raphael-theriault-swi force-pushed the node-import branch from da2827e to a8d3f28 Compare July 8, 2025 16:00

Add Node.js support for --import flag

bc3a331

raphael-theriault-swi force-pushed the node-import branch from a8d3f28 to bc3a331 Compare July 8, 2025 16:18

raphael-theriault-swi requested a review from swiatekm July 8, 2025 16:19

swiatekm approved these changes Jul 8, 2025

View reviewed changes

swiatekm requested a review from pavolloffay July 8, 2025 17:13

Merge branch 'main' into node-import

1fea4ce

pavolloffay approved these changes Oct 28, 2025

View reviewed changes

		import { register } from "node:module";
		register("@opentelemetry/instrumentation/hooks.mjs");

-import { register } from "node:module";
-register("@opentelemetry/instrumentation/hooks.mjs");
+import { diag } from '@opentelemetry/api';
+import * as module from 'node:module';
+if (typeof module.register === 'function') {
+    module.register('@opentelemetry/instrumentation/hooks.mjs');
+} else {
+    diag.warn(`OpenTelemetry Operator auto-instrumentation could not instrument ESM code: Node.js ${process.version} does not support 'module.register()'`);
+}

Conversation

raphael-theriault-swi commented Nov 1, 2024

Uh oh!

Uh oh!

Uh oh!

swiatekm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

trentm Nov 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raphael-theriault-swi Nov 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raphael-theriault-swi commented Nov 8, 2024

Uh oh!

trentm left a comment

Choose a reason for hiding this comment

Uh oh!

swiatekm commented Nov 21, 2024

Uh oh!

swiatekm commented Nov 21, 2024

Uh oh!

raphael-theriault-swi commented Nov 21, 2024

Uh oh!

swiatekm left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raphael-theriault-swi commented Nov 22, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raphael-theriault-swi Nov 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raphael-theriault-swi commented Feb 18, 2025

Uh oh!

e2digital-eurostar commented Jun 18, 2025

Uh oh!

swiatekm commented Jun 24, 2025

Uh oh!

raphael-theriault-swi commented Jun 25, 2025

Uh oh!

raphael-theriault-swi commented Jul 8, 2025

Uh oh!

github-actions bot commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Test Results

Uh oh!

trentm Nov 6, 2024 •

edited

Loading

raphael-theriault-swi Nov 7, 2024 •

edited

Loading

swiatekm left a comment •

edited

Loading

raphael-theriault-swi Nov 22, 2024 •

edited

Loading

github-actions bot commented Jul 8, 2025 •

edited

Loading