esm: add support for loaders

[travi]

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

I'm excited to start migrating projects of mine that are tested with cucumber over to native ESM. However, as mentioned in #1304 (comment), loaders do not appear to work with the recent release candidate. I'm unsure if this is intended to work at this point, but I wasnt able to figure it out on my own.

instead, i meet the following error:

(node:62231) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension "" for /path/to/project/node_modules/@cucumber/cucumber/bin/cucumber-js
    at defaultGetFormat (internal/modules/esm/get_format.js:71:15)
    at getFormat (file:///path/to/project/node_modules/quibble/lib/quibble.mjs:65:12)
    at Loader.getFormat (internal/modules/esm/loader.js:104:42)
    at Loader.getModuleJob (internal/modules/esm/loader.js:242:31)
    at async Loader.import (internal/modules/esm/loader.js:176:17)
    at async Object.loadESM (internal/process/esm_loader.js:68:5) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

Describe the solution you'd like
A clear and concise description of what you want to happen.

ideally, the cucumber-js binary would accept --loader=<loader-name> directly

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

an alternative that should work is providing the loader through NODE_OPTIONS, like NODE_OPTIONS="--loader=testdouble" cucumber-js ...

Additional context
Add any other context or screenshots about the feature request here.

As requested in #1304 (comment), I've created a fairly minimal reproducible example at https://github.com/travi-test/cucumber-esm/.

The error can be demonstrated by running npm run test:integration (after npm install, of course). If the --loader=testdouble is removed from the test:integration:base script, the error goes away. In this minimal example that results in the tests passing because I'm not actually using the loader.

In a more realistic project, this obviously results in the loader not working, resulting in failures. An example of a more real project can be found in the cucumber-esm branch of https://github.com/pragmatic-divops/cli where I'm attempting to begin the process of converting the project to ESM.

[davidjgoss]

@travi thanks for raising. I think it does make sense to add something, I'll aim to get something done for the next RC (though I note that loaders are still experimental in Node, so any support on our side would be experimental too).

[travi]

I think it does make sense to add something, I'll aim to get something done for the next RC

great! i'm happy to try it out and provide feedback

loaders are still experimental in Node, so any support on our side would be experimental too

understood. i think that is important to keep visible, but is in line with how i've seen other projects handle the current state of things, so i don't expect that to present a problem for adoption

[pk]

Hello everyone, I'm hitting a massive wall with the ESM and CommonJS module loading. Here is a bit of background...

I'm testing Angular application using Cucumber & Playwright. In order to do this I want to use Angular Material test harnesses which encapsulate the selectors and interaction into an API. I wrote the Playwright adapter for Angular Material harnesses and all was fine till Angular 13....

In Angular 13 they stopped distributing CommonJS build and only distribute ESM build and now the whole test suite fails because Cucumber-JS uses CommonJS imports but Angular only exports in ESM.

From what I understand, if CucumberJS will load code using ESM all will work again because I can set my testsuite to compile (TS) to ESM, change my Playwright Adapter to ESM and load ESM Angular.

Is there any viable / experimental branch I can give a shot and possibly help? I really need this to work and I'm more than happy to work on it.

[aurelien-reeves]

Did you tried v8.0.0-rc.2?

Using esm code with it should work as-is. you may just have to rename a potential cucumber.js file into cucumber.cjs

[davidjgoss]

+1 to @aurelien-reeves comment above, please do give that a go

I can set my testsuite to compile (TS) to ESM

I think this might be where you hit a snag, if you are compiling TypeScript on the fly, because we don't have support for ESM loaders yet (which is what this issue is, I'll rename it for clarity).

[pk]

@davidjgoss & @aurelien-reeves I've now managed to run the whole thing on ESM modules. It has been pretty much rough ride but the Cucumber-JS was not really an issue.

What I did is that at the moment I'm compiling the TypeScript code before running a test suite avoiding the TS-Node for now.

Thanks for your work.

[aurelien-reeves]

Thanks for your feedback @pk

[travi]

this comment and following conversation appear to apply very directly to the situation here. linking here in case details of that conversation can help with understanding what a reasonable path forward might be in the short/long term.

[davidjgoss]

Just following up on this now I understand the state of loaders a bit better.

ts-node and others support a --loader option on their own CLIs by doing all the work in a child process with that option passed through via NODE_OPTIONS. This has similarities with how we manage parallel running, but still would be a non-trivial change which I don't think we should commit to until/unless the loaders API becomes stable without a less friction-y way to do this.

However we should definitely aim to work when a loader has been registered with the process outside of Cucumber, and will work on that as a priority (with thanks to @travi for the detailed report and test repo).

[testgitdl]

Hello, We're still having a problem with Angular 13. Tried to change the cucumber.js config to use ESM but still get the following error when trying to run.
In package.json we have "type": "module" and the error is:
"Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: D:\project\projects\playwright\cucumber.js
require() of ES modules is not supported.
require() of D:\project\projects\playwright\cucumber.js from D:\project\node_modules@cucumber\cucumber\lib\cli\profile_loader.js is an ES module file as it is a .js file whose nearest parent package.json
contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename cucumber.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from D:\project\projects\playwright\package.json."

The project is based on https://github.com/Tallyb/cucumber-playwright.

ESM Cucumber.js now looks like:

const getWorldParams = () => {
  const params = {
    foo: 'bar',
  };

  return `--world-parameters ${JSON.stringify({ params })}`;
};
const common = {
  requireModule: ['ts-node/register'],
  require: ['src/**/*.ts', 'support/config.ts'],
  worldParameters: `${getWorldParams()}`
}
export default {
  ...common,
  format: ['json:reports/report.json', 'html:reports/cucumber-html-report.html', 'summary', 'progress-bar', '@cucumber/pretty-formatter'],
}