Skip to content

build: restructure for dual ESM/CJS output, update tooling, and enhance utilities#426

Closed
dimaslanjaka wants to merge 99 commits intohexojs:masterfrom
dimaslanjaka:esm-shim
Closed

build: restructure for dual ESM/CJS output, update tooling, and enhance utilities#426
dimaslanjaka wants to merge 99 commits intohexojs:masterfrom
dimaslanjaka:esm-shim

Conversation

@dimaslanjaka
Copy link
Contributor

@dimaslanjaka dimaslanjaka commented Jul 25, 2025

check list

  • Add test cases for the changes.
  • Passed the CI test.

Description

Summary

This PR modernizes the build, testing, and linting infrastructure while introducing utility enhancements and ensuring compatibility with both ESM and CommonJS consumers.

Changes

Why

To align with modern Node.js packaging practices, ensure smooth compatibility across ESM and CJS consumers, streamline the build/test/lint process, and expand utility functionality.

Notes

  • Build now produces consistent dual outputs with proper exports handling
  • Tests and linting run in a more maintainable, ESM-friendly setup
  • New JSON utilities simplify handling of circular structures

- Added Rollup config to generate both ESM (`.mjs`) and CJS (`.js`) builds
- Defined `exports` field in `package.json` with explicit paths for import, require, and types
- Updated `tsconfig.json` to output to `tmp/dist` for staging Rollup input
- Included additional devDependencies: Rollup plugins and Babel for bundling
- Refined build/clean scripts for integrated TypeScript + Rollup workflow
@dimaslanjaka dimaslanjaka changed the title build: add Rollup bundling and dual module support build: support both ESM and CommonJS via Rollup Jul 25, 2025
Replaced CommonJS `require` with `JSON.parse(path.join(...))` to load JSON in a way compatible with ESM builds. This fixes runtime errors when the ESM bundle is used in environments expecting `require` to work, ensuring `highlight_alias.json` loads correctly in both CJS and ESM contexts.
Replaced CommonJS `require` with `JSON.parse(path.join(...))` to load JSON in a way compatible with ESM builds. This fixes runtime errors when the ESM bundle is used in environments expecting `require` to work, ensuring `highlight_alias.json` loads correctly in both CJS and ESM contexts.
Copy link
Member

@SukkaW SukkaW left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a fan of using rollup directly (hard to setup, needs to manually install many dependencies). Probably use bunchee instead?

@dimaslanjaka
Copy link
Contributor Author

dimaslanjaka commented Jul 28, 2025

Not a fan of using rollup directly (hard to setup, needs to manually install many dependencies). Probably use bunchee instead?

can bunche target bundles to older node versions? like babel.
remembering hexo works from node 12.
or strict version of node to make unsupported for old node version?

…rt ESM and CJS

Replaced all `module.exports` / `export =` syntax with `export`/`export default` syntax.
This allows the codebase to be compatible with both ECMAScript Modules (ESM) and CommonJS (CJS) environments.
…parately

- Updated `package.json` to define separate `main` (CJS) and `module` (ESM) entry points.
- Replaced Rollup-based build with separate TypeScript builds for CJS and ESM using `tsconfig.cjs.json` and `tsconfig.esm.json`.
- Removed `rollup.config.cjs` and related dependencies (`rollup`, `@rollup/*`, `rollup-plugin-dts`).
- Simplified `build` and `clean` scripts accordingly.
- Ensured `exports` field in `package.json` is compliant with dual package support for Node.js.
@dimaslanjaka dimaslanjaka changed the title build: support both ESM and CommonJS via Rollup build: support both ESM and CommonJS Jul 28, 2025
@dimaslanjaka
Copy link
Contributor Author

ok, now i change to default hybrid typesript compiler. without rollup. this also work on ESM, but sometimes need using babel to transform for htmlparser2 which isnt yet supported for ESM

- Moved `types` field into the `exports` map for better ESM/CJS compatibility
- Added `highlight_alias.json` to exports for external access
@dimaslanjaka
Copy link
Contributor Author

dimaslanjaka commented Jul 28, 2025

might need shim export for backward-compatibility import, this script will automate map export like older hexo-util.

file scripts/build_exports.mjs

import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const packageJsonPath = path.join(__dirname, '../package.json');
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf-8'));

const defaultExports = {
  '.': {
    'import': './dist/esm/index.js',
    'require': './dist/cjs/index.js',
    'types': './dist/esm/index.d.ts'
  },
  './highlight_alias.json': './highlight_alias.json',
  './dist/highlight_alias.json': './highlight_alias.json'
};

fs.readdirSync(path.join(__dirname, '../lib')).forEach(file => {
  defaultExports[`./dist/${file.replace('.ts', '')}`] = {
    'import': `./dist/esm/${file.replace('.ts', '.js')}`,
    'require': `./dist/cjs/${file.replace('.ts', '.js')}`,
    'types': `./dist/esm/${file.replace('.js', '.d.ts')}`
  };
});

// Sort the exports to ensure consistent output
const sortedExports = Object.keys(defaultExports).sort().reduce((obj, key) => {
  obj[key] = defaultExports[key];
  return obj;
}, {});

packageJson.exports = sortedExports;

fs.writeFileSync(packageJsonPath, JSON.stringify(packageJson, null, 2) + '\n', 'utf-8');

sample result at dimaslanjaka/hexo-util

- Added `./dist/highlight_alias.json` export to support legacy import paths
- Ensures compatibility without changing file structure
- Assigned `rewire()` result to a separate variable for clarity
- Applied `.default ||` fallback to support both default and named exports
- Updated `__set__` calls to use the original rewire module object
@coveralls
Copy link

coveralls commented Jul 28, 2025

Coverage Status

coverage: 73.017% (-23.9%) from 96.875%
when pulling bfaacfd on dimaslanjaka:esm-shim
into d497bc7 on hexojs:master.

…esolution

- Replaced manual `fs.readFileSync` and `path.join` with direct JSON import
- Enabled `resolveJsonModule` in base `tsconfig` for cleaner JSON usage
- Removed redundant `resolveJsonModule` from `tsconfig.esm.json`
…read

- Switched from TypeScript JSON module import back to `fs.readFileSync` for broader compatibility
- Disabled `resolveJsonModule` in `tsconfig.base.json`
- Ensured `highlight_alias.json` is copied during build and ignored in all directories via `.gitignore`
- Added test cases for `stripIndent` option in highlight output
- Verified ESM import works via a new `.mjs` test script
@dimaslanjaka dimaslanjaka requested a review from SukkaW July 28, 2025 06:43
- Moved `cp highlight_alias.json` into `build:highlight` script for clarity
- Ensured `build:highlight` is called as part of the main `build` sequence
- Cleaned up redundant script declaration
- Rewrote `build_highlight_alias.js` as `build_highlight_alias.mjs` using ES module syntax
- Removed legacy CommonJS script and replaced its usage in build pipeline
- Maintains backward compatibility by writing output to both root and `dist/` directories
…circular reference support

Replaced the 'deepmerge' dependency with a custom deepMerge implementation that supports deep cloning,
array and object merging, circular reference handling, and a configurable recursion depth limit to prevent
maximum call stack errors. This refactor reduces external dependencies and improves control over merge behavior.
…ap, Set, Function)

Enhanced the custom deepMerge and deepClone utilities to handle additional JavaScript built-in types:

- ✅ Added support for deep cloning and merging of:
  - `Date`: returns a clone of the most recent date
  - `RegExp`: clones the source's regular expression
  - `Map`: merges entries recursively by key
  - `Set`: performs a union of both sets
  - `Function`: prefers the source function without cloning

- ♻️ Updated type guards and object detection logic to handle these cases properly
- 🧪 Added comprehensive test coverage for all supported special types

This improves robustness when merging complex data structures and ensures consistency across edge cases.
- Move the `convertCjs` function from `test/utils.cjs` to `scripts/pretest.mjs` and invoke it before running tests.
- Remove redundant `convertCjs` calls from test files (`highlight_direct.cjs`, `highlight.spec.ts`, `spawn.spec.ts`).
- Clean up `test/utils.cjs` by removing the `convertCjs` function and related imports.
- Ensure CJS file extension conversion and import path rewriting are handled in a single place, improving maintainability and reducing test
Replaced `jsonStringifyWithCircular` and `jsonParseWithCircular` with concise aliases `jsonStringify` and `jsonParse`. Updated exports, README examples, and tests to use new function names for clarity and consistency, while preserving compatibility through internal aliasing.
… globals

- Removed unused `path` and `fileURLToPath` imports from ESLint config
- Added `eslint.config.mjs` to lint ignore list
- Enforced `import/extensions` rule for `.js` and `.ts` files
- Added explicit globals for test environment (`describe`, `it`, `beforeEach`, etc.)
- Added `eslint-import-resolver-node` dependency for import resolution
- Updated `tsconfig.json` to allow JS imports and resolve JSON modules
- Implemented `getDirname` and `getFilename` to provide CJS-like `__dirname` and `__filename` in ESM across platforms
- Added internal regex-based path parsing for POSIX and Windows
- Exported new utilities from `lib/index.ts`
- Added unit tests to validate correct behavior and ensure returned values are non-empty and end with `.ts` or `.js`
…tructure and CJS compatibility

- Moved all individual exports from `index.ts` to new `index-exports.ts`
- Updated `index.ts` to re-export everything from `index-exports.ts` and provide a default export object
- Added CommonJS `module.exports` fallback for compatibility
@D-Sketon
Copy link
Member

The PR is too large; splitting it into smaller ones might be better.

@D-Sketon
Copy link
Member

Not a fan of using rollup directly (hard to setup, needs to manually install many dependencies). Probably use bunchee instead?

can bunche target bundles to older node versions? like babel.
remembering hexo works from node 12.
or strict version of node to make unsupported for old node version?

The next major version of Hexo will drop Node 18. Given this, do we still need Babel?

…index.ts

- Removed `index-exports.ts` and moved all utility exports directly into `index.ts`
- Replaced `tsconfig.cjs.json` and `tsconfig.esm.json` builds with a unified `tsup` config
- Updated `package.json` scripts to use `tsup` for building
- Simplified highlight alias build script by removing manual ESM/CJS output handling
- Adjusted tests and utils to reflect new dist file locations and bundler naming behavior
@dimaslanjaka dimaslanjaka changed the title build: support both ESM and CommonJS build: restructure for dual ESM/CJS output, update tooling, and enhance utilities Jul 31, 2025
- add build.js to handle tsup build and patch .cjs imports for local files
- remove tsup.config.js in favor of build.js
- update build script in package.json to use build.js instead of tsup CLI
- update pretest.mjs to reference build.js and package.json instead of tsup.config.js
@dimaslanjaka
Copy link
Contributor Author

The PR is too large; splitting it into smaller ones might be better.

because supporting both ESM and CJS, we need refactor all codebases like (import with extension, drop require, pollyfil __dirname, refactor tests for mocha suitable with ESM environment, etc)

@dimaslanjaka
Copy link
Contributor Author

The next major version of Hexo will drop Node 18. Given this, do we still need Babel?

emm no need babel, but when still support ESM and CJS, still need for bundle typescript to both environment safely without rewriting codebase for ESM and CJS independently.

- Introduced `lib/cross_dirname.ts` providing `getDirname` and `getFilename` utilities
- Works in both CommonJS and ES modules without relying on Node.js globals
- Handles path extraction from error stack for platform independence
- Added README documentation and usage examples
- Added unit tests to validate `getDirname` and `getFilename` behavior
@dimaslanjaka
Copy link
Contributor Author

dimaslanjaka commented Jul 31, 2025

@D-Sketon another approach without editing too much codebase to supporting esm and cjs

  • test unit using jest which has built-in transformer
  • bundler using rollup (without babel when targeting node 20+ just using plugin json,commonjs,module resolver)
    but @SukkaW not a fan of rollup bundler. so, editing all codebases too ESM and CJS independently is the solution.

when using tsup, need .cjs patch to fix ESM compiled file imported under cjs compiled file (like this PR do)

@dimaslanjaka
Copy link
Contributor Author

dissapointed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants