Skip to content

Concat required libs#2

Open
samvimes01 wants to merge 10 commits intometarhia:mainfrom
samvimes01:feat/concat_required_libs
Open

Concat required libs#2
samvimes01 wants to merge 10 commits intometarhia:mainfrom
samvimes01:feat/concat_required_libs

Conversation

@samvimes01
Copy link

@samvimes01 samvimes01 commented Dec 24, 2025

  • add mode field to build.json with app, iife and default lib modes
  • add libDir field to build.json to treat cases like metacom/dist
  • skip import lines alongside with require
  • refactor - split code to separate concerns, improved naming, update readme
  • add config validation with metaschema
  • add various tests
  • changed filename comment to #region comment to allow collapse of code region in vscode-like editors

metarhia/metautil#311

  • tests and linter show no problems (npm t)
  • tests are added/updated for bug fixes and new features
  • code is properly formatted (npm run fix)
  • description of changes is added in CHANGELOG.md
  • update .d.ts typings

- add `require` field to `build.json` for node_modules deps that are concatenated first
- add `libDir` field to `build.json` to treat cases like `metacom/dist`
- skip `import` lines alongside with `require`
- add test for including required libs from node_modules
- changed filename comment to `#region` comment to allow collapse of code region in vscode-like editors

metarhia/metautil#311
Copy link
Member

@tshemsedinov tshemsedinov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build steps in short

  1. npm library build
    1.1. Remove require/import of local sub-modules
    1.2. Convert module.exports to export
    1.3. Concatenate library files but not dependencies
  2. application build
    2.1. Add links to node_modules files from server /static/ folder
    2.2. Alternatively application can use any bundler to build ours npm files in traditional way

`// Version ${packageJson.version} ${packageName} ${licenseName}\n\n`;
const requiredLibs = (buildConfig.require || [])
.map((lib) => {
const fileName = `./node_modules/${lib}/${lib}.mjs`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not build dependencies from node_modules

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added iife build mode (separate commit - easy to revert).
This will include deps from node_modules to prepare lib for worker. We will need it for metacom in serviceworker

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need a time to find better solution

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left same comment here metarhia/metacom#528 (comment)

As for the iife mode there are not so many options I can think about.
1 - the one used here - include whole lib from deps since metarhia build don't treeshake
2 - use 3d party builder like esbuild rollup rolldown to build iife that includes only required classes/functions
3 - drop service worker support for Firefox and use ESM modules same as in main thread no need in iife.
4 - or make a special iife file for each lib that uses metarhia-build (metautil, probably metaschema, etc.) and force end user to include importScripts('metautil.iife.js') that exposes global metautilIIFE variable to SW context.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied changes - as we discussed in telegram @tshemsedinov

After Firefox fixed bug with service worker imports https://bugzilla.mozilla.org/show_bug.cgi?id=1360870

Changed the builder

  • lib mode builds lib.mjs file and adds metarhia deps imports like import metautil from './metautil.js' - .js
  • then in app mode symlinks .mjs as .js so end user app will fnd correct esm files served from static folder

Rebuild corresponding metacom and matautil PRs
metacom - (to master branch) metarhia/metacom#528
metautil - (to created by you build branch) https://github.com/metarhia/metautil/pull/315/changes
also there is a metachema - (to master) metarhia/metaschema#480

@samvimes01
Copy link
Author

samvimes01 commented Dec 29, 2025

I've changed how deps are processed. No need for require field in build json.
If it's node: deps - lib build will skip this line and logs warning.
For node_modules deps it will find all such deps and put import from only once at the top of the bundle.

I've added mode field to build.json

  1. New app mode.
    When builder is used in the app with app mode it will link libs listed in the order field to application/static folder

  2. New iife mode.
    When lib is required to run inside worker context where ES modules unavailable.
    This wraps lib in iife and concat it with node_modules deps instead of importing such deps. (workers throws errors when spot import or export keyword)

Research repo -
https://github.com/samvimes01/js-workers-examples?tab=readme-ov-file#references

Paper Repo PR
Part 1: Building metarhia stack bundler. bundler
Part 2: Refactoring Metacom: Achieving Single Source of Truth metacom bundler
Part 3: Service Workers and Metacom: IIFE Bundles in Action metacom service worker

Slides:
https://docs.google.com/presentation/d/189hazIpNgWe3vUhf6HYzocx3V3695gjjzuByP5rRmKE/edit?usp=sharing

- process file as a whole instead line by line to simplify multiline import processing
- hide mode executors in modes/index.js
- fixed use strict regex
- break execution when not from console (e.g. load module by impress)
- user must import deps before module
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants