RFC: Lazy make for reexports in side effects free barrel file

[ahabhgk]

Summary

Lazily build reexports in side-effect-free barrel files to minimize performance overhead from excessive module inclusion, avoiding unnecessary resolve and build processes for these modules.

Motivation

Optimize Rspack's build performance when processing barrel files, with the main benefits seen in project scenarios that import libraries, improving Rspack's overall build performance.

For case import { Button } from "antd"; Button; (release build of rspack and added resolving/building log when module starts resolve/build):
before: compiled in 528ms, 12689 lines of resolving/building logs
after: compiled in 82ms, 1003 lines of resolving/building logs

Prior Arts

• Parcel: Deferring Assets
• Turbopack: skip processing for some specific barrel file

Glossary

• barrel file: a file that imports modules from other files and exports the said modules

  export { value as A } from "./A";
  export { default as B } from "./B";
  export * as C from "./C";
  export * from "./D";
  module.exports.E = require("./E").value;
  module.exports.F = require("./F");
  _export_star(require("./a"), exports); // _export_star is from @swc/helpers

User Guide

• Controlled by experiments.lazyBarrel during experimental phase.
• Will be enabled by default when sideEffects is turned on after stable, this feature depends on factory_meta.side_effects_free ("sideEffects": false in package.json or configured by module.rule.sideEffects) and thus requires sideEffects configuration.

const rspackConfig = {
  // ...
  module: {
    rules: [{
      test: /specific-lib/,
      sideEffects: true, // add `factory_meta.side_effects_free = true` for specific module
    }],
  },
  optimization: {
    sideEffects: "flag" // or true
  },
  experiments: {
    lazyBarrel: true,
  }
}

Detailed Design

1. When importing specifiers from barrel file modules, the imported specifiers are collected as forwarded_ids and passed to the barrel file module.
2. When collecting dependencies during barrel file parsing, if the barrel file module's factory_meta.side_effects_free is true, all re-export dependencies will be marked as lazy.
3. After the barrel file is built, the used re-export dependencies will be determined based on the forwarded_ids passed in. The used re-export dependencies will have their lazy mark removed and be added to the process dependencies queue for building, while unused re-export dependencies will remain lazy and won't be built (no resolve, no build, and no loaders...).
   1. To maintain consistency with the dependency relationships when this feature is disabled, all dependencies (whether lazy or not) will be stored in the barrel file's corresponding mgm.all_dependencies. The data storage remains completely identical - the only difference being some dependencies are marked as lazy.
   2. These lazy-marked dependencies will also be marked as weak (weak-marked dependencies will be skipped during rendering if there are no strong (non-weak) references, ensuring these modules won't appear in the final output). This guarantees correctly rendering (avoiding issues like panics when failing to get modules from dependencies).
   3. The process of "The used re-export dependencies will have their lazy mark removed and be added to the process dependencies queue for building" will be abstracted into a new Task: ProcessUnlazyDependenciesTask.
   4. The barrel file's export dependencies need to be stored as an artifact to ensure correctness during incremental builds and hot builds with persistent cache enabled.

4. When another module imports the barrel file again, the barrel file will be revisited. During the revisit, the previously collected re-export dependencies from the barrel file build will be matched against the forwarded_ids. Those that matched will be considered as used, have their lazy mark removed, and be added to the process dependencies queue for building.

5. forwarded_ids will first attempt to match against known named exports (export { x as A } from "./a";, export * as A from "./a", export { a, b }, export const A, export default A). When no match is found, it indicates the specifier comes from star reexports (export * from "./a"), at which point the remaining star reexports will have their lazy marks removed and be added to the process dependencies queue for building.

6. Re-exports can still continue to pass forwarded_ids downstream for subsequent barrel files to use. export { x as A } from "./a" will pass forwarded_ids: x downstream, and the subsequent barrel file will only build the re-export of x while keeping other re-export dependencies lazy. export * from "./a" and export * as A from "./a" will pass forwarded_ids: * downstream, requiring the subsequent barrel file to build all re-export dependencies.

Drawbacks

• Modules marked as lazy will skip processes like factorize, resolve, build, and loader execution
  • These modules won't trigger related hooks or loaders, meaning side effects in plugins and loaders won't execute (given the differences between Rspack's incremental make and webpack's make, there have been no reported issues caused by unexecuted side effects in plugins and loaders)
  • Related errors won't be reported either, such as: resolve errors caused by missing modules, loader errors thrown during loader execution, etc.

FAQ

1. Can this also supports CommonJS?

   Yes, theoretically this can also support CommonJS, but before that we need to improve Rspack's CJS tree shaking first, the static analyze part specifically, so currently we will only support ESM, and support CJS in the future PR.

2. Why depend on "sideEffects": false in package.json, can Rspack automatically analyze a module is side effects free or not?

   Rspack has the ability to analyze a module is side effects free or not of course, it's already used by optimization.sideEffects to tree shaking side effects free modules. But these analyzed modules still require checking whether their dependencies have side effects - only when all dependencies are also side effects free can the module be considered truly side effects free. However, during the make phase, dependencies must first be built before their side effects can be analyzed, and the lazyBarrel is intend to avoid to build those dependencies. "sideEffects": false in package.json or module.rule.sideEffects doesn't require dependency checking, as it signifies the entire package is side effects free. Therefore, lazyBarrel can only rely on this marker rather than automatic analysis.

3. So export * from "./x" is not a problem any more?

   No, export * from "./x" is still a bad practice for your build performance, lazyBarrel won't build the star re-export only when you import a specifier, and the side effects free barrel file's named exports contains the specifier (export { x as A } from "./a";, export * as A from "./a", export { a, b }, export const A, export default A (the name is default)), which means the star re-export is not used. So Rspack will still build the star re-export if:

   1. barrel file is not side effects free
   2. the star re-export is used

[ahabhgk]

Implementation PR: #11117

[rikisamurai]

Hi, just wondering — would this kind of change still cause a rebuild of all files in util.ts

// util.ts
export * from './date'
export * from './number'
export * from './price'

// re-export.ts
export * from './util'

// main.ts
import { foo } from 're-export'

[ahabhgk]

Unfortunately yes, in this case Rspack can't locate the foo export in util.ts, so it has to build ./date, ./number, and ./price to locate the foo export.

As long as Rspack can locate the foo export in ./util.ts, there's no need to build ./util.ts's dependencies:

For example in this case Rspack will only build ./date, and won't build ./number, and ./price.

//util.ts
export { foo } from './date'
export * from './number'
export * from './price'

Or in this case Rspack even won't build ./util and its dependencies, unless there is another bar exported from ./util and imported by ./main.ts.

// re-exports.ts
export const foo = 1
export * from './util'