[DRAFT] Various improvements to the MFE build process

[regisb]

I'm listing here several changes I've experimented to improve the build process of MFEs. This is work-in-progress as I'm trying out different things. But the PRs listed below should be good for a review.

• Exclude node_modules/ from babel-loader: feat: faster babel-loader by excluding node_module frontend-build#644
• Get rid of BundleAnalyzer feat: get rid of bundle analyzer on by default in prod frontend-build#646
• Trim lodash and fortawesome imports:
  • frontend-build: feat: lighter build by rewriting lodash imports frontend-build#645
  • footer: feat: avoid pulling entire lodash library into build frontend-component-footer#515
  • ora-grading: feat: lighter build by avoiding full lodash import frontend-app-ora-grading#404
  • authoring: feat: lighter build by rewriting lodash imports frontend-app-authoring#1772
• Lighter production dependencies:
  • frontend-build (and fix yarn build): chore: lighter prod dependencies frontend-build#649
  • frontend-platform: chore: reorganize prod dependencies frontend-platform#787
  • profile: chore: lighter prod dependencies frontend-app-profile#1193
  • TODO: all other frontend apps
• Harmonize fortawesome dependencies:
  • header: chore: upgrade fortawesome dependencies frontend-component-header#584
  • TODO: upgrade from v5 in communications, gradebook, dashboard, ora-grading, learning
• Get rid of Husky:
  • chore: remove husky 🪓🐶 frontend-app-authoring#1775
  • chore: remove husky 🪓🐶 frontend-app-authn#1470
  • chore: remove husky 🪓🐶 frontend-app-communications#220
  • chore: remove husky 🪓🐶 frontend-app-discussions#761
  • chore: remove husky 🪓🐶 frontend-app-gradebook#442
  • chore: remove husky 🪓🐶 frontend-app-learner-dashboard#594
  • chore: remove husky 🪓🐶 frontend-app-ora-grading#405
  • chore: remove husky 🪓🐶 frontend-component-header#585
  • chore: remove husky 🪓🐶 frontend-component-footer#516
  • chore: remove husky 🪓🐶 frontend-platform#788

Notes

Semantic releases

It was decided in Paragon to use semantic releases: openedx/paragon#72

As a consequence, working on local versions of frontend-build, frontend-platform, etc. is extremely difficult. I was unable to test changes to devDependencies with npm link or even yalc.

Why did renovate[bot] stop upgrading fortawesome dependencies in frontend-component-header?

frontend-component-header depends on versions of @FortAwesome packages that are different from other MFEs. As a consequence, header has its own set of packages in node_modules/@openedx/frontend-component-header/node_modules/@FortAwesome. This adds extra data on disk, and probably in the compressed build as well.

Outdated 'file-loader'

frontend-build makes extensive use of file-loader, which was since replaced by assets-modules:

https://github.com/openedx/frontend-build/blob/master/config/webpack.prod.config.js
https://v4.webpack.js.org/loaders/file-loader/
https://webpack.js.org/guides/asset-modules/

Funky import resolution

Each and every frontend app is adding its own custom module resolution rules...

frontend-app-learning: @src imports

frontend-app-learning's webpack.prod.config.js has this weird section:

config.resolve.alias = {
  ...config.resolve.alias,
  '@src': path.resolve(__dirname, 'src'),
};

This change was introduced here: openedx/frontend-app-learning#1330

EDIT: Actually, I think this is the cleanest way to proceed, and we should implement the same resolve rule in all MFEs. Curious to learn what others think.

authoring: CourseAuthoring/* imports

const config = createConfig('webpack-prod', {
  resolve: {
    alias: {
      CourseAuthoring: path.resolve(__dirname, 'src/'),
    },
    fallback: {
      fs: false,
      constants: false,
    },
  },
});

This change was introduced here: https://github.com/openedx/frontend-app-authoring/pull/638/files#diff-a60dea17f5d642506d3c4eb9e4f3d0890f7b1601be813adf18583a248f4ce78a

ora-grading, learner-dashboard: import straight from the src directory

https://github.com/openedx/frontend-app-ora-grading/blob/master/webpack.prod.config.js
https://github.com/openedx/frontend-app-learner-dashboard/blob/master/webpack.prod.config.js

config.resolve.modules = [
  path.resolve(__dirname, './src'),
  'node_modules',
];

Changes were made in the first commit:
openedx/frontend-app-ora-grading#2
openedx/frontend-app-learner-dashboard@787e780

MFE re-installing npm???

"npm" is part of the authoring dependencies: https://github.com/openedx/frontend-app-authoring/blob/d1a6af51a4945360ce0cf3f7a5720f9f61ebef25/package.json#L81

openedx/frontend-app-authoring#1133

Outdated frontend-build patches

We should really really revert this PR: openedx/frontend-app-learning#1350

EDIT: Braden already noticed this before me and proposed a fix: openedx/frontend-app-learning#1621 🥇

Replacing webpack by rsbuild

I've managed to produce an rsbuild-based configuration file that works across all MFEs, with only minor changes to MFE apps. Details are here: overhangio/tutor-mfe#252

(tl;dr: results are very promising)

[bradenmacdonald]

It was decided in Paragon to use semantic releases: openedx/paragon#72 As a consequence, working on local versions of frontend-build, frontend-platform, etc. is extremely difficult. I was unable to test changes to devDependencies with npm link or even yalc.

Can you expand on this? I believe I've gotten it to work for local build purposes at least by linking it in with Local module configuration for Webpack, though this approach doesn't support other tooling like TypeScript checks.

[regisb]

So my goal was to test the changes of devDependencies in frontend-build. But whenever I installed my local version of frontend-build, I could not run npm install in frontend-app-profile anymore, because there was a clash in version: frontend-app-profile requires frontend-build >= 14.0.0 but I had frontend-build "1.0.0-semantically-released".

In the end I think I managed to get it wot work with lots of rm -rf, ln -s ... and explicit calls to npm install <all frontend-build dependencies>. I also modified the version in my local checkout of frontend-build. But I spent a few hours on this, and the result was not clean.

AFAIU local module configuration works for runtime loading, but not for installation of 3rd-party dependencies.

[bradenmacdonald]

@regisb If you see that, a workaround is to edit the package.json of your local copy of frontend-build and change the version number into the one that is expected. I personally think the semantically-released thing should be scrapped because of this annoyance.

[bradenmacdonald]

Outdated frontend-build patches

We should really really revert this PR: openedx/frontend-app-learning#1350

I have a PR here: openedx/frontend-app-learning#1621

[regisb]

On the topic of webpack vs rsbuild, I suggest we continue the conversation here @brian-smith-tcril @bradenmacdonald: overhangio/tutor-mfe#252 (spoiler: results are awesome)

[brian-smith-tcril]

MFE re-installing npm???

"npm" is part of the authoring dependencies: https://github.com/openedx/frontend-app-authoring/blob/d1a6af51a4945360ce0cf3f7a5720f9f61ebef25/package.json#L81

openedx/frontend-app-authoring#1133

Based on the changes in that PR my best guess is someone accidentally ran

npm install npm start

and ended up with npm and start added as dependencies.