Embroider v4 compatBuild no longer supports Webpack; vite's buildOnce is a hardcoded parameter

[ChrisN-transnexus]

I'm currently upgrading an Ember 5.12 / Embroider 3 / Webpack 5 v2 addon to Ember 6.8. I updated all the dependencies I could, including moving from Embroider 3 to 4. I started getting the error:

Build Error (Builder) buildOnce is not a function

I tracked this down to the parameter signature for compatBuild in packages/compat/src/default-pipeline.ts:

https://github.com/embroider-build/embroider/blob/main/packages/compat/src/default-pipeline.ts#L42

Based on this new parameter signature, which has a hard-coded vite buildOnce function, @embroider/compat's compatBuild() no longer supports Webpack in v4. The previous parameter signature was compatBuild(app, Webpack, options). Now it is compatBuild(emberApp, buildOnce, options).

Am I reading this correctly?

I use Webpack plugins via packagerOptions.webpackConfig in ember-cli-build. I don't believe some of the things I need to do via Webpack are supported in vite. Does that mean this addon will never upgrade beyond Embroider 3? What happens to my v2 addon when Embroider 4 becomes a hard requirement for Ember 7 or 8, etc?

Also, why isn't this breaking change mentioned in the upgrade guides or deprecations docs? Upgrading Embroider 2 to 3 was previously covered, explicitly showing that compatBuild works with Webpack:

https://github.com/embroider-build/embroider/blob/main/docs/upgrade-guides.md

Where is the upgrading from v3 to v4 doc?

Furthermore, why did the Ember team promote Webpack as the future bundler for Ember applications in 2020/2021 if just four or five years later, the Ember team is going to start abandoning that commitment in favor of vite?

[kategengler]

I'm going to break out your comment by question and answer:

Does Embroider v4 only support Vite and not support Webpack?

Yes. As is covered in the README

Is moving from Embroider v3 to v4 a breaking change?

Yes. Ember projects, including Embroider, follow semver. Any major version bump denotes a breaking change, so as is proper, the version incremented from 3 to 4. Version 3 still works and is supported in Ember 6.8. You'll want to upgrade following directions in the Ember 6.8 blog post for the classic blueprint with the --embroider option.

I don't believe some of the things I need to do via Webpack are supported in vite. Does that mean this addon will never upgrade beyond Embroider 3?

This will depend on what you are doing. I suspect it is all supported via vite or plugins. The v2 addon blueprint is not yet the default for addons because there were things to figure out and is in-progress RFC. You were an early adopter of v2 addons.

Vite only became the default applications as of 6.8.

Also, why isn't this breaking change mentioned in the upgrade guides or deprecations docs?

This is fair. For apps we have it well covered between the ember-vite-codemod and the documentation in Ember 6.8. For addons, this is exactly what I feared for years with many experimental v2 addon blueprints floating around: We have a large variety of blueprints in the wild and no good way to get them to the official blueprint (once we have it) other than manually. Fortunately, v2 addons continue to work in projects so long as they adhere to the publish format, so many build tools and setups are valid.

Furthermore, why did the Ember team promote Webpack as the future bundler for Ember applications in 2020/2021 if just four or five years later, the Ember team is going to start abandoning that commitment in favor of vite?

Ignoring that embroider with webpack was never the default for an app or addon blueprint provided by Ember, four or five years is about an eternity in JavaScript tooling-time. 2020 was only about 5 years after we developed ember-cli with broccoli.

The Ember project is stability without stagnation which means we move to new tools and try new things but we also support upgrade paths. We have discussed deprecation warnings for Embroider with webpack but as of right now Embroider v3 is still supported and what you should use if you need or want to continue using webpack.

[ChrisN-transnexus]

Thank you for your reply. I appreciate you took the time to reply in-depth on my questions.

Embroider 3 has several security vulnerabilities which are patched in Embroider 4. My projects cannot apply those security patches if they cannot be updated past Embroider 3. Also, when Embroider 4 becomes a hard requirement for Ember 7 or 8, etc, my projects cannot be upgraded to those versions of Ember without implementing Vite.

I have researched Vite, which was never intended to be a direct replacement for Webpack, and I can see that there are some plugins that may meet my needs. However, due to security concerns, I have to be very selective about which dependencies are imported into the codebase. Many of the Vite plugins available now won't meet our OpenSSF / in-house security requirements.

As far as v2 addons, yes, I agree the blueprints are all over the place, and it would be great to have some finality to them. However, as I recall, I was forced to update my addons to the v2 format early-on in order to get Embroider 3 and ember-auto-import 2 to work, and the reason I moved to Embroider 3 was to get Webpack 5 support. Now Webpack support is being removed.

I understand how semver works and that moving from v3 to v4 will break some things. What I was frustrated with is how, as you acknowledged, there is no upgrade doc from Embroider 3 to 4 explaining that Webpack is no longer supported. I was forced to figure this out on my own by digging through the Embroider source code after getting a nebulous build error. If the Ember team is going to support Webpack for many years (an "eternity," to use your words) then suddenly stop supporting Webpack, there should be some documentation online or in the repo explaining this.

I never said Webpack was the default bundler for Ember / Embroider. You said that, just now. What I said was Webpack was promoted as the future bundler, as in these Ember blog entries:

https://blog.emberjs.com/the-ember-times-issue-193/
https://blog.emberjs.com/2020-ember-roadmap/#toc_modernize-our-build-system

You could obviously point out that those official Ember blog entries are five years old, but what about this?:

https://emberjs.com/editions/polaris/

That official Ember editions page states that Polaris (meaning Ember 5 / 6) will have a "new build pipeline (Embroider) that hooks into the tool of your choice (webpack, rollup, etc)," Ember 6.8 is the current LTS, supported through the end of this year, so that commitment to support Webpack is not five years old since Polaris is the current edition of Ember. To me, this means there is an explicit current commitment on the official Ember website to support Webpack, but the latest Ember / Embroider stack does not. Since Embroider 4 does not allow me to use the bundler of my choice, I feel that is a bait-and-switch on the developers who use Ember for enterprise products.

As a developer working on enterprise applications who can't easily switch build tools just 2 years after I committed to Webpack, I feel a little betrayed by the Ember team when I read on the Ember website that Polaris will support Webpack, when it actually does not, at least not past Embroider 3.

[kategengler]

I will come back for the rest but I want to clarify that Polaris does not mean Ember 5/6. The current edition is Octane. The next edition is Polaris. We are in an in-between state.

[ef4]

I want to draw a distinction here between what Embroider supports and what Ember chooses as defaults.

What Embroider Supports

Embroider has always been about having layers that are build-tool agnostic so that people who care about supporting any particular build-tool on Ember can do it. It's not about "promoting webpack" or "promoting vite". That's why @embroider/core is distinct from @embroider/webpack or @embroider/vite. We are open to all build tools that people are willing to pour energy into integrating, and that hasn't changed. This change:

-compatBuild(app, Webpack, options)
+compatBuild(app, buildOnce, options)

is not hard coding one build tool or another. It's about no longer supporting the idea of wrapping the build tool (whether Webpack or Vite or Esbuild, etc) inside Broccoli.

Before, the second argument was an entire Broccoli plugin that wrapped your build tool. Now you cannot do that, but you can pass a hook that covers what to do when somebody says ember build. That's what the buildOnce from @embroider/vite is doing.

Webpack could absolutely still work under this API too. Vite is not "hard-coded", if it was there would be no need for a parameter here at all. We even still have most of the infrastructure to make webpack work. We didn't delete @embroider/webpack, it's still in the repo with failing tests waiting for somebody who cares about it to support the work.

I use Webpack plugins via packagerOptions.webpackConfig in ember-cli-build.

Since we no longer support wrapping the build tool inside broccoli (and ember-cli-build.js is a broccoli file), the way you would use webpack plugins in Embroider 4 is to put them directly into your own webpack.config.js. @embroider/webpack itself would also provide some plugins that go into the webpack config, and that would be how it all integrates.

What happens to my v2 addon when Embroider 4 becomes a hard requirement for Ember 7 or 8, etc?

We designed v2 addons to be build-tool agnostic. They deliberately do not have the ability to manipulate their build environment, so that no addon can accidentally lock an app into one build tool or another. We actively test v2 addon support all the way from Ember 3.4 up to canary, under broccoli, webpack, vite, rollup, esbuild, and rolldown.

Ember Defaults

Now, even though Embroider is deliberately designed to support many different build tools (especially including ones that don't even exist yet), Ember as a community values having one default way to do things. We did RFC 977 and decided on Vite.

Embroider 3 with Webpack is still supported by the upcoming Ember 7.0, which is just starting its full year of minor releases. Then the final 7.x LTS will get another 54 weeks of security support.

That means people get at least two years from now to absorb a breaking change. And really that's three years from Embroider 4.0 releasing, because we ensured it was truly solid and already adopted by large, old codebases before we even proposed it as the default.

I'm not going to apologize for this being somehow not good enough for enterprise development.

Embroider 3 has several security vulnerabilities which are patched in Embroider 4.

Embroider 3 is still fully supported, please report those vulnerabilities, they should be addressed.

[ChrisN-transnexus]

I will come back for the rest but I want to clarify that Polaris does not mean Ember 5/6. The current edition is Octane. The next edition is Polaris. We are in an in-between state.

OK, you're helping my position by saying that Polaris is the next edition of Ember, because the Ember blogs I posted here explicitly committed to supporting Webpack in Polaris.

[kategengler]

Whatever a blog says or the website says doesn't mean a commitment of support in perpetuity. Plans change.

For a summary of the current state here:

• You opted into an "cutting-edge" tool (Embroider with webpack) that the Ember team did not deem ready for the wider community (or it would have been the default).
• You can still use that tool with Embroider v3 which still supports the current version of Ember.
• You believe you cannot move to the default build tool in latest Ember (Embroider with vite) but have not said what tools or use cases are not covered.

A summary of the Ember Projects general commitments:

• Changes to Ember's shipped & documented experience require RFCs that allow the community many chances to comment and influence the features going into or being removed from Ember.
• Projects as generated with ember new blueprint will adhere to semver. Removed or changing features, as directed by an RFC, are deprecated in advance of a major version before being removed in that version.
• Experiments will happen outside of that blueprint that have their own semver commitments.

[ChrisN-transnexus]

Before, the second argument was an entire Broccoli plugin that wrapped your build tool. Now you cannot do that, but you can pass a hook that covers what to do when somebody says ember build. That's what the buildOnce from @embroider/vite is doing.

How do I do this? Where is the documentation on this resolving this breaking change?

Webpack could absolutely still work under this API too. Vite is not "hard-coded", if it was there would be no need for a parameter here at all. We even still have most of the infrastructure to make webpack work. We didn't delete @embroider/webpack, it's still in the repo with failing tests waiting for somebody who cares about it to support the work.

I apologize for incorrectly using the word "hard-coded." That was a mistake. I guess a better term would have been "tightly-coupled," since buildOnce is in @embroider/vite but not in @embroider/webpack. What does it mean that "most of the infrastructure to make webpack work" is still available? "Most" is a very broad word. And the end developer would need to build the missing parts in their local repo? How? Why have the Embroider developers who have this knowledge not shared it in a doc?

In the time it took two Embroider contributors to argue with me here, you could have written a doc describing how to make Webpack work with Embroider 4.

Since we no longer support wrapping the build tool inside broccoli (and ember-cli-build.js is a broccoli file), the way you would use webpack plugins in Embroider 4 is to put them directly into your own webpack.config.js. @embroider/webpack itself would also provide some plugins that go into the webpack config, and that would be how it all integrates.

Where is the documentation on any of this? It sounds like you're telling me I should be able to do all this on my own, to reverse engineer buildOnce in @embroider/vite and wrap @embroider/webpack in that function and integrate it into embroider 4. All without any knowledge of how buildOnce works.

All I see is a README file which shows one build tool implementation, which is vite: https://github.com/embroider-build/embroider/blob/main/README.md#options There is no mention of being able to use Webpack.

What happens to my v2 addon when Embroider 4 becomes a hard requirement for Ember 7 or 8, etc?

We designed v2 addons to be build-tool agnostic. They deliberately do not have the ability to manipulate their build environment, so that no addon can accidentally lock an app into one build tool or another. We actively test v2 addon support all the way from Ember 3.4 up to canary, under broccoli, webpack, vite, rollup, esbuild, and rolldown.

I understand Ember v2 addons are agnostic. However, every Ember addon comes with a test-app which runs automated tests. That test-app needs a build-tool, and that build tool should match the build-tool used in apps which consume the addon, if the same team is developing both.

Embroider 3 has several security vulnerabilities which are patched in Embroider 4.

Embroider 3 is still fully supported, please report those vulnerabilities, they should be addressed.

#2722 This is a breaking change, however, according to Broccoli.

[ChrisN-transnexus]

I'm not going to apologize for this being somehow not good enough for enterprise development.

Embroider 3 has several security vulnerabilities which are patched in Embroider 4.

Embroider 3 is still fully supported, please report those vulnerabilities, they should be addressed.

Yep, the vulnerability issue I opened was immediately addressed in Embroider 3, thanks!

(HINT: it wasn't addressed, my issue was closed with the suggestion I move to VITE!)

[NullVoxPopuli]

I suggested you use your package manager's overrides/resolution feature for the time being as well <3

edit: full response offers help + exploring webpack options as its faster to prototype than to talk #2722 (comment)

edit2: https://github.com/NullVoxPopuli/embroider/pull/10/changes#diff-18a23055e6c99f2145a95ecd30dc737fd82f7eb9e5f91e2c0c2e37ef42de9b57R21

edit3: #2729

[NullVoxPopuli]

@ChrisN-transnexus all the tests are passing on #2729 -- idk if you want to try it locally. there are likely details of how v2 apps work that we'll need to go over (my best guess is that we do vite-codemod, remove vite, custom build the tarball on that branch, copy the tarball over to your project, you install that, and we see what errors we run in to)

cc @mansona for codemod viz
cc @ef4 for quality/direction checking