Rarely used build_runner commands

[davidmorgan]

I am wondering whether any of these little-documented, rarely-mentioned features can be removed

dart run build_runner serve
dart run build_runner test
dart run build_runner run
dart run build_runner daemon

Does anyone use them?

[sunderee]

Myself not. But I'd love to see what others have to say. IMHO the only two that should remain are build and perhaps watch, just in case someone wants to be able to keep rerunning code generation as they work.

I am concerned about the potentially small sample space you'll get with this issue, because while most people will see your post on Twitter/X, perhaps some popular package maintainers will not, and it might negatively influence their work.

Just because I've personally never seen anyone use any of the four mentioned doesn't mean people don't use them.

On a different note: why would you like to remove them, do they represent technical debt?

Edit: it's "watch" not "serve".

[davidmorgan]

In the worst case if I remove something that someone was using I can put it back :) but I'll try hard not to do that.

Yes, it's technical debt: as I refactor to make "build" and "watch" do what I want them to do, I sometimes need to update the others as well. Since I don't have a good idea how they're used, I'm not sure whether I'm successfully keeping them working ... or gradually breaking them more and it's going unnoticed.

Also, the more commands there are, the longer the e2e tests take to run, which slows down sending/preparing PRs.

[sunderee]

Understood. I mean, if nobody complains in the reasonable time you wish to have this issue open for discussion, then going ahead makes sense. Again, it'd be great to hear from maintainers of some popular packages that rely on build_runner.

[grahamsmith]

Might be nonsense but could be lack of awareness of what these lesser known friends are for and can do?

Granted watch and run are the main functions I use.

[schultek]

webdev uses the build_runner daemon, which I think internally does the same as serve (spinning up a webserver and serving files from a specified directory, in webdev's case 'web'). Jaspr (specifically its CLI) in extension also uses the daemon and serve, since it uses parts of webdev. I've only ever directly used serve when testing or debugging the cli implementation, never as an end user.

I think the daemon is pretty vital in order to interface with other programs and clis, and shouldn't be removed.

Regarding serve and its functionality, I never understood why this is even part of build_runner. There's a weird coupling between build_runner serve, webdev and build_web_compilers, all kinda doing the same thing / having the same concern. I'm not very fond of the whole system, as I think the whole dart build system doesn't lend itself well for having a web tooling pipeline build on top of it. It feels more like a hacky mushed together set of tools and not like a robust solution for dart web, and it suffers from it: It's overly complex (sooo complex omg), slow and not very extensible. I frequently run into limitations trying to build cool Jaspr features. I had to completely fork build_web_compilers and republish as jaspr_web_compilers to do the things I wanted, but it's a pain to maintain because of the complexity.
It's all stemming from the fact that the builder paradigm is not at all suited for web tooling. Because builders only allow to create new files. But web tooling has to do things like js compilation, css pre-processing and minification, maybe some pre-processing or post-processing of other files like images, etc. - so always some transforming of existing files, which is not allowed for builders and requires tricky workarounds (thats why basically the whole build_web_compilers is a giant workaround). And pluggability is basically zero. So IMO the whole dart web pipeline should actually be ripped out of build_runner and allowed to be it's own thing.

I know its a massive task and I'm just dreaming here, but maybe removing serve is a good first motivation to look at this more closely.

[davidmorgan]

It's used here as part of webdev serve: https://github.com/dart-lang/webdev/blob/main/webdev/lib/src/daemon_client.dart

Oh, so it is. That answers that question! Thanks.

To expand on what I meant, the goal is not to actually override the source file, but to output a file with the same name either as an intermediate or final build result in some other directory. For example for css minification, I don't want to override the source, but output a file with the same name in some build/ output directory. But with build_runner there is no concept of "output directory" at least in the builders. Webdev solves this with a scratch space and then copying over all resulting files into the build directory, but from a build_runner perspective this would still be treated conceptually as "overridding" the original files and therefore prevented.

I see, interesting. Do you have an idea for a good way for this to work ... e.g. if a builder could declare one output directory per package and can write to that? (Maybe: write one file to that for each input file / with the same name?)

The second important point is pluggability. When I have a builder from some package that reads file a and outputs file b, there is no way for me to preprocess file a before letting that builder continue - again because I can't output files with the same path. There are weird workarounds with post-processing builders, which is what build_web_compilers does to avoid this.

Hmmmm if it's about processing files to create a same-structured output folder ... what if we allow any builder to "replace" a file, handle the intermediates internally to build_runner, then have one "output" folder that is the final state of all the files that got touched? (or maybe just ... all the files?)

Thats fair and understandable. I just wanted to vent some feedback :)

Definitely interested in discussing possibilities here--anything that is a good fit for web might be useful for other things too.

[simolus3]

I also find build_runner serve very useful, I use it for quick tests to debug my Dart libraries on the web. Without build_web_compilers, ddc is almost impossible to use.

I also use dart run build_runner test to test a large package with many test files on the web. A modular compilation with build_web_compilers and ddc is much faster than running dart test which invokes dart2js on every test entrypoint.

[davidmorgan]

I also find build_runner serve very useful, I use it for quick tests to debug my Dart libraries on the web. Without build_web_compilers, ddc is almost impossible to use.

Would webdev serve do the same thing? (I haven't really used either).

I also use dart run build_runner test to test a large package with many test files on the web. A modular compilation with build_web_compilers and ddc is much faster than running dart test which invokes dart2js on every test entrypoint.

Oh, that's very interesting. Thanks!

[simolus3]

Would webdev serve do the same thing? (I haven't really used either).

It probably does. I was trying to avoid the webdev package where I don't need it because build_runner got the job done, but I tend to agree that separating those concerns and moving some of the serve logic into webdev or an intermediate package wrapping build_daemon could be reasonable.

[davidmorgan]

It probably does. I was trying to avoid the webdev package where I don't need it because build_runner got the job done, but I tend to agree that separating those concerns and moving some of the serve logic into webdev or an intermediate package wrapping build_daemon could be reasonable.

It's probably time I learned what webdev does :) ... thanks.