Build automatically using GitHub workflow

[Semphriss]

Yet another feature that made me think that if it's useful for me, there's a chance it might be useful for others :)

This uses GitHub Actions to automatically build w64devkit.

---

This is the way I made it for myself, I'm open to bring modifications to the PR if something in there would be interesting.

The builds are triggered on either of two events:

1. Each commit pushed to master, and each pull request open to master, which upload builds to the Action page (example):
   
   This offers a nice an easy way to download builds for any commit on the master branch. Unfortunately, those are only available for logged in users.

2. Each tag pushed on the repo, which automatically uploads the builds to a new release marked as draft (example - the only thing I did manually was to publish the release):
   
   This offers a ready-made template for releases based on git tags. I tried to strictly follow the format used by the latest releases, as to minimize manual intervention, but the release is not automatically published by GitHub Actions, so manual intervention is always possible before publishing a release.

---

The upsides I've noticed:

• The builds happen on GitHub rather than on my machine, so I don't have to wait 30 or so minutes per build before my machine becomes usable again (or before I can power it off and switch OS).
• The builds run in parallel on GitHub, because it uses one VM for each architecture-flavor pair. Locally, each build would have to be done one at a time, or the machine's cores would be divided between each concurrent build.
• It allows effortlessly seeing when a PR or a commit breaks the builds, and in the case of PRs, it allows the author of the PR to notice those errors by themselves without needing active feedback from others.
• It offers a reliable, somewhat reproducible (or at least well-documented) environment to build w64devkit in. Also, in situations where it might be an issue, there's no need to trust that anybody didn't tamper with the files.

The downsides:

• It takes 1h30 to finish all builds, and no download is available before all builds finished. For single configurations, building locally may be faster. However, for an all-configs build (6 configs * 30 minutes per config on average = 3 hours total), GitHub Actions remains faster.

---

If there's interest in this PR, I can explain in further detail any decision I took while creating the workflow file. I'm also willing to adapt this PR as necessary.

[Peter0x44]

I'm using this, it's useful for me. It needs minor adjustments now though, there isn't a -mini version any longer.

[Semphriss]

@Peter0x44 I've updated the script and rebased on master.

The workflow still doesn't use the multibuild.sh script, since steps would need to be extracted to make them fit in GitHub's file format. There are still a few minor differences between the script and the workflow file, notably about the version suffix. I preferred to leave it like that since it makes the maintenance of the workflow easier, but I'm open to update it as necessary.

[Peter0x44]

The fortran variant is now gone, so I think this will need a few adjustments.

[skeeto]

Yup, I'm about to make a new 2.0.0 release with significant distribution changes. It's now a self-extracting 7-zip archive, and gfortran is always included. The two release artifacts will be named:

• w64devkit-x64-2.0.0.exe
• w64devkit-x86-2.0.0.exe

Like when I got rid of "mini", I want to simplify it for newcomers, so they don't have to make a decision about which to download. It's just 64-bit and 32-bit, with 64-bit (deliberately) sorting first. These releases are also a third of the size of the originals and extract ~100x faster (literally!) than Windows built-in zip support.

I hope this will also help with virus scanner nonsense, because extracted files won't have Mark of the Web (i.e. "forbid this file from any use"), which Windows built-in zip support does, but 7-zip, especially the self extract, does not. If someday I figure out how to navigate Windows code signing, signing just the SFX will be enough.

[Peter0x44]

On the subject of signing, there is some discussion here in msys2 for doing it to the installer:
msys2/msys2-installer#73 (comment)
Though, there are no satisfactory options available yet.

[skeeto]

Thanks, Peter0x44, that's useful information!

[Semphriss]

I've updated the workflow file to w64devkit 2.0.0.

[Peter0x44]

Error: This request has been automatically failed because it uses a deprecated version of actions/upload-artifact: v2. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/

This needs some adjustments to stop using this deprecated action version.

[Semphriss]

Error: This request has been automatically failed because it uses a deprecated version of actions/upload-artifact: v2. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/

This needs some adjustments to stop using this deprecated action version.

Thanks, it should be fixed now.

[Peter0x44]

IMO, we should really look seriously at merging this @skeeto . Github CI minutes are unlimited and free for all opensource rpos, and it would really help people make their own custom w64devkit builds and test them without having to install Docker.

We could also do additional QA or testing, like running the gcc testsuite, which currently almost no one is doing for windows.
This second part is optional, but I think even the first alone is good enough reason. If you fix something in a branch, you can immediately point someone to a build, without expecting them to do it.

[Peter0x44]

The PR as is doesn't work, if you look at the logs you will see the w64devkit build actually completes, but it seems to hang afterwards
Maybe it has something to do with "exporting layers" into an image?
Either way, next commit has an LLM authored fix that seems to work. But I don't know why/how, I'm not familiar with docker to know if it's a good idea or not.

https://github.com/Peter0x44/w64devkit/actions/runs/18144719156/job/51643843066

Peter0x44@cd7cdd2

If @skeeto could enable workflows in the repo on the actions tab and approve this run, we could have the info to know if it actually works here.

[skeeto]

could enable workflows in the repo on the actions tab and approve this run

When I look I see it's already set to "Allow all actions and reusable workflows (Any action or reusable workflow can be used, regardless of who authored it or where it is defined.)". I cannot find a way to approve a particular run.

Maybe it has something to do with "exporting layers" into an image?

At the end of the log I see:

2025-09-30T23:01:05.0180486Z #83 exporting to image
2025-09-30T23:01:05.0180929Z #83 exporting layers
2025-09-30T23:03:57.8001278Z ##[error]Process completed with exit code 143.

A search on that error suggests it's hitting some resource usage threshold that kills the build. That's after all the building and compressing, which I would expect are the most intensive parts. It's going over the limit on writing out the final image, perhaps because it's larger (at ~6G) than GitHub allows.

The intermediate build artifacts (object files, etc.) are just that large, and there's no way to reduce this while retaining a cache (see below).

like running the gcc testsuite, which currently almost no one is doing for windows.

That's a good idea, and for me the most convincing argument for having a pipeline. (And of course only needs to run on GCC and Binutils updates. Or GCC dependencies like GMP.)

I'm not familiar with docker to know if it's a good idea or not.

The "builder" stage indicates the LLM is confused about how Docker is used in this project — it thinks the resulting image is going to be distributed, rather than run just once to extract the real build artifact — which is fair because w64dk's usage is unusual, if not unique. That probably needs to be clarified in the prompt.

Tangent: I've long considered adding some stages to break up the build into more of a dependency tree, like a Makefile. Individual downloads could be distinct stages, and so bumping one dependency version won't require re-downloading everything. After bootstrapping, building each individual tool (native gcc, vim, gdb, busybox, etc.) could be done in their own stages, so that patching/changing one tool doesn't require rebuilding all subsequent tools.

it would really help people make their own custom w64devkit builds and test them without having to install Docker.

Because the build pulls on free, external resources it relies on the generosity of hosts that have no association with w64dk. Docker in general does a good job caching intermediate layers, and so someone like me doing repeated builds on their own machine will only download these resources on the occasional version bump. I've arranged for all the downloading to happen early, minimizing cache disruption. Hardly any of my own testing/development builds reach out to the internet.

I'm less confident about the download caching in a CI pipeline. Typical pipelines are wasteful and reckless, needlessly hammering external resources, because that's what's easy. They're locusts that feed on community slack. Language package managers report ludicrous download numbers because it's been normalized for pipelines to re-download dependencies on every run. Apparently it's difficult to create an efficient pipeline, because hardly anyone does it.

I don't want to run a pipeline here that's hitting external hosts any more than necessary, e.g. re-downloading every run. So I first need to be convinced that builds are appropriately cached. For example, changing a word in README.md should only take a couple of minutes to rebuild. Just the time to recompress the 7z archive with the new README.md, because that file is copied in at the last step. Requiring a full rebuild is locust behavior.

The corollary is that a good, cached build in a pipeline here might on net reduce the load on generous, external hosts. People using the pipeline to do their own build will share a cache, rather than each caching on their own machines.

[Peter0x44]

If you're concerned about hitting external hosts too much, I would just make another repo where the tarballs are committed, and have the CI clone that repo, or download "raw" tarballs.
Then it's only GitHub's bandwidth being used, which is surely okay, considering it's their compute in use also.

I consider downloading maybe 150MiB worth of tarballs is a drop in the ocean. The convenience is well worth it.
Being unable to implement this caching is really not important, IMO.

[Peter0x44]

When I look I see it's already set to "Allow all actions and reusable workflows (Any action or reusable workflow can be used, regardless of who authored it or where it is defined.)". I cannot find a way to approve a particular run.

I think github just blocks all workflows that don't exist in the base branch from running. There's no way to approve or make it run before it's committed, apart from on the fork.

[Peter0x44]

perhaps not as expensive to build as w64devkit, but the upstream mingw-w64 has a workflow with precisely the "inefficiencies" or "slack" you're complaining about. It runs on every commit.

https://github.com/mingw-w64/mingw-w64/blob/master/.github/workflows/build.yml

As a side note, lets not aim any insults like "locust" at other developers, even in passing or indirectly. I don't think that's appropriate. Allegedly, in some parts of the world, it is even a slur.

[Peter0x44]

In any case, I managed to make the caching work in this attempt:
Peter0x44@9fa342e
You can see in the workflow run of the next commit:
https://github.com/Peter0x44/w64devkit/actions/runs/18277592358/job/52033185158
That most things got cached and it worked correctly.

Since @Semphriss seems to be missing for now, I will just submit it as a separate PR from this one.

[Semphriss]

Apologies for the lack of reply, I just came back from vacation.

I have had the same problem with my personal fork. The only way I had found to resolve it was to combine everything in one single step and delete everything apart from the resulting executable at the end of that step, but I found that to be quite impractical.

Depending on how good it sounds, it may be possible to download the tarballs outside of the Dockerfile, cache them, and copy them into the build process. They can easily be cached individually, so no need to re-download all tarballs when only one that happens to be near the top of the list happens to be updated, but it does have the downside of complicating the build process.

As for the pipeline heaviness, in my experience it isn't anything too extreme. Assuming it runs once per commit, if it was implemented at the creation of this repository, it would've run 373 times over the course of 5 and a half years, or about once every 5 days on average, which doesn't sound excessive to me.

I can try to update this PR as necessary, but it seems that @Peter0x44 already got that covered :)