[CI] Improve release build process #6601
Description
Current process relies on docker and cross-compilation (including cross-compiling for different os'es).
Under the best circumstances, it takes roughly 30+ mins to complete all binary builds for all platforms. under less than ideal circumstances, a single failure of any of the binary builds requires a re-run, which delays a release until all binary builds have completed.
The first improvement i think that could be made here is to remove docker from the process - we can build the binaries on a runner (and in the case of macos, use a macos runner for that task). the second option (and this is relevant for retries) is that i think we should also be using workflow artifacts vs the dockerfile downloding the archive from teh github release.
if we keep using the archive from the github release - what can happen is a release looks like it has been published with the binaries, but there may be a problem with the release Dockerfile so the images are not built and the entire workflow needs to be re-run.
propose:
- matrix to run the binary builds on the runner natively
- save the binaries as a workflow artifact
- update release dockerfile to use the workflow artifact
- publish a github release using the workflow artifact in draft mode.
currently, the binary builds all happen in a composite. part of this work may involve moving that workflow back into the stacks-core repo (at a minimum, the matrix needs to be triggered from stacks-core, so it makes sense to move all jobs unless used elsewhere).
side note: this process update may also benefit #6556 , since the same process could be used to create an ad-hoc mutli-platform image for testing. building the multi-platform image in docker on a default runner takes ~2 hours, whereas building teh binaries takes ~15 minutes or so.