feat: download busybox from fc-busybox release instead of embedding (#2281)

* feat: download busybox from fc-busybox release instead of embedding

Download busybox binary from e2b-dev/fc-busybox GitHub release at
Docker build time (via ADD) and at local build time (via curl in
Makefile). Removes ~3.2MB of committed binaries from git.

- Dockerfile: ADD from GitHub release, uses TARGETARCH
- Makefile: fetch-busybox target for local dev builds
- Both amd64 and arm64 binaries from same reproducible CI build

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: fetch busybox in CI before tests and lint

CI runs go test/lint directly (not via Docker), so the busybox
binary for go:embed must be downloaded first. Add fetch-busybox
to orchestrator test setup and lint workflow.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: verify busybox checksum, handle stale version and partial downloads

- Dockerfile: download with curl + sha256sum verification per arch
- Makefile: track version in .version file to detect stale binaries,
  write to .tmp first to avoid partial downloads

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: verify busybox against published SHA256SUMS from fc-busybox release

- Dockerfile: download SHA256SUMS from release, verify binary against it
  (no more hardcoded checksums that need manual updating)
- Makefile: track version+arch in .version file to detect stale binaries,
  write to .tmp first to avoid partial downloads

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move busybox download after COPY, fix sha256sum filename mismatch

- Move RUN after COPY ./orchestrator/pkg so the target directory exists
- Download as /tmp/{binary_name} (not /tmp/busybox) so sha256sum -c
  can find the file by the name in SHA256SUMS

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: don't re-download busybox in Docker, restore version tracking for local dev

- Dockerfile: build go directly instead of make build-local, so
  fetch-busybox doesn't overwrite the SHA256-verified binary
- Makefile: restore stamp file for version tracking in local dev,
  only used by build-local/build-debug (not Docker)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: subshell for cd /tmp in checksum verify, add fetch-busybox to test/lint

- Dockerfile: wrap cd /tmp in subshell so mv/chmod use correct WORKDIR,
  use absolute paths for destination
- Makefile: add fetch-busybox dependency to test and lint targets

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add SHA256 verification to fetch-busybox Makefile target

Download SHA256SUMS from the fc-busybox release and verify the binary
checksum, matching the Dockerfile verification pattern. All build paths
now verify integrity.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove fetch-busybox from test/lint targets

test and lint are called inside test.Dockerfile which doesn't have the
pkg/ directory. CI workflows have their own fetch-busybox setup steps.
Local dev gets busybox via build-local/build-debug before running
test/lint.

fetch-busybox remains on: build-local, build-debug (the entry points
for local builds that produce the full binary).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: single-source BUSYBOX_VERSION in Makefile, pass to Docker via build-arg

Makefile is the single source of truth for BUSYBOX_VERSION. Docker
build receives it via --build-arg. Removes the duplicate default
from the Dockerfile.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: extract busybox download+verify into shared script

Single fetch-busybox.sh script used by both Dockerfile and Makefile.
Eliminates duplicated download/SHA256 verification logic.
Dockerfile back to using make build-local (fetch-busybox skips since
the script already placed the binary).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use grep -wF for exact match in SHA256SUMS verification

Regex grep with dots in version string matches unintended lines.
Fixed-string whole-word match prevents false matches against .sig
or other suffixed entries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add fetch-busybox to test, lint, and build-template targets

All Makefile targets that compile Go code now depend on fetch-busybox
so a fresh checkout works without manual setup. The script skips
download if the binary already exists with matching version/arch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: tomassrnka

.github/workflows/lint.yml

@@ -35,6 +35,10 @@ jobs:
       - name: Parse .tool-versions
         uses: wistia/parse-tool-versions@v2.1.1
 
+      - name: Fetch busybox for orchestrator embed
+        if: contains(matrix.modules, 'orchestrator')
+        run: make -C packages/orchestrator fetch-busybox
+
       - name: golangci-lint ${{ matrix.modules }}
         uses: golangci/golangci-lint-action@v8
         with:

.github/workflows/pr-tests.yml

@@ -56,6 +56,9 @@ jobs:
 
       - name: Setup orchestrator tests
         run: |
+          # Download busybox for go:embed
+          make -C packages/orchestrator fetch-busybox
+
           # Enable unprivileged uffd mode
           echo 1 | sudo tee /proc/sys/vm/unprivileged_userfaultfd
 

packages/orchestrator/.gitignore

@@ -8,3 +8,7 @@ bin
 .shared
 /tmp
 .local-build
+
+# Downloaded at build time from fc-busybox release
+pkg/template/build/core/systeminit/busybox
+pkg/template/build/core/systeminit/busybox.stamp

packages/orchestrator/Dockerfile

@@ -3,7 +3,14 @@ ARG GOLANG_VERSION=1.25.4
 # It has to match with the host OS version (Ubuntu 22.04 = bookworm)
 ARG DEBIAN_VERSION=bookworm
 
+# Busybox version from fc-busybox GitHub release.
+# Passed via --build-arg from Makefile (single source of truth).
+# TARGETARCH is set automatically by Docker --platform.
+ARG BUSYBOX_VERSION
+
 FROM golang:${GOLANG_VERSION}-${DEBIAN_VERSION} AS builder
+ARG BUSYBOX_VERSION
+ARG TARGETARCH
 
 # Cached golang dependencies
 WORKDIR /build/shared
@@ -24,11 +31,16 @@ WORKDIR /build
 COPY ./shared/pkg ./shared/pkg
 COPY ./clickhouse/pkg ./clickhouse/pkg
 
+COPY ./orchestrator/scripts ./orchestrator/scripts
 COPY ./orchestrator/pkg ./orchestrator/pkg
 COPY ./orchestrator/cmd ./orchestrator/cmd
 COPY ./orchestrator/main.go ./orchestrator/main.go
 COPY ./orchestrator/Makefile ./orchestrator/Makefile
 
+# Download busybox and verify SHA256 against published checksums.
+RUN ./orchestrator/scripts/fetch-busybox.sh "${BUSYBOX_VERSION}" "${TARGETARCH}" \
+    ./orchestrator/pkg/template/build/core/systeminit/busybox
+
 WORKDIR /build/orchestrator
 
 ARG COMMIT_SHA

packages/orchestrator/Makefile

@@ -14,6 +14,10 @@ BUILD_ARCH ?= $(shell go env GOARCH)
 #   BUILD_PLATFORM=linux/arm64 make build
 BUILD_PLATFORM ?= linux/$(BUILD_ARCH)
 
+# Busybox version — single source of truth for both Docker and local builds.
+BUSYBOX_VERSION ?= 1.36.1
+BUSYBOX_EMBED := pkg/template/build/core/systeminit/busybox
+
 .PHONY: init
 init:
 	brew install protobuf
@@ -25,17 +29,23 @@ generate:
 .PHONY: build
 build:
 	$(eval COMMIT_SHA := $(shell git rev-parse --short HEAD))
-	@docker build --platform $(BUILD_PLATFORM) --output=bin --build-arg COMMIT_SHA="$(COMMIT_SHA)" -f ./Dockerfile ..
+	@docker build --platform $(BUILD_PLATFORM) --output=bin --build-arg COMMIT_SHA="$(COMMIT_SHA)" --build-arg BUSYBOX_VERSION="$(BUSYBOX_VERSION)" -f ./Dockerfile ..
+
+# Download busybox and verify SHA256 against published checksums.
+# Skips if binary exists and version/arch match the stamp file.
+.PHONY: fetch-busybox
+fetch-busybox:
+	@./scripts/fetch-busybox.sh "$(BUSYBOX_VERSION)" "$(BUILD_ARCH)" "$(BUSYBOX_EMBED)"
 
 .PHONY: build-local
-build-local:
+build-local: fetch-busybox
 	# Allow for passing commit sha directly for docker builds
 	$(eval COMMIT_SHA ?= $(shell git rev-parse --short HEAD))
 	CGO_ENABLED=1 GOOS=linux GOARCH=$(BUILD_ARCH) go build -o bin/orchestrator -ldflags "-X=main.commitSHA=$(COMMIT_SHA)" .
 	CGO_ENABLED=1 GOOS=linux GOARCH=$(BUILD_ARCH) go build -o bin/clean-nfs-cache -ldflags "-X=main.commitSHA=$(COMMIT_SHA)" ./cmd/clean-nfs-cache
 
 .PHONY: build-debug
-build-debug:
+build-debug: fetch-busybox
 	CGO_ENABLED=1 GOOS=linux GOARCH=$(BUILD_ARCH) go build -race -gcflags=all="-N -l" -o bin/orchestrator .
 
 .PHONY: run-debug
@@ -102,7 +112,7 @@ build-and-upload/template-manager: build upload/template-manager
 
 
 .PHONY: test
-test:
+test: fetch-busybox
 	go test -race -v ./...
 
 .PHONY: test-docker
@@ -118,7 +128,7 @@ test-docker:
 	@echo "Done"
 
 .PHONY: build-template
-build-template:
+build-template: fetch-busybox
 	TEMPLATE_BUCKET_NAME=$(TEMPLATE_BUCKET_NAME) \
 	GOOGLE_SERVICE_ACCOUNT_BASE64=$(GOOGLE_SERVICE_ACCOUNT_BASE64) \
 	DOCKER_AUTH_BASE64=$(DOCKER_AUTH_BASE64) \
@@ -137,5 +147,5 @@ migrate:
 	./scripts/upload-envs.sh /mnt/disks/fc-envs/v1 $(TEMPLATE_BUCKET_NAME)
 
 .PHONY: lint
-lint:
+lint: fetch-busybox
 	golangci-lint run --fix ./...

packages/orchestrator/pkg/template/build/core/systeminit/busybox_1.36.1-2_amd64

@@ -1,11 +1,11 @@
 //go:build amd64
 
-// Busybox v1.36.1 static binary for amd64 (musl, minimal ~16 applets).
-// Custom build added in #1002 — origin unknown, no distro tag in binary.
+// Busybox static binary for amd64.
+// Downloaded from https://github.com/e2b-dev/fc-busybox/releases at build time.
 
 package systeminit
 
 import _ "embed"
 
-//go:embed busybox_1.36.1-2_amd64
+//go:embed busybox
 var BusyboxBinary []byte

packages/orchestrator/pkg/template/build/core/systeminit/busybox_1.36.1-2_arm64

@@ -1,12 +1,11 @@
 //go:build arm64
 
-// Busybox v1.36.1 static binary for arm64 (glibc, full 271 applets).
-// Source: Debian busybox-static 1:1.36.1-9 (https://packages.debian.org/busybox-static)
-// TODO: rebuild both binaries from the same minimal config for consistency.
+// Busybox static binary for arm64.
+// Downloaded from https://github.com/e2b-dev/fc-busybox/releases at build time.
 
 package systeminit
 
 import _ "embed"
 
-//go:embed busybox_1.36.1-2_arm64
+//go:embed busybox
 var BusyboxBinary []byte

packages/orchestrator/pkg/template/build/core/systeminit/busybox_amd64.go

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+#
+# Download busybox from e2b-dev/fc-busybox GitHub release and verify SHA256.
+# Skips download if binary exists and version/arch match the stamp file.
+#
+# Usage:
+#   ./scripts/fetch-busybox.sh <version> <arch> <output_path>
+#
+# Example:
+#   ./scripts/fetch-busybox.sh 1.36.1 amd64 pkg/template/build/core/systeminit/busybox
+
+set -euo pipefail
+
+VERSION="${1:?Usage: fetch-busybox.sh <version> <arch> <output_path>}"
+ARCH="${2:?Usage: fetch-busybox.sh <version> <arch> <output_path>}"
+OUTPUT="${3:?Usage: fetch-busybox.sh <version> <arch> <output_path>}"
+STAMP="${OUTPUT}.stamp"
+
+# Skip if binary exists and version/arch match
+if [ -f "$OUTPUT" ] && [ "$(cat "$STAMP" 2>/dev/null)" = "${VERSION}-${ARCH}" ]; then
+  exit 0
+fi
+
+RELEASE_URL="https://github.com/e2b-dev/fc-busybox/releases/download/v${VERSION}"
+BINARY="busybox_v${VERSION}_${ARCH}"
+
+echo "Downloading busybox v${VERSION} (${ARCH})..."
+
+curl -sfL -o "/tmp/${BINARY}" "${RELEASE_URL}/${BINARY}"
+curl -sfL -o "/tmp/SHA256SUMS" "${RELEASE_URL}/SHA256SUMS"
+
+(cd /tmp && grep -wF "${BINARY}" SHA256SUMS | sha256sum -c -)
+
+mkdir -p "$(dirname "$OUTPUT")"
+mv "/tmp/${BINARY}" "$OUTPUT"
+chmod +x "$OUTPUT"
+echo "${VERSION}-${ARCH}" > "$STAMP"
+rm -f /tmp/SHA256SUMS