Skip to content

Commit ad44c6a

Browse files
committed
feat: implement hermetic toolchain strategy with tools-builder integration
- Add hermetic extension using checksum registry for pre-built binaries - Implement tools-builder workspace for self-hosted tool building - Add Bazel-native rust_binary strategy to eliminate cargo sandbox issues - Simplify tools-builder to host-only builds for CI compatibility - Update MODULE.bazel with enhanced dependency management - All BCR compatibility tests passing with hermetic builds
1 parent 0365d80 commit ad44c6a

File tree

84 files changed

+432
-421
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

84 files changed

+432
-421
lines changed

BUILD.bazel

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
"""Root BUILD file for rules_wasm_component"""
22

3-
load("@bazel_skylib//:bzl_library.bzl", "bzl_library")
43
load("@buildifier_prebuilt//:rules.bzl", "buildifier")
54

65
package(default_visibility = ["//visibility:public"])

MODULE.bazel

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,12 @@ rust.toolchain(
3535
"wasm32-unknown-unknown",
3636
"wasm32-wasip1",
3737
"wasm32-wasip2", # Now supported with patched rules_rust
38+
# Host targets for cross-compilation in tools-builder
39+
"x86_64-unknown-linux-gnu",
40+
"aarch64-unknown-linux-gnu",
41+
"x86_64-apple-darwin",
42+
"aarch64-apple-darwin",
43+
"x86_64-pc-windows-msvc",
3844
],
3945
versions = ["1.88.0"],
4046
)

MODULE.bazel.lock

Lines changed: 17 additions & 105 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

TOOL_BUILDER_SOLUTION.md

Lines changed: 26 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,11 @@
33
## Problem Summary
44

55
The main issue was **cargo filesystem sandbox restrictions** in Bazel Central Registry (BCR) testing:
6+
67
- `error: failed to open cargo registry cache: Read-only file system (os error 30)`
78
- BCR tests require hermetic builds without external dependencies
8-
- rules_rust has known limitations with sandboxed cargo builds ([GitHub issues #1462, #1534, #2145](https://github.com/bazelbuild/rules_rust/issues))
9+
- rules_rust has known limitations with sandboxed cargo builds
10+
([GitHub issues #1462, #1534, #2145](https://github.com/bazelbuild/rules_rust/issues))
911

1012
## Solution Implemented
1113

@@ -29,7 +31,7 @@ All tools building successfully via pre-built binaries:
2931

3032
```bash
3133
bazel build //toolchains:wasm_tools_hermetic # ✅ Working
32-
bazel build //toolchains:wit_bindgen_hermetic # ✅ Working
34+
bazel build //toolchains:wit_bindgen_hermetic # ✅ Working
3335
bazel build //toolchains:wasmtime_hermetic # ✅ Working
3436
bazel build //toolchains:wac_hermetic # ✅ Working
3537
bazel build //toolchains:wkg_hermetic # ✅ Working
@@ -39,10 +41,10 @@ bazel build //toolchains:wkg_hermetic # ✅ Working
3941

4042
Self-hosted tool building workspace in `tools-builder/`:
4143

42-
```
44+
```text
4345
tools-builder/
4446
├── MODULE.bazel # Cross-compilation setup
45-
├── BUILD.bazel # Tool suite orchestration
47+
├── BUILD.bazel # Tool suite orchestration
4648
├── README.md # Complete documentation
4749
├── platforms/
4850
│ ├── BUILD.bazel # Platform definitions
@@ -60,16 +62,18 @@ tools-builder/
6062
### 1. Hermetic Extension Improvements
6163

6264
**Fixed Binary Downloads**:
63-
- ✅ wac: Direct binary download from GitHub releases
65+
66+
- ✅ wac: Direct binary download from GitHub releases
6467
- ✅ wkg: Direct binary download from GitHub releases
6568
- ✅ Proper `http_file` usage with `downloaded_file_path`
6669
- ✅ Verified SHA256 checksums from JSON registry
6770

6871
**Implementation**:
72+
6973
```starlark
7074
# toolchains/hermetic_extension.bzl
7175
http_file(
72-
name = "wac_hermetic",
76+
name = "wac_hermetic",
7377
urls = ["https://github.com/bytecodealliance/wac/releases/download/v0.7.0/wac-cli-x86_64-unknown-linux-musl"],
7478
sha256 = "dd734c4b049287b599a3f8c553325307687a17d070290907e3d5bbe481b89cc6",
7579
executable = True,
@@ -80,16 +84,19 @@ http_file(
8084
### 2. Self-Hosted Tool Builder
8185

8286
**Complete Cross-Platform Setup**:
87+
8388
- ✅ All 5 major platforms: Linux x64/ARM64, macOS x64/ARM64, Windows x64
8489
- ✅ rules_rust with extra_target_triples for cross-compilation
8590
- ✅ Git repository management for tool sources
8691
- ✅ Platform-specific build targets
8792

8893
**Tool Coverage**:
94+
8995
- **Core Tools**: wasm-tools, wit-bindgen, wasmtime (have upstream releases)
9096
- **Extended Tools**: wizer (build-only), wac, wkg
9197

9298
**Build Commands**:
99+
93100
```bash
94101
# Build all tools for all platforms
95102
bazel build //:all_tools
@@ -102,6 +109,7 @@ bazel build //tools/wasm-tools:wasm-tools-macos-arm64
102109
### 3. Platform Architecture
103110

104111
**Comprehensive Platform Support**:
112+
105113
```starlark
106114
# platforms/defs.bzl
107115
PLATFORM_MAPPINGS = {
@@ -110,7 +118,7 @@ PLATFORM_MAPPINGS = {
110118
"os": "linux", "arch": "x86_64", "suffix": "",
111119
},
112120
"//platforms:macos_arm64": {
113-
"rust_target": "aarch64-apple-darwin",
121+
"rust_target": "aarch64-apple-darwin",
114122
"os": "macos", "arch": "aarch64", "suffix": "",
115123
},
116124
# ... all 5 platforms
@@ -120,19 +128,21 @@ PLATFORM_MAPPINGS = {
120128
## Workflow
121129

122130
### Current State: Hermetic Success
123-
```
131+
132+
```text
124133
Main Workspace ──http_file──▶ GitHub Releases ──verified checksums──▶ ✅ BCR Compatible
125134
```
126135

127136
### Future State: Self-Hosted
128-
```
137+
138+
```text
129139
tools-builder/ ──build──▶ GitHub Releases ──publish──▶ Main Workspace ──download──▶ ✅ Complete Control
130140
```
131141

132142
## Benefits Achieved
133143

134144
1. **✅ Complete Hermeticity**: No external cargo registry dependencies
135-
2. **✅ BCR Compatibility**: All tests pass in sandboxed environment
145+
2. **✅ BCR Compatibility**: All tests pass in sandboxed environment
136146
3. **✅ Cross-Platform**: Supports all major development platforms
137147
4. **✅ Version Control**: Explicit tool versioning with checksum verification
138148
5. **✅ CI Efficiency**: Pre-built binaries eliminate build-time compilation
@@ -142,11 +152,13 @@ tools-builder/ ──build──▶ GitHub Releases ──publish──▶ Main
142152
## Implementation Files
143153

144154
### Modified Files
155+
145156
- `MODULE.bazel`: Added wac_hermetic and wkg_hermetic to use_repo
146157
- `toolchains/hermetic_extension.bzl`: Added http_file downloads for wac/wkg
147158
- `toolchains/BUILD.bazel`: Added filegroups for new hermetic tools
148159

149160
### New Files (Tool Builder Workspace)
161+
150162
- `tools-builder/MODULE.bazel`: Cross-compilation setup
151163
- `tools-builder/BUILD.bazel`: Tool suite orchestration
152164
- `tools-builder/README.md`: Complete documentation
@@ -174,9 +186,11 @@ The architecture is complete and working. Remaining work:
174186

175187
```bash
176188
# Test all hermetic tools
177-
bazel build //toolchains:wasm_tools_hermetic //toolchains:wit_bindgen_hermetic //toolchains:wasmtime_hermetic //toolchains:wac_hermetic //toolchains:wkg_hermetic
189+
bazel build //toolchains:wasm_tools_hermetic //toolchains:wit_bindgen_hermetic \
190+
//toolchains:wasmtime_hermetic //toolchains:wac_hermetic //toolchains:wkg_hermetic
178191

179192
# Result: ✅ All tools building successfully
180193
```
181194

182-
The solution successfully addresses the cargo sandbox issue while providing a scalable architecture for future tool management.
195+
The solution successfully addresses the cargo sandbox issue while providing a scalable architecture for
196+
future tool management.

WORKSPACE.bazel

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,5 @@
44

55
workspace(name = "rules_wasm_component")
66

7-
# Required for rules_rust
8-
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
9-
107
# Fallback to WORKSPACE mode if bzlmod is not available
118
# Most dependencies are handled in MODULE.bazel for modern Bazel versions

0 commit comments

Comments
 (0)