Skip to content

Commit a701d0c

Browse files
committed
refactor: eliminate system strategy from all toolchains for hermetic builds
- Remove system strategy support from 7 toolchains (wasm, jco, wkg, wasmtime, etc) - Update all toolchain defaults from system to hermetic alternatives (hybrid, download, npm) - Remove system tool detection and PATH-based discovery logic - Update toolchain validation to exclude system from supported strategies - Modernize tools-builder workspace with cargo genrule approach - Update documentation to reflect hermetic-only build requirements This ensures complete build hermeticity by eliminating non-reproducible system tool dependencies across the entire toolchain ecosystem.
1 parent 0ea155b commit a701d0c

File tree

15 files changed

+112
-404
lines changed

15 files changed

+112
-404
lines changed

TOOL_BUILDER_SOLUTION.md

Lines changed: 10 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,9 @@ The main issue was **cargo filesystem sandbox restrictions** in Bazel Central Re
1313

1414
### Dual-Track Approach
1515

16-
1. **Immediate Solution: Hermetic Pre-built Binaries**
17-
-Added wac and wkg to `toolchains/hermetic_extension.bzl`
18-
-Using `http_file` for single binary downloads with verified checksums
16+
1. **Immediate Solution: Hermetic Download Strategies**
17+
-All tools use download strategy with verified checksums
18+
-Cross-platform support for all major platforms
1919
- ✅ All 5 core tools (wasm-tools, wit-bindgen, wasmtime, wac, wkg) working
2020

2121
2. **Long-term Solution: Self-Hosted Tool Builder Workspace**
@@ -71,13 +71,10 @@ tools-builder/
7171
**Implementation**:
7272

7373
```starlark
74-
# toolchains/hermetic_extension.bzl
75-
http_file(
76-
name = "wac_hermetic",
77-
urls = ["https://github.com/bytecodealliance/wac/releases/download/v0.7.0/wac-cli-x86_64-unknown-linux-musl"],
78-
sha256 = "dd734c4b049287b599a3f8c553325307687a17d070290907e3d5bbe481b89cc6",
79-
executable = True,
80-
downloaded_file_path = "wac",
74+
# wasm/extensions.bzl
75+
wasm_toolchain.register(
76+
strategy = "download",
77+
version = "1.235.0", # wasm-tools, wit-bindgen, wac all downloaded with verified checksums
8178
)
8279
```
8380

@@ -153,9 +150,9 @@ tools-builder/ ──build──▶ GitHub Releases ──publish──▶ Main
153150

154151
### Modified Files
155152

156-
- `MODULE.bazel`: Added wac_hermetic and wkg_hermetic to use_repo
157-
- `toolchains/hermetic_extension.bzl`: Added http_file downloads for wac/wkg
158-
- `toolchains/BUILD.bazel`: Added filegroups for new hermetic tools
153+
- `wasm/extensions.bzl`: Updated toolchain defaults to use download strategy
154+
- `toolchains/*.bzl`: Enhanced download strategies with cross-platform support
155+
- `MODULE.bazel`: Uses standard toolchain download strategies
159156

160157
### New Files (Tool Builder Workspace)
161158

docs-site/src/content/docs/getting-started.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ Now that you have your first component working, explore more advanced features:
119119
- [**JavaScript Components**](/languages/javascript/) - ComponentizeJS integration
120120

121121
### Advanced Topics
122+
- [**Toolchain Configuration**](/guides/toolchain-configuration/) - Strategies, versions, CI/CD setup
122123
- [**Component Composition**](/composition/wac/) - Building multi-component systems
123124
- [**OCI Publishing**](/production/publishing/) - Distribute via container registries
124125
- [**Performance Optimization**](/production/performance/) - Wizer pre-initialization

docs-site/src/content/docs/installation.md

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -86,16 +86,13 @@ wasm_toolchain = use_extension(
8686
)
8787
wasm_toolchain.register(
8888
name = "wasm_tools",
89-
version = "1.0.60", # Optional, defaults to latest stable
90-
)
91-
92-
# Configure wit-bindgen version
93-
wasm_toolchain.register(
94-
name = "wit_bindgen",
95-
version = "0.30.0", # Optional, defaults to latest stable
89+
strategy = "download", # Default: hermetic builds
90+
version = "1.235.0", # Pin specific version
9691
)
9792
```
9893

94+
> **💡 Need advanced configuration?** See the [Toolchain Configuration Guide](/guides/toolchain-configuration/) for strategies, version management, CI/CD setup, and corporate environments.
95+
9996
## Language-Specific Setup
10097

10198
### Rust Configuration
@@ -167,7 +164,7 @@ For JavaScript/TypeScript components:
167164
# Configure jco (JavaScript Component Tools)
168165
jco = use_extension("@rules_wasm_component//wasm:extensions.bzl", "jco")
169166
jco.register(
170-
strategy = "npm", # or "system" or "download"
167+
strategy = "npm", # or "download"
171168
version = "1.4.0", # Optional for npm/download strategies
172169
)
173170
```

docs/toolchain_configuration.md

Lines changed: 25 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
# Toolchain Configuration
22

3-
The rules_wasm_component supports flexible toolchain configuration with three acquisition strategies: system tools, downloaded binaries, and building from source.
3+
The rules_wasm_component supports flexible toolchain configuration with multiple acquisition strategies: downloaded binaries, building from source, and hybrid approaches.
44

55
## Quick Reference
66

7-
### Use System Tools (Default)
7+
### Use Downloaded Tools (Default)
88

99
```starlark
10-
# MODULE.bazel - Uses tools from PATH (CI-friendly)
10+
# MODULE.bazel - Downloads prebuilt binaries for hermetic builds
1111
bazel_dep(name = "rules_wasm_component", version = "0.1.0")
1212
```
1313

@@ -39,26 +39,28 @@ wasm_toolchain.register(
3939

4040
## Configuration Options
4141

42-
### Strategy: `"system"` (Default)
42+
### Strategy: `"download"` (Default)
4343

44-
Uses tools installed on the system PATH. Perfect for:
44+
Downloads prebuilt binaries from GitHub releases for hermetic builds. Perfect for:
4545

46-
- CI environments where tools are pre-installed
47-
- Developer machines with `cargo install wasm-tools wac-cli wit-bindgen-cli`
48-
- Consistent with existing CI setup
46+
- Reproducible builds with pinned versions
47+
- Corporate environments without system tool dependencies
48+
- CI environments requiring hermeticity
4949

5050
```starlark
5151
wasm_toolchain.register(
52-
strategy = "system",
52+
strategy = "download",
53+
version = "1.235.0",
5354
)
5455
```
5556

56-
**Requirements:**
57+
**Benefits:**
5758

58-
- `wasm-tools`, `wac`, and `wit-bindgen` must be in PATH
59-
- No version control - uses whatever is installed
59+
- Hermetic builds - no system dependencies
60+
- Version control with pinned releases
61+
- Works across all supported platforms
6062

61-
### Strategy: `"download"`
63+
### Strategy: `"download"` (Explicit)
6264

6365
Downloads prebuilt binaries from GitHub releases:
6466

@@ -128,10 +130,11 @@ You can register multiple toolchains for different purposes:
128130
# MODULE.bazel
129131
wasm_toolchain = use_extension("@rules_wasm_component//wasm:extensions.bzl", "wasm_toolchain")
130132

131-
# System tools for CI/development
133+
# Download tools for CI/development
132134
wasm_toolchain.register(
133-
name = "system_tools",
134-
strategy = "system",
135+
name = "ci_tools",
136+
strategy = "download",
137+
version = "1.235.0",
135138
)
136139

137140
# Pinned version for production
@@ -163,16 +166,12 @@ bazel build --extra_toolchains=@dev_tools_toolchains//:wasm_tools_toolchain //..
163166

164167
### GitHub Actions (Recommended)
165168

166-
Uses system strategy with pre-installed tools:
169+
Uses download strategy for hermetic builds:
167170

168171
```yaml
169172
# .github/workflows/ci.yml
170-
- name: Install WASM tools
171-
run: |
172-
cargo install wasm-tools wac-cli wit-bindgen-cli
173-
174173
- name: Build with Bazel
175-
run: bazel build //... # Uses system tools automatically
174+
run: bazel build //... # Downloads tools automatically for hermetic builds
176175
```
177176
178177
### Docker Builds
@@ -211,7 +210,7 @@ wasm_toolchain.register(
211210

212211
## Migration Examples
213212

214-
### From CI Script to System Strategy
213+
### From CI Script to Hermetic Downloads
215214

216215
**Before:**
217216

@@ -225,8 +224,7 @@ cargo install wasm-tools wac-cli wit-bindgen-cli
225224

226225
```yaml
227226
# .github/workflows/ci.yml
228-
- run: cargo install wasm-tools wac-cli wit-bindgen-cli
229-
- run: bazel build //... # Automatically uses system tools
227+
- run: bazel build //... # Automatically downloads tools for hermetic builds
230228
```
231229
232230
### From Fixed Version to Flexible Strategy
@@ -245,14 +243,9 @@ wasm_toolchain.register(strategy = "download", version = "1.235.0")
245243

246244
## Troubleshooting
247245

248-
### "Tool not found" with system strategy
249-
250-
```bash
251-
# Install missing tools
252-
cargo install wasm-tools wac-cli wit-bindgen-cli
246+
### "Tool not found" errors
253247

254-
# Or switch to download strategy
255-
```
248+
All tools are now downloaded automatically for hermetic builds. If you encounter issues, verify network connectivity and consider using custom URLs for corporate environments.
256249

257250
### "Download failed" with download strategy
258251

examples/js_component/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ The jco toolchain can be configured using different strategies:
7575
```starlark
7676
# MODULE.bazel
7777
jco = use_extension("@rules_wasm_component//wasm:extensions.bzl", "jco")
78-
jco.register(strategy = "system")
78+
jco.register(strategy = "npm")
7979
```
8080

8181
### NPM Installation

platforms/defs.bzl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,4 +6,4 @@
66
# Standard platform definitions already exist in platforms/BUILD.bazel
77
# This file can be used for any cross-workspace platform logic if needed
88

9-
# For now, this is a minimal file to satisfy load dependencies
9+
# For now, this is a minimal file to satisfy load dependencies

toolchains/jco_toolchain.bzl

Lines changed: 5 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -106,51 +106,20 @@ def _jco_toolchain_repository_impl(repository_ctx):
106106
platform = _detect_host_platform(repository_ctx)
107107
version = repository_ctx.attr.version
108108

109-
if strategy == "system":
110-
_setup_system_jco_tools(repository_ctx)
111-
elif strategy == "download":
109+
if strategy == "download":
112110
_setup_downloaded_jco_tools(repository_ctx, platform, version)
113111
elif strategy == "npm":
114112
_setup_npm_jco_tools(repository_ctx)
115113
else:
116114
fail(format_diagnostic_error(
117115
"E001",
118116
"Unknown jco strategy: {}".format(strategy),
119-
"Must be 'system', 'download', or 'npm'",
117+
"Must be 'download' or 'npm'",
120118
))
121119

122120
# Create BUILD files
123121
_create_jco_build_files(repository_ctx)
124122

125-
def _setup_system_jco_tools(repository_ctx):
126-
"""Set up system-installed jco tools"""
127-
128-
# Validate system tools
129-
tools = [("jco", "jco"), ("node", "node"), ("npm", "npm")]
130-
131-
for tool_name, binary_name in tools:
132-
validation_result = validate_system_tool(repository_ctx, binary_name)
133-
134-
if not validation_result["valid"]:
135-
fail(validation_result["error"])
136-
137-
if "warning" in validation_result:
138-
print(validation_result["warning"])
139-
140-
# Create symlink to system tool (Bazel-native approach)
141-
tool_path = validation_result.get("path", repository_ctx.which(binary_name))
142-
if tool_path:
143-
repository_ctx.symlink(tool_path, tool_name)
144-
else:
145-
# Fallback: use PATH resolution
146-
repository_ctx.file(tool_name, binary_name, executable = True)
147-
148-
print("Using system {}: {} at {}".format(
149-
tool_name,
150-
binary_name,
151-
validation_result.get("path", "system PATH"),
152-
))
153-
154123
def _setup_downloaded_jco_tools(repository_ctx, platform, version):
155124
"""Download prebuilt jco tools"""
156125

@@ -371,9 +340,9 @@ jco_toolchain_repository = repository_rule(
371340
implementation = _jco_toolchain_repository_impl,
372341
attrs = {
373342
"strategy": attr.string(
374-
doc = "Tool acquisition strategy: 'system', 'download', or 'npm'",
375-
default = "system",
376-
values = ["system", "download", "npm"],
343+
doc = "Tool acquisition strategy: 'download' or 'npm'",
344+
default = "npm",
345+
values = ["download", "npm"],
377346
),
378347
"version": attr.string(
379348
doc = "jco version to use",

toolchains/wasm_toolchain.bzl

Lines changed: 9 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -134,9 +134,7 @@ def _wasm_toolchain_repository_impl(repository_ctx):
134134
for warning in compatibility_warnings:
135135
print("Warning: {}".format(warning))
136136

137-
if strategy == "system":
138-
_setup_system_tools_enhanced(repository_ctx)
139-
elif strategy == "download":
137+
if strategy == "download":
140138
_setup_downloaded_tools(repository_ctx) # Use simple method for stability
141139
elif strategy == "build":
142140
_setup_built_tools_enhanced(repository_ctx)
@@ -148,41 +146,12 @@ def _wasm_toolchain_repository_impl(repository_ctx):
148146
fail(format_diagnostic_error(
149147
"E001",
150148
"Unknown strategy: {}".format(strategy),
151-
"Must be 'system', 'download', 'build', 'bazel', or 'hybrid'",
149+
"Must be 'download', 'build', 'bazel', or 'hybrid'",
152150
))
153151

154152
# Create BUILD files for all strategies
155153
_create_build_files(repository_ctx)
156154

157-
def _setup_system_tools_enhanced(repository_ctx):
158-
"""Set up system-installed tools from PATH with validation"""
159-
160-
tools = ["wasm-tools", "wac", "wit-bindgen", "wrpc", "wasmsign2"]
161-
162-
for tool_name in tools:
163-
# Validate system tool
164-
validation_result = validate_system_tool(repository_ctx, tool_name)
165-
166-
if not validation_result["valid"]:
167-
fail(validation_result["error"])
168-
169-
if "warning" in validation_result:
170-
print(validation_result["warning"])
171-
172-
# Create wrapper executable
173-
repository_ctx.file(tool_name, """#!/bin/bash
174-
exec {} "$@"
175-
""".format(tool_name), executable = True)
176-
177-
print("Using system tool: {} at {}".format(
178-
tool_name,
179-
validation_result.get("path", "system PATH"),
180-
))
181-
182-
def _setup_system_tools(repository_ctx):
183-
"""Set up system-installed tools from PATH (legacy)"""
184-
_setup_system_tools_enhanced(repository_ctx)
185-
186155
def _setup_downloaded_tools_enhanced(repository_ctx):
187156
"""Download prebuilt tools with enhanced error handling and caching"""
188157

@@ -817,8 +786,10 @@ def _setup_bazel_native_tools(repository_ctx):
817786

818787
# Create placeholder wac (will be updated to rust_binary later)
819788
repository_ctx.file("wac", """#!/bin/bash
820-
echo "wac: using fallback to hermetic binary"
821-
exec @wac_hermetic//:wac "$@"
789+
echo "wac: Bazel-native rust_binary not yet implemented"
790+
echo "Using placeholder - switch to 'download' strategy in MODULE.bazel for full functionality"
791+
echo "To use wac, switch to 'download' strategy in MODULE.bazel"
792+
exit 1
822793
""", executable = True)
823794

824795
# Create placeholder wit-bindgen (will be updated to rust_binary later)
@@ -1054,9 +1025,9 @@ wasm_toolchain_repository = repository_rule(
10541025
implementation = _wasm_toolchain_repository_impl,
10551026
attrs = {
10561027
"strategy": attr.string(
1057-
doc = "Tool acquisition strategy: 'system', 'download', 'build', 'bazel', or 'hybrid'",
1058-
default = "system",
1059-
values = ["system", "download", "build", "bazel", "hybrid"],
1028+
doc = "Tool acquisition strategy: 'download', 'build', 'bazel', or 'hybrid'",
1029+
default = "hybrid",
1030+
values = ["download", "build", "bazel", "hybrid"],
10601031
),
10611032
"version": attr.string(
10621033
doc = "Version to use (for download/build strategies)",

toolchains/wasm_tools_repositories.bzl

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,4 +55,12 @@ def register_wasm_tool_repositories():
5555
build_file = "//toolchains:BUILD.wizer",
5656
)
5757

58+
# wasmsign2: WebAssembly component signing tool
59+
git_repository(
60+
name = "wasmsign2_src",
61+
remote = "https://github.com/wasm-signatures/wasmsign2.git",
62+
tag = "0.2.6",
63+
build_file = "//toolchains:BUILD.wasmsign2",
64+
)
65+
5866
print("✅ Modernized WASM tool repositories registered - replaced all git clone operations")

0 commit comments

Comments
 (0)