Skip to content

Commit 7a3dab0

Browse files
committed
refactor: consolidate dependency management with stratified hybrid approach
- Document dependency management patterns in CLAUDE.md with clear decision matrix - Simplify toolchain strategies to download-only (remove build/source/hybrid options) - Migrate wasmsign2-cli to JSON registry for consistent security auditing - Clean up JSON registry files to keep only latest + previous stable versions * wasm-tools: 4 versions → 2 (1.240.0, 1.239.0) * file-ops-component: 3 versions → 2 (rc.3, rc.2) * nodejs: 3 versions → 2 (20.18.0, 18.20.8) * wasi-sdk: 3 versions → 2 (27, 26) * wasmtime: 3 versions → 2 (37.0.2, 35.0.0) - Update MODULE.bazel to use download strategy for wasm_toolchain This establishes clear patterns: - Multi-platform GitHub binaries → JSON Registry + secure_download - Bazel Central Registry deps → bazel_dep - Source builds → git_repository (when needed) - Universal WASM binaries → JSON Registry - NPM packages → hermetic npm Benefits: faster builds (prebuilt binaries), easier security audits (central checksums), simpler maintenance (one strategy per tool), clear documentation.
1 parent bb88a20 commit 7a3dab0

File tree

10 files changed

+281
-154
lines changed

10 files changed

+281
-154
lines changed

CLAUDE.md

Lines changed: 240 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -295,6 +295,246 @@ http_archive(
295295
- **Path Handling**: Use Bazel's path utilities
296296
- **Platform Detection**: Use `@platforms//` constraints, not `uname`
297297

298+
## Dependency Management Patterns
299+
300+
### 🎯 RULE #2: STRATIFIED HYBRID APPROACH
301+
302+
**Use the RIGHT download pattern for each dependency category**
303+
304+
This project uses a **stratified hybrid approach** to dependency management, selecting the most appropriate mechanism based on the characteristics of each dependency type.
305+
306+
### Decision Matrix
307+
308+
| Dependency Type | Pattern | Location | Why |
309+
|----------------|---------|----------|-----|
310+
| **Multi-platform GitHub binaries** | JSON Registry + secure_download | `checksums/tools/*.json` | Solves platform × version matrix, central security auditing |
311+
| **Bazel Central Registry deps** | `bazel_dep` | `MODULE.bazel` | Ecosystem standard, automatic dependency resolution |
312+
| **Source builds** | `git_repository` | `wasm_tools_repositories.bzl` | Bazel standard, maximum flexibility |
313+
| **Universal WASM binaries** | JSON Registry (preferred) or `http_file` | `checksums/tools/*.json` or `MODULE.bazel` | Platform-independent, security auditable |
314+
| **NPM packages** | Hermetic npm + package.json | `toolchains/jco_toolchain.bzl` | Ecosystem standard, package lock files |
315+
316+
### Pattern 1: JSON Registry (Multi-Platform GitHub Binaries)
317+
318+
**Use for**: Tools with different binaries per platform (wasm-tools, wit-bindgen, wac, wkg, wasmtime, wizer, wasi-sdk, nodejs, tinygo)
319+
320+
**Why**: Elegantly handles the combinatorial explosion of (platforms × versions × URL patterns)
321+
322+
**Structure**:
323+
```json
324+
{
325+
"tool_name": "wasm-tools",
326+
"github_repo": "bytecodealliance/wasm-tools",
327+
"latest_version": "1.240.0",
328+
"supported_platforms": ["darwin_amd64", "darwin_arm64", "linux_amd64", "linux_arm64", "windows_amd64"],
329+
"versions": {
330+
"1.240.0": {
331+
"release_date": "2025-10-08",
332+
"platforms": {
333+
"darwin_arm64": {
334+
"sha256": "8959eb9f494af13868af9e13e74e4fa0fa6c9306b492a9ce80f0e576eb10c0c6",
335+
"url_suffix": "aarch64-macos.tar.gz"
336+
}
337+
// ... other platforms
338+
}
339+
}
340+
}
341+
}
342+
```
343+
344+
**Usage**:
345+
```python
346+
# In toolchain .bzl file
347+
from toolchains.secure_download import secure_download_tool
348+
349+
secure_download_tool(ctx, "wasm-tools", "1.240.0", platform)
350+
```
351+
352+
**Benefits**:
353+
- ✅ Single source of truth for all versions and checksums
354+
- ✅ Central security auditing (`checksums/` directory)
355+
- ✅ Supports multiple versions side-by-side
356+
- ✅ Platform detection and URL construction automatic
357+
- ✅ Clean API via `registry.bzl`
358+
359+
### Pattern 2: Bazel Central Registry (`bazel_dep`)
360+
361+
**Use for**: Standard Bazel ecosystem dependencies (rules_rust, bazel_skylib, platforms, rules_cc, etc.)
362+
363+
**Why**: Bazel's standard mechanism with automatic dependency resolution
364+
365+
**Structure**:
366+
```starlark
367+
# MODULE.bazel
368+
bazel_dep(name = "rules_rust", version = "0.65.0")
369+
bazel_dep(name = "bazel_skylib", version = "1.8.1")
370+
bazel_dep(name = "platforms", version = "1.0.0")
371+
```
372+
373+
**Benefits**:
374+
- ✅ Ecosystem standard - no learning curve
375+
- ✅ Automatic transitive dependency resolution
376+
- ✅ Maintained by Bazel team
377+
- ✅ Built-in security and version compatibility
378+
379+
**Do NOT**:
380+
- ❌ Duplicate BCR deps in JSON registry
381+
- ❌ Use http_archive for tools available in BCR
382+
383+
### Pattern 3: Git Repository (Source Builds)
384+
385+
**Use for**: Custom forks, bleeding edge versions, or when source builds are required
386+
387+
**Why**: Bazel-native source repository management
388+
389+
**Structure**:
390+
```starlark
391+
# wasm_tools_repositories.bzl
392+
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
393+
394+
git_repository(
395+
name = "wasm_tools_src",
396+
remote = "https://github.com/bytecodealliance/wasm-tools.git",
397+
tag = "v1.235.0",
398+
build_file = "//toolchains:BUILD.wasm_tools",
399+
)
400+
```
401+
402+
**When to use**:
403+
- Custom fork with patches
404+
- Need bleeding edge from main branch
405+
- Binary not available for your platform
406+
- Building from source is required for licensing
407+
408+
**Prefer download over build**: When prebuilt binaries are available and work correctly, use Pattern 1 (JSON Registry) instead for faster, more hermetic builds.
409+
410+
### Pattern 4: Universal WASM Binaries
411+
412+
**Use for**: WebAssembly components (platform-independent .wasm files)
413+
414+
**Preferred**: JSON Registry for consistency and security auditing
415+
```json
416+
// checksums/tools/file-ops-component.json
417+
{
418+
"tool_name": "file-ops-component",
419+
"github_repo": "pulseengine/bazel-file-ops-component",
420+
"latest_version": "0.1.0-rc.3",
421+
"supported_platforms": ["wasm"], // Universal
422+
"versions": {
423+
"0.1.0-rc.3": {
424+
"release_date": "2025-10-15",
425+
"platforms": {
426+
"wasm": {
427+
"sha256": "8a9b1aa8a2c9d3dc36f1724ccbf24a48c473808d9017b059c84afddc55743f1e",
428+
"url": "https://github.com/.../file_ops_component.wasm"
429+
}
430+
}
431+
}
432+
}
433+
}
434+
```
435+
436+
**Alternative**: `http_file` for very simple cases (legacy)
437+
```starlark
438+
# MODULE.bazel (only for simple cases)
439+
http_file(
440+
name = "component_external",
441+
url = "https://github.com/.../component.wasm",
442+
sha256 = "abc123...",
443+
downloaded_file_path = "component.wasm",
444+
)
445+
```
446+
447+
**Recommendation**: Migrate all WASM components to JSON Registry for:
448+
- Consistent security auditing
449+
- Version management
450+
- Same tooling as other downloads
451+
452+
### Pattern 5: NPM Packages
453+
454+
**Use for**: Node.js ecosystem tools (jco, componentize-js)
455+
456+
**Why**: npm is the standard package manager with lock file support
457+
458+
**Structure**:
459+
```python
460+
# Download hermetic Node.js first (Pattern 1)
461+
secure_download_tool(ctx, "nodejs", "20.18.0", platform)
462+
463+
# Use hermetic npm for package installation
464+
ctx.execute([npm_path, "install", "@bytecodealliance/[email protected]"])
465+
```
466+
467+
**Benefits**:
468+
- ✅ Hermetic builds (no system Node.js dependency)
469+
- ✅ Package lock files for reproducibility
470+
- ✅ Ecosystem standard
471+
472+
### Adding New Dependencies
473+
474+
**Decision Tree**:
475+
476+
1. **Is it in Bazel Central Registry?**
477+
- YES → Use `bazel_dep` (Pattern 2)
478+
- NO → Continue to step 2
479+
480+
2. **Is it a GitHub release with platform-specific binaries?**
481+
- YES → Create JSON in `checksums/tools/` (Pattern 1)
482+
- NO → Continue to step 3
483+
484+
3. **Is it a universal WASM component?**
485+
- YES → Create JSON in `checksums/tools/` with platform "wasm" (Pattern 4)
486+
- NO → Continue to step 4
487+
488+
4. **Is it an NPM package?**
489+
- YES → Use hermetic npm installation (Pattern 5)
490+
- NO → Continue to step 5
491+
492+
5. **Must it be built from source?**
493+
- YES → Use `git_repository` (Pattern 3)
494+
- NO → Reconsider if this dependency is needed
495+
496+
### Security Best Practices
497+
498+
1. **Always verify checksums**: All downloads MUST have SHA256 verification
499+
2. **Central audit trail**: Prefer JSON registry for auditability
500+
3. **Version pinning**: Always specify exact versions, never use "latest"
501+
4. **Minimal versions**: Keep only latest stable + previous stable in JSON files
502+
5. **Review changes**: All checksum changes require careful PR review
503+
504+
### Maintenance Guidelines
505+
506+
**Adding a new version to JSON registry**:
507+
```bash
508+
# 1. Download binaries for all platforms
509+
# 2. Calculate SHA256 checksums
510+
shasum -a 256 wasm-tools-1.241.0-*.tar.gz
511+
512+
# 3. Add version block to JSON file
513+
# 4. Update "latest_version" if appropriate
514+
# 5. Remove old versions if keeping only latest + previous
515+
```
516+
517+
**Updating a BCR dependency**:
518+
```starlark
519+
# Simply change version in MODULE.bazel
520+
bazel_dep(name = "rules_rust", version = "0.66.0") # Updated
521+
```
522+
523+
**Removing old versions**:
524+
- Keep latest stable version
525+
- Keep previous stable version (for rollback capability)
526+
- Remove all older versions
527+
- Update tests if they pin to old versions
528+
529+
### Anti-Patterns to Avoid
530+
531+
**DO NOT** create custom download mechanisms
532+
**DO NOT** hardcode URLs in .bzl files
533+
**DO NOT** duplicate BCR dependencies in JSON registry
534+
**DO NOT** use http_archive for multi-platform binaries (use JSON registry)
535+
**DO NOT** keep more than 2 versions per tool without strong justification
536+
**DO NOT** use "strategy options" - pick ONE best approach per tool
537+
298538
## Current State
299539

300540
### Toolchains Implemented

MODULE.bazel

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ use_repo(wasi_wit_ext, "wasi_cli", "wasi_cli_v020", "wasi_clocks", "wasi_clocks_
7272
wasm_toolchain = use_extension("//wasm:extensions.bzl", "wasm_toolchain")
7373
wasm_toolchain.register(
7474
name = "wasm_tools",
75-
strategy = "hybrid", # Hybrid strategy: download binaries + build wasmsign2 from source
75+
strategy = "download", # Download prebuilt binaries from GitHub releases
7676
version = "1.240.0",
7777
)
7878
use_repo(wasm_toolchain, "wasm_tools_toolchains")

checksums/registry.bzl

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -811,6 +811,8 @@ def list_available_tools():
811811
"wizer",
812812
"nodejs",
813813
"jco",
814+
"file-ops-component",
815+
"wasmsign2-cli",
814816
]
815817

816818
def validate_tool_compatibility(tools_config):

checksums/tools/file-ops-component.json

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -35,16 +35,6 @@
3535
"size_kb": 853
3636
}
3737
}
38-
},
39-
"0.1.0-rc.1": {
40-
"release_date": "2024-10-24",
41-
"description": "Initial test release",
42-
"platforms": {
43-
"wasm_component": {
44-
"sha256": "UNKNOWN",
45-
"url_suffix": "file_ops_component.wasm"
46-
}
47-
}
4838
}
4939
},
5040
"security": {

checksums/tools/nodejs.json

Lines changed: 0 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -5,41 +5,6 @@
55
"last_checked": "2025-08-17T10:30:00Z",
66
"note": "Node.js runtime for hermetic jco toolchain installation",
77
"versions": {
8-
"18.19.0": {
9-
"release_date": "2024-01-09",
10-
"platforms": {
11-
"darwin_amd64": {
12-
"sha256": "0a749fcdf5d6bf46e1c17b3ea01e050b4d1ec3f3073b14aa745527b45a759c74",
13-
"url_suffix": "darwin-x64.tar.gz",
14-
"binary_path": "node-v{}-darwin-x64/bin/node",
15-
"npm_path": "node-v{}-darwin-x64/bin/npm"
16-
},
17-
"darwin_arm64": {
18-
"sha256": "8907c42a968765b77730fb319458d63ec4ed009265f8012097c3a052407aa99b",
19-
"url_suffix": "darwin-arm64.tar.gz",
20-
"binary_path": "node-v{}-darwin-arm64/bin/node",
21-
"npm_path": "node-v{}-darwin-arm64/bin/npm"
22-
},
23-
"linux_amd64": {
24-
"sha256": "61632bb78ee828d6e8f42adc0bc2238a6b8200007093988d3927176a372281e8",
25-
"url_suffix": "linux-x64.tar.xz",
26-
"binary_path": "node-v{}-linux-x64/bin/node",
27-
"npm_path": "node-v{}-linux-x64/bin/npm"
28-
},
29-
"linux_arm64": {
30-
"sha256": "cf94ab72e45b855257545fec1c017bdf30a9e23611561382eaf64576b999e72d",
31-
"url_suffix": "linux-arm64.tar.xz",
32-
"binary_path": "node-v{}-linux-arm64/bin/node",
33-
"npm_path": "node-v{}-linux-arm64/bin/npm"
34-
},
35-
"windows_amd64": {
36-
"sha256": "5311913d45e1fcc3643c58d1e3926eb85437b180f025fe5857413c9f02403645",
37-
"url_suffix": "win-x64.zip",
38-
"binary_path": "node-v{}-win-x64/node.exe",
39-
"npm_path": "node-v{}-win-x64/npm.cmd"
40-
}
41-
}
42-
},
438
"18.20.8": {
449
"release_date": "2024-09-03",
4510
"platforms": {

checksums/tools/wasi-sdk.json

Lines changed: 0 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -53,31 +53,6 @@
5353
"url_suffix": "windows.tar.gz"
5454
}
5555
}
56-
},
57-
"25": {
58-
"release_date": "2024-11-01",
59-
"platforms": {
60-
"darwin_amd64": {
61-
"sha256": "55e3ff3fee1a15678a16eeccba0129276c9f6be481bc9c283e7f9f65bf055c11",
62-
"url_suffix": "macos.tar.gz"
63-
},
64-
"darwin_arm64": {
65-
"sha256": "e1e529ea226b1db0b430327809deae9246b580fa3cae32d31c82dfe770233587",
66-
"url_suffix": "macos.tar.gz"
67-
},
68-
"linux_amd64": {
69-
"sha256": "52640dde13599bf127a95499e61d6d640256119456d1af8897ab6725bcf3d89c",
70-
"url_suffix": "linux.tar.gz"
71-
},
72-
"linux_arm64": {
73-
"sha256": "52640dde13599bf127a95499e61d6d640256119456d1af8897ab6725bcf3d89c",
74-
"url_suffix": "linux.tar.gz"
75-
},
76-
"windows_amd64": {
77-
"sha256": "PLACEHOLDER_NEEDS_REAL_CHECKSUM_64_CHARS_XXXXXXXXXXXXXXXX",
78-
"url_suffix": "windows.tar.gz"
79-
}
80-
}
8156
}
8257
}
8358
}

0 commit comments

Comments
 (0)