Skip to content

🔗 Integrate external bazel-file-ops-component for file operations #183

@avrabe

Description

@avrabe

Overview

The bazel-file-ops-component repository now provides production-ready, pre-built WebAssembly components for file operations. This issue tracks integrating these external components into rules_wasm_component to replace the embedded Go binary in tools/file_ops/.

Benefits

For rules_wasm_component

  • Smaller repository - Remove embedded file operations binary
  • Better separation of concerns - File ops maintained in dedicated repo
  • Improved security - Pre-built, signed components with SLSA provenance
  • Easier updates - Consume new versions via standard Bazel dependency management
  • Better testing - File ops component has dedicated CI/CD pipeline

For users

  • Cryptographic verification - All components signed with Cosign
  • Supply chain security - SLSA provenance attestation
  • Transparent builds - Public CI/CD pipeline
  • Faster builds - No need to compile file ops component locally

Current Status

Available Artifacts (as of v0.1.0-rc.2)

GitHub Releases:

OCI Registry:

  • Location: ghcr.io/pulseengine/bazel-file-ops-component
  • Tags: v0.1.0-rc.1, v0.1.0-rc.2, latest
  • Signed with Cosign (keyless GitHub OIDC)
  • SLSA provenance included

Security:

  • All releases signed with Cosign
  • Keyless signing via GitHub OIDC
  • SLSA provenance for supply chain verification
  • SHA256 checksums for integrity

Component Features

The external component provides all functionality currently in tools/file_ops/main.go:

  • ✅ JSON batch processing (process-json-batch)
  • ✅ Individual file operations (copy, move, delete, mkdir, etc.)
  • ✅ Path traversal protection
  • ✅ WASI Preview 2 compatible
  • ✅ Component Model v1 compliant

Required Changes

1. Add Dependency in MODULE.bazel

bazel_dep(
    name = "bazel-file-ops-component",
    version = "0.1.0",  # Use appropriate version
)

# Fetch pre-built WASM component
http_file = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file")

http_file(
    name = "file_ops_component_wasm",
    url = "https://github.com/pulseengine/bazel-file-ops-component/releases/download/v0.1.0/file_ops_component.wasm",
    sha256 = "<checksum>",
    downloaded_file_path = "file_ops_component.wasm",
)

Alternative using OCI registry (future):

oci_pull = use_repo_rule("@rules_oci//oci:pull.bzl", "oci_pull")

oci_pull(
    name = "file_ops_component_oci",
    image = "ghcr.io/pulseengine/bazel-file-ops-component",
    tag = "v0.1.0",
    # Cosign verification can be added here
)

2. Update tools/bazel_helpers/file_ops_actions.bzl

Modify the file operations functions to use external component:

def _get_file_ops_component(ctx):
    """Get file operations component (external or embedded)."""
    
    # Try external component first
    if hasattr(ctx.attr, "_file_ops_component"):
        return ctx.attr._file_ops_component.files.to_list()[0]
    
    # Fallback to embedded (for backward compatibility)
    return ctx.attr._file_ops_binary.files.to_list()[0]

# Update file_ops_action and related rules
def _file_ops_action_impl(ctx):
    component = _get_file_ops_component(ctx)
    
    # Use component with wasmtime
    # ... existing implementation ...

3. Add Configuration Options

Allow users to choose implementation:

# In MODULE.bazel or workspace configuration
file_ops_config(
    implementation = "auto",  # auto, external, embedded
    verify_signatures = True,  # Verify Cosign signatures
    oci_registry = "ghcr.io",
)

4. Update Toolchain Configuration

Modify toolchains/ to support external component selection.

5. Add Signature Verification (Optional but Recommended)

Integrate Cosign verification for downloaded components:

# Verify component signature before use
cosign_verify(
    component = "@file_ops_component_wasm//file:file_ops_component.wasm",
    certificate_identity_regexp = "https://github.com/pulseengine/bazel-file-ops-component",
    certificate_oidc_issuer = "https://token.actions.githubusercontent.com",
)

Migration Strategy

Phase 1: Add External Component (Optional)

  • Add external component as optional dependency
  • Keep embedded version as default
  • Allow users to opt-in via configuration
  • Goal: Test integration, gather feedback

Phase 2: Make External Default (Recommended Fallback)

  • Switch default to external component
  • Keep embedded version as fallback
  • Add deprecation notice for embedded version
  • Goal: Migrate most users to external component

Phase 3: Deprecate Embedded (Future)

  • Remove embedded tools/file_ops/ directory
  • Only support external component
  • Provide migration guide
  • Goal: Simplify codebase, complete migration

Phase 4: Remove Embedded (Next Major Version)

  • Complete removal of embedded file operations
  • External component only
  • Goal: Clean architecture

Testing Requirements

  • Integration tests with external component
  • Backward compatibility tests
  • Performance comparison (external vs embedded)
  • Signature verification tests
  • Fallback mechanism tests
  • Cross-platform testing (Linux, macOS)

Documentation Updates

  • Update README with new architecture
  • Document configuration options
  • Create migration guide
  • Update examples to use external component
  • Add security verification instructions

Timeline Suggestion

  1. Week 1-2: Add external component as optional dependency (Phase 1)
  2. Week 3-4: Testing and validation
  3. Week 5-6: Make external component default (Phase 2)
  4. Month 3: Deprecation notices for embedded version (Phase 3)
  5. Next Major Version: Remove embedded version (Phase 4)

Security Verification

Users can verify component authenticity:

# Verify OCI signature
cosign verify \
  --certificate-identity-regexp="https://github.com/pulseengine/bazel-file-ops-component" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
  ghcr.io/pulseengine/bazel-file-ops-component:v0.1.0

# Verify SLSA provenance
cosign verify-attestation \
  --type slsaprovenance \
  --certificate-identity-regexp="https://github.com/pulseengine/bazel-file-ops-component" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
  ghcr.io/pulseengine/bazel-file-ops-component:v0.1.0

# Verify SHA256 checksum
sha256sum -c file_ops_component.wasm.sha256

References

Questions?

Feel free to ask questions or request clarification on any aspect of this integration. The bazel-file-ops-component maintainers are ready to assist with the integration process.


Status: Ready for implementation
Priority: Medium (enhances security and modularity)
Complexity: Medium (well-defined interface, backward compatibility needed)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions