feat: ReaperPod CRD, reaper-controller, and Helm chart (#31)

* feat(controller): add ReaperPod CRD and reaper-controller binary

Introduce a ReaperPod custom resource that provides a simplified,
Reaper-native way to run workloads. A new reaper-controller binary
watches ReaperPod resources and creates real Pods with runtimeClassName
pre-configured, translating the simplified spec (inline volumes,
dnsMode, overlayName) into full Pod specs with owner references.

- CRD types with own simple structs (avoids k8s-openapi schemars dep)
- Pod builder with 9 unit tests covering all field translations
- Reconciler using kube::runtime::Controller with owns() for Pod watches
- Deployment manifests, Dockerfile, build/generate scripts
- Example YAMLs (simple-task, with-volumes, with-node-selector)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test(controller): add Kind integration tests for ReaperPod CRD

Phase 4b tests: CRD installation, controller deployment, ReaperPod
creation with Pod verification, status mirroring, exit code propagation,
annotation translation, kubectl column output, and GC on delete.

Also updates build-controller-image.sh to match agent build pattern
(--cluster-name, --skip-build, --quiet) and wires controller image
build into phase_setup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update CRD plan — all 10 steps complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(test): add --crd-only flag for fast CRD controller test iteration

Skips cargo tests, integration tests, and agent tests — runs only
Phase 4b controller tests (CRD install, ReaperPod lifecycle, status
mirroring, annotation translation, GC).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(test): ensure reaper-system namespace exists for --crd-only runs

The controller deployment targets the reaper-system namespace, which is
normally created during agent setup. When running with --crd-only, agent
setup is skipped, causing "namespace not found" errors. Create the
namespace idempotently before applying the controller manifest.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update examples and docs for CRD controller tests

- CLAUDE.md: add controller binary, CRD types, scripts, and example 09
  to project structure; document Phase 4b in integration test phases
- TESTING.md: add --crd-only and --agent-only flags; add Phase 4b
  controller tests; add CRD iteration example
- examples/README.md: add 09-reaperpod section with usage instructions
- examples/09-reaperpod/simple-task.yaml: add namespace prerequisite
- docs/CRD_PLAN.md: note namespace fix for --crd-only runs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(playground): add --with-controller flag for ReaperPod CRD setup

Installs the ReaperPod CRD, builds and loads the controller image into
Kind, and deploys the reaper-controller. Adds CRD usage examples to the
playground summary output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(playground): make CRD and controller setup the default

The ReaperPod CRD and reaper-controller are now installed by default
when setting up a playground cluster. Remove the --with-controller flag
since this is the standard mode of operation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(playground): add --context to all kubectl examples in summary

Use --context=kind-<cluster-name> in all example commands so they work
correctly when multiple Kind clusters are running. Also add an inline
ReaperPod one-liner example.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(controller): use fixed Pod name matching ReaperPod name

Replace generateName with a fixed name so the Pod created by the
controller has the same name as the ReaperPod. This enables
`kubectl logs <reaperpod-name>` to work directly without needing
to look up the Pod name from the ReaperPod status.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(helm): replace Ansible with Helm chart for installation

Add a Helm chart (deploy/helm/reaper/) that installs everything needed
to run Reaper: node DaemonSet with init container for binary installation,
controller Deployment, CRDs, RuntimeClass, and RBAC.

- Add Dockerfile.node and scripts/install-node.sh for init container
- Add scripts/build-node-image.sh for Kind image builds
- Rewrite setup-playground.sh to use Helm instead of Ansible
- Update test-phases.sh to use Helm-based setup
- Make CRD/controller tests idempotent (work with Helm or standalone)
- Mark Ansible deployment as DEPRECATED (kept but no longer default)
- Update README.md and CLAUDE.md to reflect Helm as default

All 8 CRD integration tests pass with the new Helm-based flow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update all references from Ansible to Helm

Update TESTING.md, RELEASING.md, and deploy/kubernetes/README.md to
reflect Helm as the default installation method. Mark Ansible references
as deprecated throughout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: remove playground-release job (--release flag removed)

setup-playground.sh no longer supports --release (Helm handles
installation). Remove the CI job that tested release resolution modes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: build reaper-controller in CI, replace Ansible with Helm

- Add --features controller --bin reaper-controller to musl build step
- Include reaper-controller in uploaded artifacts
- Replace 'Install Ansible' step with 'Install Helm' for kind-integration

The controller binary was missing from CI artifacts, causing
build-controller-image.sh --skip-build to fail in kind-integration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(test): replace yield_now with 50ms sleep to avoid ETXTBSY flake

yield_now() is insufficient under CI load — the kernel may still hold
the inode after chmod. A small sleep reliably avoids the ETXTBSY race
when exec'ing a freshly-written shell script.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: increase kind-integration timeout from 20 to 30 minutes

With CRD controller tests (Phase 4b) added on top of agent GC tests
(some taking 90-120s each), 20 minutes is too tight. Increase to 30
minutes to give the full suite room to complete.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add CI test runtime optimization to TODO

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: retrigger CI pipeline

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Author: miguelgila

.github/workflows/ci.yml

@@ -95,10 +95,11 @@ jobs:
             -v ${{ github.workspace }}:/work \
             -w /work \
             messense/rust-musl-cross:x86_64-musl \
-            cargo build --release --features agent \
+            cargo build --release --features agent,controller \
               --bin containerd-shim-reaper-v2 \
               --bin reaper-runtime \
               --bin reaper-agent \
+              --bin reaper-controller \
               --target x86_64-unknown-linux-musl
       - uses: actions/upload-artifact@v4
         with:
@@ -107,6 +108,7 @@ jobs:
             target/x86_64-unknown-linux-musl/release/containerd-shim-reaper-v2
             target/x86_64-unknown-linux-musl/release/reaper-runtime
             target/x86_64-unknown-linux-musl/release/reaper-agent
+            target/x86_64-unknown-linux-musl/release/reaper-controller
           retention-days: 1
 
   kind-integration:
@@ -136,13 +138,15 @@ jobs:
           chmod +x ./kind
           sudo mv ./kind /usr/local/bin/kind
 
-      - name: Install Ansible
-        run: pip install ansible
+      - name: Install Helm
+        uses: azure/setup-helm@v4
+        with:
+          version: "v3.14.0"
 
       - name: Run integration tests
         uses: nick-fields/retry@v3
         with:
-          timeout_minutes: 20
+          timeout_minutes: 30
           max_attempts: 2
           command: ./scripts/run-integration-tests.sh --skip-cargo
 
@@ -155,10 +159,6 @@ jobs:
           path: /tmp/reaper-integration-logs/
           retention-days: 7
 
-  playground-release:
-    name: Playground Release Tests
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - name: Test --release resolution modes
-        run: ./scripts/test-playground-release.sh
+  # playground-release job removed: setup-playground.sh now uses Helm
+  # instead of --release flag. The release-utils.sh library is retained
+  # for the deprecated install-reaper.sh script.

CLAUDE.md

@@ -190,13 +190,18 @@ reaper/
 ├── src/
 │   ├── config.rs                    # Shared config file loader (/etc/reaper/reaper.conf)
 │   ├── annotations.rs               # Shared pod annotation parsing and validation
+│   ├── crds/                        # ReaperPod CRD types (feature-gated: controller)
 │   └── bin/
 │       ├── containerd-shim-reaper-v2/
 │       │   └── main.rs              # Shim implementation (ttrpc server, Task trait)
-│   └── reaper-runtime/
-│       ├── main.rs              # OCI runtime CLI (fork-first architecture)
-│       ├── state.rs             # State persistence (/run/reaper/<id>/)
-│       └── overlay.rs           # Overlay filesystem (Linux-only)
+│       ├── reaper-controller/       # CRD controller binary
+│       │   ├── main.rs              # Entry point, CRD watcher setup
+│       │   ├── reconciler.rs        # ReaperPod → Pod reconciliation
+│       │   └── pod_builder.rs       # ReaperPod spec → Pod spec translation
+│       └── reaper-runtime/
+│           ├── main.rs              # OCI runtime CLI (fork-first architecture)
+│           ├── state.rs             # State persistence (/run/reaper/<id>/)
+│           └── overlay.rs           # Overlay filesystem (Linux-only)
 ├── tests/                       # Integration tests
 │   ├── integration_basic_binary.rs
 │   ├── integration_io.rs        # FIFO stdout/stderr
@@ -213,15 +218,28 @@ reaper/
 │   ├── 05-kubemix/              # Jobs, DaemonSets, and Deployments on 10-node cluster
 │   ├── 06-ansible-jobs/         # Sequential Jobs: install Ansible, then run playbook
 │   ├── 07-ansible-complex/     # DaemonSet bootstrap + role-based Ansible playbooks
-│   └── 08-mix-container-runtime-engines/  # Mixed runtimes: OpenLDAP (default) + SSSD (Reaper)
+│   ├── 08-mix-container-runtime-engines/  # Mixed runtimes: OpenLDAP (default) + SSSD (Reaper)
+│   └── 09-reaperpod/                     # ReaperPod CRD: simplified Reaper-native workloads
 ├── scripts/
-│   ├── run-integration-tests.sh # Full integration test suite
-│   └── install-reaper.sh        # Installation script (Ansible wrapper)
+│   ├── run-integration-tests.sh   # Full integration test suite
+│   ├── install-reaper.sh          # Installation script (Ansible, DEPRECATED)
+│   ├── build-node-image.sh        # Build node installer image for Kind
+│   ├── build-controller-image.sh  # Build controller Docker image for Kind
+│   ├── install-node.sh            # Init container script for node DaemonSet
+│   └── generate-crds.sh           # Generate CRD YAML from Rust types
 ├── deploy/
-│   ├── ansible/
-│   │   └── install-reaper.yml   # Deployment playbook
+│   ├── helm/reaper/               # Helm chart (recommended installation)
+│   │   ├── Chart.yaml
+│   │   ├── values.yaml
+│   │   ├── crds/                  # CRD definitions
+│   │   └── templates/             # DaemonSet, Controller, RBAC, RuntimeClass
+│   ├── ansible/                   # DEPRECATED — use Helm chart instead
+│   │   └── install-reaper.yml
 │   └── kubernetes/
-│       └── runtimeclass.yaml    # RuntimeClass definition
+│       ├── runtimeclass.yaml
+│       ├── reaper-controller.yaml
+│       └── crds/
+│           └── reaperpods.reaper.io.yaml
 └── docs/
     ├── SHIMV2_DESIGN.md         # Shim v2 protocol implementation
     ├── SHIM_ARCHITECTURE.md     # Architecture deep-dive
@@ -259,12 +277,13 @@ reaper/
 
 Key environment variables:
 - `CI`: Set by GitHub Actions automatically. Enables CI-specific behavior.
-- `REAPER_BINARY_DIR`: Override the binary directory location for Ansible installer.
+- `REAPER_BINARY_DIR`: Override the binary directory location (legacy Ansible installer).
 
 Files involved:
-- [scripts/run-integration-tests.sh](scripts/run-integration-tests.sh): Detects CI mode and sets `REAPER_BINARY_DIR`
-- [scripts/install-reaper.sh](scripts/install-reaper.sh): Accepts `REAPER_BINARY_DIR` and passes it to Ansible
-- [deploy/ansible/install-reaper.yml](deploy/ansible/install-reaper.yml): Uses `local_binary_dir` variable (set from `REAPER_BINARY_DIR`)
+- [scripts/setup-playground.sh](scripts/setup-playground.sh): Creates Kind cluster and installs via Helm
+- [scripts/build-node-image.sh](scripts/build-node-image.sh): Builds reaper-node installer image for Kind
+- [scripts/build-controller-image.sh](scripts/build-controller-image.sh): Builds reaper-controller image for Kind
+- [deploy/helm/reaper/](deploy/helm/reaper/): Helm chart (DaemonSet, Controller, CRD, RuntimeClass)
 
 ### Building Binaries for Integration Tests
 
@@ -287,14 +306,15 @@ See [MEMORY.md](.claude/projects/-Users-miguelgi-Documents-CODE-Explorations-rea
 
 ## Integration Test Structure
 
-The integration test suite ([scripts/run-integration-tests.sh](scripts/run-integration-tests.sh)) has four phases:
+The integration test suite ([scripts/run-integration-tests.sh](scripts/run-integration-tests.sh)) has five phases:
 
 1. **Phase 1**: Rust cargo tests (unit and integration tests)
-2. **Phase 2**: Infrastructure setup (Kind cluster, build binaries, install Reaper via Ansible)
+2. **Phase 2**: Infrastructure setup (Kind cluster, build images, install Reaper via Helm)
 3. **Phase 3**: Kubernetes readiness checks (API server, RuntimeClass, ServiceAccount)
 4. **Phase 4**: Integration tests (DNS, overlay, process cleanup, exec support, etc.)
+5. **Phase 4b**: Controller tests (ReaperPod CRD lifecycle, status mirroring, exit codes, annotations, GC)
 
-All tests must pass for the suite to succeed.
+Use `--crd-only` or `--agent-only` to run subsets. All tests must pass for the suite to succeed.
 
 ## Development Workflow
 

Cargo.lock

@@ -31,8 +31,12 @@ axum = { version = "0.8", optional = true }
 futures = { version = "0.3", optional = true }
 chrono = { version = "0.4", optional = true }
 
+# reaper-controller dependencies
+schemars = { version = "0.8", optional = true }
+
 [features]
 agent = ["kube", "k8s-openapi", "prometheus-client", "axum", "futures", "chrono"]
+controller = ["kube", "k8s-openapi", "schemars", "futures", "chrono"]
 
 [dev-dependencies]
 tempfile = "3"
@@ -51,6 +55,11 @@ name = "reaper-agent"
 path = "src/bin/reaper-agent/main.rs"
 required-features = ["agent"]
 
+[[bin]]
+name = "reaper-controller"
+path = "src/bin/reaper-controller/main.rs"
+required-features = ["controller"]
+
 [lints.rust]
 unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tarpaulin_include)'] }
 

Cargo.toml

@@ -0,0 +1,25 @@
+## Build stage: cross-compile static musl binary
+ARG RUST_TARGET=x86_64-unknown-linux-musl
+ARG MUSL_IMAGE=x86_64-musl
+
+FROM messense/rust-musl-cross:${MUSL_IMAGE} AS builder
+ARG RUST_TARGET
+
+WORKDIR /work
+COPY . .
+
+# Set build metadata
+RUN export GIT_HASH=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown") && \
+    export BUILD_DATE=$(date -u '+%Y-%m-%d') && \
+    cargo build --release --features controller --bin reaper-controller --target "${RUST_TARGET}"
+
+RUN cp target/${RUST_TARGET}/release/reaper-controller /reaper-controller
+
+## Runtime stage: distroless static (no shell, no libc needed for musl)
+FROM gcr.io/distroless/static-debian12:nonroot
+
+COPY --from=builder /reaper-controller /usr/local/bin/reaper-controller
+
+USER nonroot:nonroot
+
+ENTRYPOINT ["/usr/local/bin/reaper-controller"]

Dockerfile.controller

@@ -0,0 +1,38 @@
+# Multi-stage build for reaper-node installer image.
+# Contains containerd-shim-reaper-v2, reaper-runtime, and an install script.
+# Used as an init container in the Helm chart's DaemonSet to install
+# Reaper binaries onto cluster nodes.
+
+# --- Builder stage (amd64) ---
+FROM messense/rust-musl-cross:x86_64-musl AS builder-amd64
+WORKDIR /work
+COPY . .
+RUN cargo build --release \
+  --bin containerd-shim-reaper-v2 \
+  --bin reaper-runtime \
+  --target x86_64-unknown-linux-musl
+
+# --- Builder stage (arm64) ---
+FROM messense/rust-musl-cross:aarch64-musl AS builder-arm64
+WORKDIR /work
+COPY . .
+RUN cargo build --release \
+  --bin containerd-shim-reaper-v2 \
+  --bin reaper-runtime \
+  --target aarch64-unknown-linux-musl
+
+# --- Runtime stage ---
+FROM alpine:3.19
+
+ARG TARGETARCH=amd64
+
+# Copy both architectures, select based on TARGETARCH
+COPY --from=builder-amd64 /work/target/x86_64-unknown-linux-musl/release/containerd-shim-reaper-v2 /binaries/amd64/containerd-shim-reaper-v2
+COPY --from=builder-amd64 /work/target/x86_64-unknown-linux-musl/release/reaper-runtime /binaries/amd64/reaper-runtime
+COPY --from=builder-arm64 /work/target/aarch64-unknown-linux-musl/release/containerd-shim-reaper-v2 /binaries/arm64/containerd-shim-reaper-v2
+COPY --from=builder-arm64 /work/target/aarch64-unknown-linux-musl/release/reaper-runtime /binaries/arm64/reaper-runtime
+
+COPY scripts/install-node.sh /install.sh
+RUN chmod +x /install.sh
+
+ENTRYPOINT ["/install.sh"]

Dockerfile.node

@@ -39,23 +39,15 @@ Reaper is a containerd shim that runs processes directly on the host system whil
 
 ### Playground (try it locally)
 
-Spin up a 3-node Kind cluster with Reaper pre-installed. No Rust toolchain needed.
-
-**Quickest way** — download pre-built binaries from the latest [GitHub Release](https://github.com/miguelgila/reaper/releases):
-
-```bash
-# Prerequisites: Docker, kind, kubectl, ansible (pip install ansible)
-./scripts/setup-playground.sh --release
-```
-
-**Build from source** — compiles inside Docker (still no local Rust needed):
+Spin up a 3-node Kind cluster with Reaper pre-installed. Compiles inside Docker — no local Rust toolchain needed.
 
 ```bash
+# Prerequisites: Docker, kind, kubectl, helm
 ./scripts/setup-playground.sh
 ```
 
-Both create a Kind cluster with 1 control-plane + 2 worker nodes, install the
-runtime on all nodes, and run a smoke test. Once ready, try:
+This creates a Kind cluster with 1 control-plane + 2 worker nodes, installs
+Reaper via Helm (node DaemonSet + controller + CRDs + RuntimeClass), and runs a smoke test. Once ready, try:
 
 ```bash
 kubectl run hello --rm -it --image=busybox --restart=Never \
@@ -91,25 +83,16 @@ docker run --rm -v "$(pwd)":/work -w /work \
 
 ### 1. Install Reaper on a Kubernetes Cluster
 
-**For Kind clusters (testing/CI):**
+**Via Helm (recommended):**
 ```bash
-# Install Ansible if not already installed
-pip install ansible  # or: brew install ansible
-
-# Install to Kind cluster
-./scripts/install-reaper.sh --kind <cluster-name>
+helm upgrade --install reaper deploy/helm/reaper/ \
+  --namespace reaper-system --create-namespace \
+  --wait --timeout 120s
 ```
 
-**For production clusters:**
-```bash
-# Create inventory file (see deploy/ansible/inventory.ini.example)
-vim inventory.ini
-
-# Install via Ansible
-ansible-playbook -i inventory.ini deploy/ansible/install-reaper.yml
-```
+This installs the node DaemonSet (binary installer), CRD, controller, RuntimeClass, and RBAC — everything needed to run Reaper workloads.
 
-See [deploy/kubernetes/README.md](deploy/kubernetes/README.md) for detailed installation instructions.
+See [deploy/helm/reaper/](deploy/helm/reaper/) for chart values and configuration.
 
 ### 2. Run a Command on the Host
 
@@ -284,7 +267,7 @@ The [examples/](examples/) directory contains runnable demos, each with a `setup
 - [Docker](https://docs.docker.com/get-docker/)
 - [kind](https://kind.sigs.k8s.io/) (Kubernetes in Docker)
 - [kubectl](https://kubernetes.io/docs/tasks/tools/)
-- [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/) (`pip install ansible`)
+- [Helm](https://helm.sh/docs/intro/install/)
 
 **Local development (building from source natively):**
 - All of the above, plus [Rust](https://www.rust-lang.org/tools/install) (toolchain version pinned in `rust-toolchain.toml`)
@@ -305,7 +288,7 @@ cargo test
 
 ## Configuration
 
-Reaper reads configuration from `/etc/reaper/reaper.conf` on each node. The Ansible installer creates this file automatically. Environment variables of the same name override file values.
+Reaper reads configuration from `/etc/reaper/reaper.conf` on each node. The Helm chart creates this file automatically via the node DaemonSet init container. Environment variables of the same name override file values.
 
 ```ini
 # /etc/reaper/reaper.conf

README.md

@@ -1,5 +1,11 @@
 # Ansible Playbooks for Reaper Runtime
 
+> **DEPRECATED**: Ansible-based installation is deprecated in favor of the **Helm chart** (`deploy/helm/reaper/`).
+> The Helm chart handles binary installation via DaemonSet init containers, CRD installation, controller deployment,
+> and RuntimeClass creation — all in a single `helm install`. See the [Helm chart README](../helm/reaper/README.md).
+>
+> These playbooks remain available for legacy workflows but are no longer actively tested in CI.
+
 This directory contains Ansible playbooks for deploying and managing Reaper runtime on Kubernetes cluster nodes.
 
 ## Overview

deploy/ansible/README.md

@@ -0,0 +1,16 @@
+apiVersion: v2
+name: reaper
+description: Reaper — lightweight Kubernetes runtime for direct host execution
+type: application
+version: 0.1.0
+appVersion: "0.2.10"
+keywords:
+  - kubernetes
+  - runtime
+  - containerd
+  - host-execution
+home: https://github.com/miguelgila/reaper
+sources:
+  - https://github.com/miguelgila/reaper
+maintainers:
+  - name: Miguel Gil Álvarez