Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
71d3ebe
feat(controller): add ReaperPod CRD and reaper-controller binary
miguelgila Mar 6, 2026
de1fa90
test(controller): add Kind integration tests for ReaperPod CRD
miguelgila Mar 6, 2026
295e774
docs: update CRD plan — all 10 steps complete
miguelgila Mar 6, 2026
601fa73
feat(test): add --crd-only flag for fast CRD controller test iteration
miguelgila Mar 6, 2026
d6b77fd
fix(test): ensure reaper-system namespace exists for --crd-only runs
miguelgila Mar 6, 2026
dd630ee
docs: update examples and docs for CRD controller tests
miguelgila Mar 6, 2026
73440bc
feat(playground): add --with-controller flag for ReaperPod CRD setup
miguelgila Mar 7, 2026
447a057
feat(playground): make CRD and controller setup the default
miguelgila Mar 7, 2026
58dc052
fix(playground): add --context to all kubectl examples in summary
miguelgila Mar 7, 2026
899d526
fix(controller): use fixed Pod name matching ReaperPod name
miguelgila Mar 7, 2026
da48f23
feat(helm): replace Ansible with Helm chart for installation
miguelgila Mar 7, 2026
63f8cc7
docs: update all references from Ansible to Helm
miguelgila Mar 7, 2026
824bfe1
ci: remove playground-release job (--release flag removed)
miguelgila Mar 7, 2026
b4c0316
ci: build reaper-controller in CI, replace Ansible with Helm
miguelgila Mar 7, 2026
00f5ecd
fix(test): replace yield_now with 50ms sleep to avoid ETXTBSY flake
miguelgila Mar 7, 2026
18a4932
ci: increase kind-integration timeout from 20 to 30 minutes
miguelgila Mar 7, 2026
b530a39
docs: add CI test runtime optimization to TODO
miguelgila Mar 7, 2026
40f3023
ci: retrigger CI pipeline
miguelgila Mar 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,10 +95,11 @@ jobs:
-v ${{ github.workspace }}:/work \
-w /work \
messense/rust-musl-cross:x86_64-musl \
cargo build --release --features agent \
cargo build --release --features agent,controller \
--bin containerd-shim-reaper-v2 \
--bin reaper-runtime \
--bin reaper-agent \
--bin reaper-controller \
--target x86_64-unknown-linux-musl
- uses: actions/upload-artifact@v4
with:
Expand All @@ -107,6 +108,7 @@ jobs:
target/x86_64-unknown-linux-musl/release/containerd-shim-reaper-v2
target/x86_64-unknown-linux-musl/release/reaper-runtime
target/x86_64-unknown-linux-musl/release/reaper-agent
target/x86_64-unknown-linux-musl/release/reaper-controller
retention-days: 1

kind-integration:
Expand Down Expand Up @@ -136,13 +138,15 @@ jobs:
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind

- name: Install Ansible
run: pip install ansible
- name: Install Helm
uses: azure/setup-helm@v4
with:
version: "v3.14.0"

- name: Run integration tests
uses: nick-fields/retry@v3
with:
timeout_minutes: 20
timeout_minutes: 30
max_attempts: 2
command: ./scripts/run-integration-tests.sh --skip-cargo

Expand All @@ -155,10 +159,6 @@ jobs:
path: /tmp/reaper-integration-logs/
retention-days: 7

playground-release:
name: Playground Release Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Test --release resolution modes
run: ./scripts/test-playground-release.sh
# playground-release job removed: setup-playground.sh now uses Helm
# instead of --release flag. The release-utils.sh library is retained
# for the deprecated install-reaper.sh script.
54 changes: 37 additions & 17 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,13 +190,18 @@ reaper/
├── src/
│ ├── config.rs # Shared config file loader (/etc/reaper/reaper.conf)
│ ├── annotations.rs # Shared pod annotation parsing and validation
│ ├── crds/ # ReaperPod CRD types (feature-gated: controller)
│ └── bin/
│ ├── containerd-shim-reaper-v2/
│ │ └── main.rs # Shim implementation (ttrpc server, Task trait)
│ └── reaper-runtime/
│ ├── main.rs # OCI runtime CLI (fork-first architecture)
│ ├── state.rs # State persistence (/run/reaper/<id>/)
│ └── overlay.rs # Overlay filesystem (Linux-only)
│ ├── reaper-controller/ # CRD controller binary
│ │ ├── main.rs # Entry point, CRD watcher setup
│ │ ├── reconciler.rs # ReaperPod → Pod reconciliation
│ │ └── pod_builder.rs # ReaperPod spec → Pod spec translation
│ └── reaper-runtime/
│ ├── main.rs # OCI runtime CLI (fork-first architecture)
│ ├── state.rs # State persistence (/run/reaper/<id>/)
│ └── overlay.rs # Overlay filesystem (Linux-only)
├── tests/ # Integration tests
│ ├── integration_basic_binary.rs
│ ├── integration_io.rs # FIFO stdout/stderr
Expand All @@ -213,15 +218,28 @@ reaper/
│ ├── 05-kubemix/ # Jobs, DaemonSets, and Deployments on 10-node cluster
│ ├── 06-ansible-jobs/ # Sequential Jobs: install Ansible, then run playbook
│ ├── 07-ansible-complex/ # DaemonSet bootstrap + role-based Ansible playbooks
│ └── 08-mix-container-runtime-engines/ # Mixed runtimes: OpenLDAP (default) + SSSD (Reaper)
│ ├── 08-mix-container-runtime-engines/ # Mixed runtimes: OpenLDAP (default) + SSSD (Reaper)
│ └── 09-reaperpod/ # ReaperPod CRD: simplified Reaper-native workloads
├── scripts/
│ ├── run-integration-tests.sh # Full integration test suite
│ └── install-reaper.sh # Installation script (Ansible wrapper)
│ ├── run-integration-tests.sh # Full integration test suite
│ ├── install-reaper.sh # Installation script (Ansible, DEPRECATED)
│ ├── build-node-image.sh # Build node installer image for Kind
│ ├── build-controller-image.sh # Build controller Docker image for Kind
│ ├── install-node.sh # Init container script for node DaemonSet
│ └── generate-crds.sh # Generate CRD YAML from Rust types
├── deploy/
│ ├── ansible/
│ │ └── install-reaper.yml # Deployment playbook
│ ├── helm/reaper/ # Helm chart (recommended installation)
│ │ ├── Chart.yaml
│ │ ├── values.yaml
│ │ ├── crds/ # CRD definitions
│ │ └── templates/ # DaemonSet, Controller, RBAC, RuntimeClass
│ ├── ansible/ # DEPRECATED — use Helm chart instead
│ │ └── install-reaper.yml
│ └── kubernetes/
│ └── runtimeclass.yaml # RuntimeClass definition
│ ├── runtimeclass.yaml
│ ├── reaper-controller.yaml
│ └── crds/
│ └── reaperpods.reaper.io.yaml
└── docs/
├── SHIMV2_DESIGN.md # Shim v2 protocol implementation
├── SHIM_ARCHITECTURE.md # Architecture deep-dive
Expand Down Expand Up @@ -259,12 +277,13 @@ reaper/

Key environment variables:
- `CI`: Set by GitHub Actions automatically. Enables CI-specific behavior.
- `REAPER_BINARY_DIR`: Override the binary directory location for Ansible installer.
- `REAPER_BINARY_DIR`: Override the binary directory location (legacy Ansible installer).

Files involved:
- [scripts/run-integration-tests.sh](scripts/run-integration-tests.sh): Detects CI mode and sets `REAPER_BINARY_DIR`
- [scripts/install-reaper.sh](scripts/install-reaper.sh): Accepts `REAPER_BINARY_DIR` and passes it to Ansible
- [deploy/ansible/install-reaper.yml](deploy/ansible/install-reaper.yml): Uses `local_binary_dir` variable (set from `REAPER_BINARY_DIR`)
- [scripts/setup-playground.sh](scripts/setup-playground.sh): Creates Kind cluster and installs via Helm
- [scripts/build-node-image.sh](scripts/build-node-image.sh): Builds reaper-node installer image for Kind
- [scripts/build-controller-image.sh](scripts/build-controller-image.sh): Builds reaper-controller image for Kind
- [deploy/helm/reaper/](deploy/helm/reaper/): Helm chart (DaemonSet, Controller, CRD, RuntimeClass)

### Building Binaries for Integration Tests

Expand All @@ -287,14 +306,15 @@ See [MEMORY.md](.claude/projects/-Users-miguelgi-Documents-CODE-Explorations-rea

## Integration Test Structure

The integration test suite ([scripts/run-integration-tests.sh](scripts/run-integration-tests.sh)) has four phases:
The integration test suite ([scripts/run-integration-tests.sh](scripts/run-integration-tests.sh)) has five phases:

1. **Phase 1**: Rust cargo tests (unit and integration tests)
2. **Phase 2**: Infrastructure setup (Kind cluster, build binaries, install Reaper via Ansible)
2. **Phase 2**: Infrastructure setup (Kind cluster, build images, install Reaper via Helm)
3. **Phase 3**: Kubernetes readiness checks (API server, RuntimeClass, ServiceAccount)
4. **Phase 4**: Integration tests (DNS, overlay, process cleanup, exec support, etc.)
5. **Phase 4b**: Controller tests (ReaperPod CRD lifecycle, status mirroring, exit codes, annotations, GC)

All tests must pass for the suite to succeed.
Use `--crd-only` or `--agent-only` to run subsets. All tests must pass for the suite to succeed.

## Development Workflow

Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,12 @@ axum = { version = "0.8", optional = true }
futures = { version = "0.3", optional = true }
chrono = { version = "0.4", optional = true }

# reaper-controller dependencies
schemars = { version = "0.8", optional = true }

[features]
agent = ["kube", "k8s-openapi", "prometheus-client", "axum", "futures", "chrono"]
controller = ["kube", "k8s-openapi", "schemars", "futures", "chrono"]

[dev-dependencies]
tempfile = "3"
Expand All @@ -51,6 +55,11 @@ name = "reaper-agent"
path = "src/bin/reaper-agent/main.rs"
required-features = ["agent"]

[[bin]]
name = "reaper-controller"
path = "src/bin/reaper-controller/main.rs"
required-features = ["controller"]

[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tarpaulin_include)'] }

Expand Down
25 changes: 25 additions & 0 deletions Dockerfile.controller
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
## Build stage: cross-compile static musl binary
ARG RUST_TARGET=x86_64-unknown-linux-musl
ARG MUSL_IMAGE=x86_64-musl

FROM messense/rust-musl-cross:${MUSL_IMAGE} AS builder
ARG RUST_TARGET

WORKDIR /work
COPY . .

# Set build metadata
RUN export GIT_HASH=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown") && \
export BUILD_DATE=$(date -u '+%Y-%m-%d') && \
cargo build --release --features controller --bin reaper-controller --target "${RUST_TARGET}"

RUN cp target/${RUST_TARGET}/release/reaper-controller /reaper-controller

## Runtime stage: distroless static (no shell, no libc needed for musl)
FROM gcr.io/distroless/static-debian12:nonroot

COPY --from=builder /reaper-controller /usr/local/bin/reaper-controller

USER nonroot:nonroot

ENTRYPOINT ["/usr/local/bin/reaper-controller"]
38 changes: 38 additions & 0 deletions Dockerfile.node
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Multi-stage build for reaper-node installer image.
# Contains containerd-shim-reaper-v2, reaper-runtime, and an install script.
# Used as an init container in the Helm chart's DaemonSet to install
# Reaper binaries onto cluster nodes.

# --- Builder stage (amd64) ---
FROM messense/rust-musl-cross:x86_64-musl AS builder-amd64
WORKDIR /work
COPY . .
RUN cargo build --release \
--bin containerd-shim-reaper-v2 \
--bin reaper-runtime \
--target x86_64-unknown-linux-musl

# --- Builder stage (arm64) ---
FROM messense/rust-musl-cross:aarch64-musl AS builder-arm64
WORKDIR /work
COPY . .
RUN cargo build --release \
--bin containerd-shim-reaper-v2 \
--bin reaper-runtime \
--target aarch64-unknown-linux-musl

# --- Runtime stage ---
FROM alpine:3.19

ARG TARGETARCH=amd64

# Copy both architectures, select based on TARGETARCH
COPY --from=builder-amd64 /work/target/x86_64-unknown-linux-musl/release/containerd-shim-reaper-v2 /binaries/amd64/containerd-shim-reaper-v2
COPY --from=builder-amd64 /work/target/x86_64-unknown-linux-musl/release/reaper-runtime /binaries/amd64/reaper-runtime
COPY --from=builder-arm64 /work/target/aarch64-unknown-linux-musl/release/containerd-shim-reaper-v2 /binaries/arm64/containerd-shim-reaper-v2
COPY --from=builder-arm64 /work/target/aarch64-unknown-linux-musl/release/reaper-runtime /binaries/arm64/reaper-runtime

COPY scripts/install-node.sh /install.sh
RUN chmod +x /install.sh

ENTRYPOINT ["/install.sh"]
41 changes: 12 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,23 +39,15 @@ Reaper is a containerd shim that runs processes directly on the host system whil

### Playground (try it locally)

Spin up a 3-node Kind cluster with Reaper pre-installed. No Rust toolchain needed.

**Quickest way** — download pre-built binaries from the latest [GitHub Release](https://github.com/miguelgila/reaper/releases):

```bash
# Prerequisites: Docker, kind, kubectl, ansible (pip install ansible)
./scripts/setup-playground.sh --release
```

**Build from source** — compiles inside Docker (still no local Rust needed):
Spin up a 3-node Kind cluster with Reaper pre-installed. Compiles inside Docker — no local Rust toolchain needed.

```bash
# Prerequisites: Docker, kind, kubectl, helm
./scripts/setup-playground.sh
```

Both create a Kind cluster with 1 control-plane + 2 worker nodes, install the
runtime on all nodes, and run a smoke test. Once ready, try:
This creates a Kind cluster with 1 control-plane + 2 worker nodes, installs
Reaper via Helm (node DaemonSet + controller + CRDs + RuntimeClass), and runs a smoke test. Once ready, try:

```bash
kubectl run hello --rm -it --image=busybox --restart=Never \
Expand Down Expand Up @@ -91,25 +83,16 @@ docker run --rm -v "$(pwd)":/work -w /work \

### 1. Install Reaper on a Kubernetes Cluster

**For Kind clusters (testing/CI):**
**Via Helm (recommended):**
```bash
# Install Ansible if not already installed
pip install ansible # or: brew install ansible

# Install to Kind cluster
./scripts/install-reaper.sh --kind <cluster-name>
helm upgrade --install reaper deploy/helm/reaper/ \
--namespace reaper-system --create-namespace \
--wait --timeout 120s
```

**For production clusters:**
```bash
# Create inventory file (see deploy/ansible/inventory.ini.example)
vim inventory.ini

# Install via Ansible
ansible-playbook -i inventory.ini deploy/ansible/install-reaper.yml
```
This installs the node DaemonSet (binary installer), CRD, controller, RuntimeClass, and RBAC — everything needed to run Reaper workloads.

See [deploy/kubernetes/README.md](deploy/kubernetes/README.md) for detailed installation instructions.
See [deploy/helm/reaper/](deploy/helm/reaper/) for chart values and configuration.

### 2. Run a Command on the Host

Expand Down Expand Up @@ -284,7 +267,7 @@ The [examples/](examples/) directory contains runnable demos, each with a `setup
- [Docker](https://docs.docker.com/get-docker/)
- [kind](https://kind.sigs.k8s.io/) (Kubernetes in Docker)
- [kubectl](https://kubernetes.io/docs/tasks/tools/)
- [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/) (`pip install ansible`)
- [Helm](https://helm.sh/docs/intro/install/)

**Local development (building from source natively):**
- All of the above, plus [Rust](https://www.rust-lang.org/tools/install) (toolchain version pinned in `rust-toolchain.toml`)
Expand All @@ -305,7 +288,7 @@ cargo test

## Configuration

Reaper reads configuration from `/etc/reaper/reaper.conf` on each node. The Ansible installer creates this file automatically. Environment variables of the same name override file values.
Reaper reads configuration from `/etc/reaper/reaper.conf` on each node. The Helm chart creates this file automatically via the node DaemonSet init container. Environment variables of the same name override file values.

```ini
# /etc/reaper/reaper.conf
Expand Down
6 changes: 6 additions & 0 deletions deploy/ansible/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Ansible Playbooks for Reaper Runtime

> **DEPRECATED**: Ansible-based installation is deprecated in favor of the **Helm chart** (`deploy/helm/reaper/`).
> The Helm chart handles binary installation via DaemonSet init containers, CRD installation, controller deployment,
> and RuntimeClass creation — all in a single `helm install`. See the [Helm chart README](../helm/reaper/README.md).
>
> These playbooks remain available for legacy workflows but are no longer actively tested in CI.

This directory contains Ansible playbooks for deploying and managing Reaper runtime on Kubernetes cluster nodes.

## Overview
Expand Down
16 changes: 16 additions & 0 deletions deploy/helm/reaper/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
apiVersion: v2
name: reaper
description: Reaper — lightweight Kubernetes runtime for direct host execution
type: application
version: 0.1.0
appVersion: "0.2.10"
keywords:
- kubernetes
- runtime
- containerd
- host-execution
home: https://github.com/miguelgila/reaper
sources:
- https://github.com/miguelgila/reaper
maintainers:
- name: Miguel Gil Álvarez
Loading
Loading