Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ target-linux/
.vscode/
.ignored/
.claude/
.omc/
cobertura.xml
.ignore/

Expand Down
11 changes: 8 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,9 @@ Host Root (/) ─── read-only lower layer
- `/tmp` is NOT bind-mounted (protected by overlay)

**Configuration:**
- Config file: `/etc/reaper/reaper.conf` (cross-distro, `KEY=VALUE` format)
- Config load order: config file defaults → env vars override
- `REAPER_CONFIG`: Override config file path (default: `/etc/reaper/reaper.conf`)
- `REAPER_OVERLAY_BASE`: Default `/run/reaper/overlay`
- `REAPER_DNS_MODE`: DNS resolution mode — `host` (default) uses node's resolv.conf, `kubernetes`/`k8s` writes kubelet-prepared resolv.conf (pointing to CoreDNS) into the overlay
- Overlay is mandatory on Linux (no fail-open)
Expand All @@ -109,9 +112,11 @@ Kubernetes volumes (ConfigMap, Secret, hostPath, emptyDir, etc.) are supported v

```
reaper/
├── src/bin/
│ ├── containerd-shim-reaper-v2/
│ │ └── main.rs # Shim implementation (ttrpc server, Task trait)
├── src/
│ ├── config.rs # Shared config file loader (/etc/reaper/reaper.conf)
│ └── bin/
│ ├── containerd-shim-reaper-v2/
│ │ └── main.rs # Shim implementation (ttrpc server, Task trait)
│ └── reaper-runtime/
│ ├── main.rs # OCI runtime CLI (fork-first architecture)
│ ├── state.rs # State persistence (/run/reaper/<id>/)
Expand Down
24 changes: 14 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,26 +300,30 @@ cargo test

## Configuration

### Overlay Filesystem

All Reaper workloads share a single overlay filesystem:
Reaper reads configuration from `/etc/reaper/reaper.conf` on each node. The Ansible installer creates this file automatically. Environment variables of the same name override file values.

```bash
# Custom overlay location (default: /run/reaper/overlay)
export REAPER_OVERLAY_BASE=/custom/path
```ini
# /etc/reaper/reaper.conf
REAPER_DNS_MODE=kubernetes
REAPER_RUNTIME_LOG=/run/reaper/runtime.log
REAPER_OVERLAY_BASE=/run/reaper/overlay
```

The host root is the read-only lower layer; writes go to a shared upper layer. This means workloads can share files with each other while protecting the host from modifications.
See [deploy/kubernetes/README.md](deploy/kubernetes/README.md#configuration) for the full list of settings.

### Overlay Filesystem

All Reaper workloads share a single overlay filesystem. The host root is the read-only lower layer; writes go to a shared upper layer. This means workloads can share files with each other while protecting the host from modifications.

For details, see [docs/OVERLAY_DESIGN.md](docs/OVERLAY_DESIGN.md).

### Runtime Logging

Enable runtime logging for debugging:
Enable runtime logging for debugging (in `/etc/reaper/reaper.conf` or via env vars):

```bash
export REAPER_SHIM_LOG=/var/log/reaper-shim.log
export REAPER_RUNTIME_LOG=/var/log/reaper-runtime.log
REAPER_SHIM_LOG=/var/log/reaper-shim.log
REAPER_RUNTIME_LOG=/var/log/reaper-runtime.log
```

## Known Issues
Expand Down
47 changes: 25 additions & 22 deletions deploy/ansible/install-reaper.yml
Original file line number Diff line number Diff line change
Expand Up @@ -144,31 +144,34 @@
executable: /bin/bash
when: reaper_configured.rc != 0

- name: Set Reaper runtime log environment for containerd
lineinfile:
path: /etc/default/containerd
line: "REAPER_RUNTIME_LOG=/run/reaper/runtime.log"
create: yes
mode: '0644'
- name: Create Reaper configuration directory
file:
path: /etc/reaper
state: directory
mode: '0755'

- name: Write Reaper configuration file
copy:
dest: /etc/reaper/reaper.conf
content: |
# Reaper runtime configuration
# Environment variables override values in this file.
# See: https://github.com/miguelgila/reaper

- name: Set Reaper DNS mode for containerd
lineinfile:
path: /etc/default/containerd
line: "REAPER_DNS_MODE={{ reaper_dns_mode | default('kubernetes') }}"
regexp: "^REAPER_DNS_MODE="
create: yes
REAPER_DNS_MODE={{ reaper_dns_mode | default('kubernetes') }}
REAPER_RUNTIME_LOG=/run/reaper/runtime.log
mode: '0644'

- name: Ensure containerd service picks up environment file
shell: |
mkdir -p /etc/systemd/system/containerd.service.d
cat > /etc/systemd/system/containerd.service.d/reaper-env.conf << 'EOF'
[Service]
EnvironmentFile=-/etc/default/containerd
EOF
systemctl daemon-reload
args:
executable: /bin/bash
- name: Remove legacy environment file drop-in (if present)
file:
path: /etc/systemd/system/containerd.service.d/reaper-env.conf
state: absent
register: legacy_dropin_removed

- name: Reload systemd if legacy drop-in was removed
systemd:
daemon_reload: yes
when: legacy_dropin_removed.changed

- name: Validate containerd configuration
command: containerd config dump
Expand Down
31 changes: 31 additions & 0 deletions deploy/kubernetes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,37 @@ kubectl logs reaper-example

Expected output: `Hello from Reaper runtime!`

## Configuration

Reaper reads configuration from `/etc/reaper/reaper.conf` on each node. The Ansible installer creates this file automatically. Environment variables of the same name override file values.

```ini
# /etc/reaper/reaper.conf — written by Ansible installer
REAPER_DNS_MODE=kubernetes
REAPER_RUNTIME_LOG=/run/reaper/runtime.log
```

Common settings:

| Variable | Default | Description |
|----------|---------|-------------|
| `REAPER_DNS_MODE` | `host` | DNS mode: `host` (node resolv.conf) or `kubernetes` (CoreDNS) |
| `REAPER_RUNTIME_LOG` | (none) | Runtime log file path |
| `REAPER_SHIM_LOG` | (none) | Shim log file path |
| `REAPER_OVERLAY_BASE` | `/run/reaper/overlay` | Overlay filesystem base directory |
| `REAPER_FILTER_ENABLED` | `true` | Enable sensitive host file filtering |

For manual installations, create the file:

```bash
sudo mkdir -p /etc/reaper
sudo tee /etc/reaper/reaper.conf << 'EOF'
REAPER_DNS_MODE=kubernetes
REAPER_RUNTIME_LOG=/run/reaper/runtime.log
EOF
sudo systemctl restart containerd
```

## Integration Testing

See [TESTING.md](../../docs/TESTING.md) for:
Expand Down
61 changes: 0 additions & 61 deletions docs/INTEGRATION_TEST_PLAN.md

This file was deleted.

127 changes: 127 additions & 0 deletions docs/PLAN-config-file.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Plan: Cross-OS Configuration File

**Goal**: Replace the Debian-specific `/etc/default/containerd` env-file approach with a Reaper-owned config file at `/etc/reaper/reaper.conf` that works on any Linux distribution.

---

## Problem

Configuration is currently injected via environment variables into containerd's process using:
1. `/etc/default/containerd` — a Debian/Ubuntu convention
2. A systemd drop-in (`EnvironmentFile=-/etc/default/containerd`)

This doesn't work on RHEL/Fedora/SUSE/Arch/Alpine where `/etc/default/` doesn't exist or isn't the convention. The systemd drop-in is also unnecessary coupling — Reaper can read its own config.

## Current REAPER_* Environment Variables

Both binaries read configuration purely from `std::env::var()`:

| Variable | Binary | Default | Purpose |
|----------|--------|---------|---------|
| `REAPER_RUNTIME_ROOT` | both | `/run/reaper` | State directory root |
| `REAPER_RUNTIME_PATH` | shim | auto-detect | Path to runtime binary |
| `REAPER_RUNTIME_LOG` | runtime | (none) | Runtime log file path |
| `REAPER_SHIM_LOG` | shim | (none) | Shim log file path |
| `REAPER_OVERLAY_BASE` | runtime | `/run/reaper/overlay` | Overlay base directory |
| `REAPER_OVERLAY_NS` | runtime | `/run/reaper/shared-mnt-ns` | Namespace bind-mount path |
| `REAPER_OVERLAY_LOCK` | runtime | `/run/reaper/overlay.lock` | Overlay lock file |
| `REAPER_DNS_MODE` | runtime | `host` | DNS mode: host, kubernetes, k8s |
| `REAPER_FILTER_ENABLED` | runtime | `true` | Enable sensitive file filtering |
| `REAPER_FILTER_MODE` | runtime | `append` | Filter mode: append or replace |
| `REAPER_FILTER_PATHS` | runtime | (none) | Colon-separated extra filter paths |
| `REAPER_FILTER_ALLOWLIST` | runtime | (none) | Colon-separated allowlist paths |
| `REAPER_FILTER_DIR` | runtime | `/run/reaper/overlay-filters` | Placeholder file directory |

## Solution

### Config file: `/etc/reaper/reaper.conf`

Simple `KEY=VALUE` format (same as existing env vars):

```
# Reaper runtime configuration
# Lines starting with # are comments. Blank lines are ignored.
# Values here are overridden by environment variables of the same name.

REAPER_DNS_MODE=kubernetes
REAPER_RUNTIME_LOG=/run/reaper/runtime.log
# REAPER_OVERLAY_BASE=/run/reaper/overlay
# REAPER_FILTER_ENABLED=true
```

### Load order (last wins)

1. Built-in defaults (in Rust code, unchanged)
2. Config file values (set as env vars if not already set)
3. Environment variables (override everything)

This means env vars always win — backward compatible with anyone already using systemd env injection.

### Config file search

1. `REAPER_CONFIG` env var (explicit override)
2. `/etc/reaper/reaper.conf` (standard location)
3. No file found → silently continue with env vars / defaults

---

## Implementation

### Phase 1: Shared config loader (`src/config.rs`)

New file shared by both binaries via `#[path]` attribute (no lib crate needed).

```rust
// ~40 lines: read file, parse KEY=VALUE, set env if not already set
pub fn load_config() {
let path = std::env::var("REAPER_CONFIG")
.unwrap_or_else(|_| "/etc/reaper/reaper.conf".to_string());
// read file, skip comments/blanks, split on first '=', set_var if not present
}
```

Key design:
- Uses `std::env::set_var` only if the key is NOT already set (env wins)
- No dependencies — just `std::fs` and `std::env`
- Silently skips if file doesn't exist (not an error)
- Logs a warning for malformed lines (if tracing is initialized — but config loads before tracing, so just silently skip malformed lines)

### Phase 2: Wire into both binaries

**`src/bin/reaper-runtime/main.rs`**: Call `config::load_config()` as the very first line of `main()`, before tracing setup (since `REAPER_RUNTIME_LOG` comes from config).

**`src/bin/containerd-shim-reaper-v2/main.rs`**: Call `config::load_config()` as the very first line of `main()`, before version check and tracing setup.

### Phase 3: Update Ansible playbook

Replace the three tasks (write to `/etc/default/containerd`, write systemd drop-in, daemon-reload) with:

1. Create `/etc/reaper/` directory
2. Write `/etc/reaper/reaper.conf` with template
3. Remove the systemd `EnvironmentFile` drop-in (no longer needed)

### Phase 4: Update documentation

- Update `CLAUDE.md` to mention `/etc/reaper/reaper.conf`
- Update `deploy/kubernetes/README.md` if it references the old approach

---

## File Changes

| File | Action | Description |
|------|--------|-------------|
| `src/config.rs` | **Create** | Shared config file loader (~40 lines) |
| `src/bin/reaper-runtime/main.rs` | **Edit** | Add `mod config; config::load_config();` at top of main() |
| `src/bin/containerd-shim-reaper-v2/main.rs` | **Edit** | Add `mod config; config::load_config();` at top of main() |
| `deploy/ansible/install-reaper.yml` | **Edit** | Write `/etc/reaper/reaper.conf` instead of `/etc/default/containerd` |
| `CLAUDE.md` | **Edit** | Document config file location |

---

## What We're NOT Doing

- **TOML/YAML config**: Adds a parsing dependency for flat key-value data
- **containerd runtime options**: Requires custom protobuf, containerd-specific
- **Distro detection**: Fragile, same result as just owning our config path
- **Removing env var support**: Env vars still override config file (backward compat)
Loading
Loading