Skip to content

feat(nix): aleph_insecure_unattested=1 mode with a plain-HTTP attest agent - #1188

Merged
odesenfans merged 12 commits into
mainfrom
od/vprogram-local-mode
Sep 7, 2026
Merged

feat(nix): aleph_insecure_unattested=1 mode with a plain-HTTP attest agent#1188
odesenfans merged 12 commits into
mainfrom
od/vprogram-local-mode

Conversation

@odesenfans

@odesenfans odesenfans commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Guest side of aleph vprogram run (local, non-confidential boot test; CLI side is aleph-rs #393).

  • aleph-attest-agent --insecure-plain-http: same routes, NoTeeBackend, plain bind. Attestation endpoint answers 500.
  • init-common.sh parses the measured cmdline token aleph_insecure_unattested=1 and starts the agent through the shared start_attest_agent helper; both inits use it. Production line unchanged. The console marker is init: INSECURE UNATTESTED MODE: attest agent serving plain HTTP without a TEE on tcp/8443.
  • boot-smoke.sh phase 3 curls the fib workload's /health through the plain agent over a SLIRP hostfwd; phases 1 and 2 assert the marker is absent.
  • New boot-smoke.yml workflow runs the smoke on a KVM-capable runner.
  • Golden measurements regenerated (every initrd embeds init-common.sh and the agent).

Naming: the token says what is lost, not where it runs. It disables attestation (no TEE, no attested identity, plain HTTP), not integrity: the dm-verity chain still verifies every disk in this mode.

Safety: the token is a launch-measurement input no production path can emit; a CRN adding it breaks the expected measurement and every attested call fails closed.

Design: aleph-rs docs/superpowers/specs/2026-09-02-vprogram-local-run-design.md (local-only).

🤖 Generated with Claude Code

https://claude.ai/code/session_014HkEvGaEvP3R3sbQvSRppF

@foxpatch-aleph foxpatch-aleph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-designed and well-executed guest-side local mode. The security story is coherent and documented in the right places: the aleph_local=1 token is a measured-cmdline input no production path emits, NoTeeBackend fails closed on attestation (pinned by a test asserting 500, never a fabricated report), --insecure-plain-http conflicts with --owner via clap (also tested), and init-instance.sh deliberately keeps its direct owner-mode agent invocation so local mode cannot downgrade confidential instances. The boot smoke gains both negative coverage (phases 1/2 assert the LOCAL MODE marker is absent) and positive e2e coverage (phase 3 through the SLIRP hostfwd), golden measurements are regenerated as required, and the new CI workflow's trigger surface matches the golden-measurement check it claims to mirror. Only minor nits remain.

nix/boot-smoke.sh (line 150): Minor: phase 3 guards command -v curl but not python3, so on a host without python3 the script dies with a bare "command not found" instead of the friendly message used for curl and the phase-2 tools. One more command -v check would match the existing pattern.

nix/init-common.sh (line 282): The LOCAL MODE marker is echoed before the agent is spawned, so a consumer that treats the marker as "agent is up" races the bind() — the smoke probe handles it with a 60s retry, but the aleph-rs CLI waiting on this line should retry too. Worth one sentence in the "keep it byte-stable" comment (e.g. "marker means starting, not ready") so the contract is explicit on the consumer side. Otherwise LGTM.

nix/boot-smoke.sh (line 150): The bind-close-print free-port pick is inherently TOCTOU-racy against QEMU's hostfwd bind; if it loses the race QEMU exits and the phase dumps the log, so it fails loudly. Fine for a smoke test — just noting it as a known flake source.

@odesenfans
odesenfans changed the base branch from dev-2.1 to main September 3, 2026 14:42
@odesenfans
odesenfans force-pushed the od/vprogram-local-mode branch from ad07d9d to 4731a0e Compare September 3, 2026 14:52

@foxpatch-aleph foxpatch-aleph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid, carefully-reasoned PR. The local-mode safety story checks out against the repo's measurement model (launch measurement covers OVMF+kernel+initrd+cmdline per flake.nix, so a production CRN injecting aleph_local=1 would break the pinned measurement and fail closed at RA-TLS), the plain agent fails closed for attestation (NoTeeBackend, tested), and the plain flag correctly conflicts with --owner. init-instance.sh is deliberately untouched, production cmdline lines are byte-identical, and golden measurements were regenerated in the same PR as the check script requires. The boot smoke covers the new path end-to-end (hostfwd -> plain agent -> loopback upstream -> fib /health) and even asserts the marker is absent in the non-local phases. Remaining findings are minor nits: a self-contradicting comment about tuple coercion, an un-checked python3 dependency in the smoke script, lockfile gaps in the workflow path filter, and the marker-before-bind race with the CLI sync contract.

rust/crates/aleph-attest-agent/src/main.rs (line 98): This comment ("unsized coercion does not happen inside a tuple") is contradicted by the if-branch on line 96: (Arc::new(NoTeeBackend::new()), None) relies on exactly that coercion to satisfy the annotated tuple type. Taken literally, the comment implies the plain branch shouldn't compile. It likely describes the earlier failure mode before the outer type annotation existed — consider rewording to say the annotation is what makes the element coercion work.

nix/boot-smoke.sh (line 149): Only curl is checked, but python3 (line 150) is equally required for phase 3 — a missing python3 aborts via set -e with a bare "command not found" instead of the friendly message the curl check gives. Consider mirroring the check.

.github/workflows/boot-smoke.yml (line 9): The paths: filter lists rust/Cargo.toml but not the lock files actually consumed by the build (rust/crates/aleph-attest-agent/Cargo.lock per flake.nix:73, and rust/Cargo.lock), so a dependency bump touching only a lockfile won't trigger the smoke.

nix/init-common.sh (line 282): The marker (the sync contract with aleph vprogram run) is echoed before the agent is backgrounded, so the CLI can observe it before the listener binds and race an immediate connect into ECONNREFUSED. Same async-start semantics as production, so non-blocking, but worth confirming the CLI-side handles the retry.

@foxpatch-aleph foxpatch-aleph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-designed change with a careful security story: local mode is gated on a measured-cmdline token that no production path can emit, attestation fails closed through NoTeeBackend (verified by a new test), --insecure-plain-http conflicts with --owner, and init-instance.sh keeps the owner/TLS agent hardcoded so the confidential instance image cannot be downgraded. The run_phase global-reset logic and the phase-3 SLIRP hostfwd are correct, and the smoke phases 1/2 assert the local-mode marker is absent. Only minor, non-blocking nits below.

nix/boot-smoke.sh (line 149): Phase 3 guards against a missing curl with a clear message, but python3 is equally required and is unguarded: with set -e, a missing python3 aborts here with a confusing 'command not found' inside the command substitution. Add a matching command -v python3 check for a symmetric failure message (or reuse the phase-2 for tool in ... loop pattern).

.github/workflows/boot-smoke.yml (line 12): The branch filters use 'dev*' (and [main, 'dev*'] on push), while golden-measurements.yml — whose trigger surface this deliberately mirrors per its comment — uses the exact branch dev. 'dev*' also matches unrelated branches like develop or dev-foo. If the glob is intentional (e.g. numbered dev branches), a one-line comment like the one in golden-measurements.yml would help; otherwise align it to dev.

nix/boot-smoke.sh (line 150): Minor: the bind-then-release free-port probe has an inherent TOCTOU window before QEMU's hostfwd claims the port; acceptable for a smoke test since a collision surfaces as a clear QEMU bind failure, but a retry or a fixed high port would make the failure mode even more obvious. Non-blocking observation.

rust/crates/aleph-attest-agent/src/main.rs (line 93): The Option plumbing is correct and the explicit Arc<dyn TeeBackend> annotation sidesteps the missing tuple coercion, but note the two new tests (CLI parse/conflict here, NoTeeBackend fail-closed in proxy.rs) were only verified by inspection in this review — no cargo toolchain was available. Worth confirming the rust CI job runs green before merge.

@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.31%. Comparing base (08ce72c) to head (57cd1a1).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1188      +/-   ##
==========================================
- Coverage   86.32%   86.31%   -0.01%     
==========================================
  Files         243      243              
  Lines       27632    27632              
  Branches     1592     1592              
==========================================
- Hits        23852    23850       -2     
- Misses       3395     3396       +1     
- Partials      385      386       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Plain HTTP on the same port with the same routes, NoTeeBackend behind the
attestation endpoint (500, never a fabricated report), no attested
identity. Selected by the guest init only when the measured cmdline
carries aleph_local=1.
…ocal mode

Parsed once in init-common.sh; both measured inits start the agent through
the shared start_attest_agent helper. Production line unchanged; local mode
prints a stable LOCAL MODE marker for the CLI runner to wait on.
…-HTTP hostfwd

Phases 1 and 2 now also assert the LOCAL MODE marker is absent.
init-common.sh and the attest-agent binary are in every initrd flavor, so
all four measured chains move.
…-owner with --insecure-plain-http

- nix/init-compose.sh: start_attest_agent (and the LOCAL MODE marker) now
  runs before the "starting /sbin/init from rootfs" line, matching init.sh.
- rust/crates/aleph-attest-agent: --insecure-plain-http now conflicts with
  --owner (owner auth binds to a served TLS key plain mode does not have),
  with a test pinning the rejection.
- nix/boot-smoke.sh: header now states it runs in CI (boot-smoke.yml) and
  locally, not just as a local tool.
- nix/init-common.sh: header lists init-compose.sh among the sourcers and
  re-wraps to the file's usual width.
- rust/crates/aleph-attest-agent/src/proxy.rs: trim the no_tee_state() doc
  comment to what the test actually asserts (fail-closed attestation).
@odesenfans
odesenfans force-pushed the od/vprogram-local-mode branch from e988d0b to 5523bbd Compare September 4, 2026 13:31
- boot-smoke.sh: guard python3 like curl before phase 3
- init-common.sh: state that the LOCAL MODE marker means starting, not listening
- attest-agent main.rs: fix the self-contradicting tuple coercion comment
- boot-smoke.yml: explain the dev* branch glob

@foxpatch-aleph foxpatch-aleph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid, security-conscious implementation. The local-mode opt-in is correctly keyed on the measured aleph_local=1 cmdline token, fails closed everywhere (NoTeeBackend 500s the attestation endpoint, --insecure-plain-http conflicts with --owner, a malicious CRN adding the token breaks the launch measurement), and the firewall still restricts the plain listener to tcp/8443. Test coverage is layered: unit tests for CLI parsing and the no-TEE fail-closed endpoint, plus an end-to-end boot-smoke phase 3 through a SLIRP hostfwd, with negative markers in phases 1/2. Golden measurements are regenerated for all four images, consistent with every initrd embedding init-common.sh and the agent. No correctness or security issues found; only minor observations below.

nix/boot-smoke.sh (line 152): Minor: the bind-then-close ephemeral port selection has a TOCTOU window between python3 exiting and QEMU binding the hostfwd; if another process claims the port in between, QEMU exits with "Could not set up host forwarding rule" and phase 3 fails as a flake. Rare on a dedicated runner, so non-blocking.

nix/boot-smoke.sh (line 155): Nit: the probe's 60-retry loop runs inside a single iteration of run_phase's 120s marker deadline, so a phase can take up to ~180s wall-clock. Harmless given the 90-minute CI timeout, but worth knowing that the deadline isn't enforced while the probe is running.

nix/init-instance.sh (line 42): Worth a one-line comment here: the instance init deliberately bypasses start_attest_agent (owner-auth mode conflicts with --insecure-plain-http), so the local_mode variable parsed by init-common.sh is inert on this image. Booting an instance image with aleph_local=1 still fails closed (no /dev/sev-guest), which is the desired behavior, but a note would spare future readers a double take.

rust/crates/aleph-attest-agent/src/main.rs (line 93): The tuple-annotation dance to coerce both arms to Arc<dyn TeeBackend> is a bit subtle but well-commented, and the surrounding comments accurately describe the fail-closed semantics (empty served key, 500 from the attestation endpoint). No change needed.

@foxpatch-aleph foxpatch-aleph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-executed guest-side local mode. The security reasoning holds up under scrutiny: the aleph_local=1 token is a launch-measurement input that no production path can emit without breaking attested clients, and the plain listener cannot complete a TLS handshake, so there is no downgrade path. The attest-agent restructure is correct (NoTeeBackend fails closed with a 500, covered by a test; --insecure-plain-http correctly conflicts with --owner), the init changes are minimal and keep init-instance.sh's owner-mode start intact, the boot-smoke phase 3 exercises the exact proxy path production uses, and all four golden measurements were regenerated as required. Only minor documentation and workflow-filter nits remain.

docs/architecture/confidential.md (line 212): This paragraph still describes the agent invocation as the unconditional aleph-attest-agent --port 8443 --upstream http://127.0.0.1:8080. With start_attest_agent the local-mode variant (--insecure-plain-http under aleph_local=1) is now part of the guest contract; a one-sentence mention here (and possibly in the cmdline-token list a few lines up) would keep the architecture doc accurate.

.github/workflows/boot-smoke.yml (line 16): The paths filter omits rust/Cargo.lock, same as golden-measurements.yml — but that workflow has a weekly cron to backstop filter misses, and this one does not, so a lockfile-only dependency bump would never run the smoke. Minor; consider adding the lock or accepting the gap.

nix/init-common.sh (line 272): Informational: the \b prefix means a dotted kernel param like x.aleph_local=1 would also enable local mode (verified with sed), slightly weakening the "no other token ends in aleph_local" claim. Purely theoretical given the measurement gate covers any injected token, so no change requested.

nix/init-common.sh (line 284): Nit: the marker says the agent is "serving" but is printed before the agent is backgrounded. The consumer contract (keep probing tcp/8443) is documented right above, so this is only a readability observation for someone reading the serial console.

…ed=1

"local" said where the guest runs, not what it gives up. The token puts
the attest agent on a fake TEE backend serving plain HTTP with no attested
identity, so nothing about the guest is verifiable from outside; the name
and the console marker now say so. dm-verity is unaffected.
@odesenfans odesenfans changed the title feat(nix): aleph_local=1 local mode with a plain-HTTP attest agent feat(nix): aleph_insecure_unattested=1 mode with a plain-HTTP attest agent Sep 7, 2026

@foxpatch-aleph foxpatch-aleph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-scoped local-testing mode: the measured cmdline token gates a plain-HTTP attest agent that preserves the production proxy/inject-secret routes, fails closed on attestation (500 via NoTeeBackend), rejects --owner, and leaves the dm-verity chain untouched; golden measurements are correctly regenerated since every initrd embeds init-common.sh and the agent. The sed word-boundary pattern mirrors the proven roothash parsing, boot-smoke phase 3 exercises the full hostfwd-to-/health path, and the new clap-flag and NoTeeBackend tests cover the changed behavior. No blocking issues found.

nix/init-compose.sh (line 209): The "starting /sbin/init from rootfs" echo now sits after start_attest_agent while init.sh keeps it before the agent start. Harmless (markers are grep-based), but worth a one-line reason or aligning the order with init.sh.

nix/boot-smoke.sh (line 152): The ephemeral port is freed when python3 exits, and SLIRP binds it only when qemu starts; if the port is grabbed in that window the phase fails with a clear SLIRP bind error in the log. Acceptable, but a retry with a fresh port would make the phase more robust.

nix/init-instance.sh (line 42): init-instance.sh now parses unattested_mode from init-common.sh but deliberately keeps its direct --owner "$owner" agent start (the helper does not support owner mode). Worth an explicit note here or in init-common.sh so the asymmetry is documented rather than implicit.

@odesenfans
odesenfans merged commit 9a4ce03 into main Sep 7, 2026
33 of 36 checks passed
@odesenfans
odesenfans deleted the od/vprogram-local-mode branch September 7, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants