Skip to content

feat(tee, attest-agent): NVIDIA CC part A, GPU evidence route - #1196

Open
odesenfans wants to merge 7 commits into
dev-2.1from
od/nvidia-cc-a-attest-agent
Open

feat(tee, attest-agent): NVIDIA CC part A, GPU evidence route#1196
odesenfans wants to merge 7 commits into
dev-2.1from
od/nvidia-cc-a-attest-agent

Conversation

@odesenfans

Copy link
Copy Markdown
Contributor

NVIDIA CC on SEV-SNP, part A of 5: attest-agent GPU evidence route

First PR of the confidential-GPU stack. Design: docs/superpowers/specs/2026-09-04-nvidia-cc-design.md, plan: docs/superpowers/plans/2026-09-04-nvidia-cc-aleph-vm.md (both in this PR).

What

  • aleph-tee: gpu_nonce(served_key, client_nonce), the key-bound, domain-separated SHA-256 SPDM nonce (aleph-gpu-nonce-v1), with a pinned vector shared with the aleph-rs mirror.
  • aleph-attest-agent: a GpuEvidenceSource trait whose production implementation runs NVIDIA's nvattest collect-evidence as a child process (chrooted into the verity rootfs by init) with the derived nonce as the last argument, parses its JSON, and refuses evidence answering any other nonce. The agent is a static musl binary inside a content-only initrd, so it cannot load NVIDIA's glibc NVML itself.
  • New route GET /.well-known/attestation/gpu?nonce=<hex> returning {tee_type:"nvidia-cc", client_nonce, gpus:[{arch,nonce,evidence,certificate}], boot_claims} over the attested channel, enabled by --gpu-claims and --gpu-collector, 404 otherwise. The existing /.well-known/attestation response is byte-identical.

Golden measurements

The agent binary is baked into the measured initrd, so this PR moves every base launch measurement. The re-seed lands in part D (nix); expect the golden-measurements CI job to be red on A, B and C and green from D on.

Testing

cargo test in rust/ and in the agent's own workspace; the pipe-draining collector has a 200 KiB output test (a poll-then-read loop deadlocked on the 64 KiB pipe buffer during review and was fixed).

Stack

A (this) -> B od/nvidia-cc-b-cc-probe -> C od/nvidia-cc-c-snp-argv -> D od/nvidia-cc-d-nix-gpu-image -> E od/nvidia-cc-e-agent.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QYHri2Q9pToxL4yVWkVJ2x

@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.85%. Comparing base (0d85bf4) to head (46bbbb1).

Additional details and impacted files
@@           Coverage Diff            @@
##           dev-2.1    #1196   +/-   ##
========================================
  Coverage    78.85%   78.85%           
========================================
  Files          146      146           
  Lines        15884    15884           
  Branches      1000     1000           
========================================
  Hits         12525    12525           
  Misses        3185     3185           
  Partials       174      174           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@foxpatch-aleph foxpatch-aleph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid, fail-closed first slice of the NVIDIA CC stack. I independently verified the pinned gpu_nonce vectors (all three match a Python recomputation), confirmed the wait()-after-try_wait() pattern at gpu.rs:152 is correct on Unix (std caches the exit status; try_wait stores it and wait returns the cached value, so no ECHILD), and confirmed the pipe-drain fix is sound with the 200 KiB regression test. The route is 404 when the GPU feature is off, nonce handling is shared with the SNP route, error detail stays in the guest log, and the CLI flags are mutually required. The dev-only Cargo.lock additions (tempfile) do not enter the measured binary. Remaining observations are minor robustness notes below; none block.

rust/crates/aleph-attest-agent/src/gpu.rs (line 137): On the timeout path the two reader-thread JoinHandles are dropped without being joined: the threads detach and normally exit once the pipes close. That is the right call for the grandchild-holds-the-pipe case (joining could block forever), but it means a collector that spawns a surviving grandchild would leak two threads per timeout. Worth a comment noting the detach is deliberate, since joining would be the worse option here.

rust/crates/aleph-attest-agent/src/proxy.rs (line 150): The lock is held across the entire web::block collect (up to COLLECT_TIMEOUT, 60 s) with no bound on how many callers queue and no timeout on acquisition, so N concurrent callers can each wait up to N*60 s with no backpressure. Callers are attested clients, so this is low risk, but a try_lock-based 503 or a bounded wait would turn pile-ups into fast failures instead of long hangs.

rust/crates/aleph-attest-agent/src/gpu.rs (line 40): The timeout/kill branch has no test coverage (the other collect() paths all do). A collector script that sleeps forever plus a shorter timeout parameterized for tests (or #[cfg(test)] override) would cover the kill-and-bail path, including the detach behavior above.

Approved design for single-GPU NVIDIA confidential computing inside a
measured SEV-SNP guest: schema slot on the V-PROGRAM, CC-mode probe and
passthrough on the CRN, driver plus NVIDIA's local verifier in a new
measured image flavor, and pure-Rust binding checks in the aleph-rs client
over evidence nonce-bound to the attested TLS key.
…xtra sidecar

Seventeen tasks in five stacked PRs: attest-agent GPU route, daemon CC-mode
probe and advertisement, SNP passthrough argv and gate, Nix gpuImage flavor
with in-guest verification, manifest gpu block and agent resolution.

The spec gains the cmdline_extra sidecar: the daemon derives the measured
cmdline from sidecars, so the runtime's fixed swiotlb token needs one,
validated against a closed allowlist before it reaches -append.
…idia-smi

The attest-agent is a static musl binary inside a content-only measured
initrd, so it cannot dlopen NVIDIA's glibc NVML. Tasks 2 and 3 now run
NVIDIA's own collect-evidence as a chrooted child process with the derived
nonce appended, and init sets the ready state with nvidia-smi. The GPU
step also prepares the rootfs chroot before verification, since nvattest
needs /proc, /sys and /dev bound there.

@foxpatch-aleph foxpatch-aleph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Part A of the NVIDIA CC stack is solid: the gpu_nonce scheme is correctly domain-separated and all three pinned vectors independently verify, the collector's pipe-draining fix is correct (and regression-tested at 200 KiB), the nonce-refusal logic in parse_output is fail-closed, the decode_nonce refactor keeps the SNP route byte-identical, and the CLI gates are fail-closed. The findings below are non-blocking: a spec/impl inconsistency on the 503 error contract, an unbounded join() after the collector child exits, a stale sentence in the report_data module doc, and the timeout branch being the one piece of collect() with no test coverage.

docs/superpowers/specs/2026-09-04-nvidia-cc-design.md (line 248): The spec added in this PR documents a 503 with {"error": "gpu not attested"} for "boot-time verification did not complete", but the implementation never emits it: gpu_attestation_endpoint (proxy.rs:141) returns 404 when GpuState is absent and 500 on collection failure, and init powers off on boot-verification failure, so the 503 state cannot occur. Either drop the 503 from the spec or serve it defensively, so the aleph-rs verifier authors don't wire up handling for an unreachable code.

rust/crates/aleph-attest-agent/src/gpu.rs (line 145): COLLECT_TIMEOUT only bounds the direct child: after try_wait() reports exit, stdout_handle.join() has no deadline. If the collector ever spawns a helper that inherits stdout and outlives the direct child (or a grandchild survives the kill() on the timeout path), the join blocks forever, the web::block worker stays pinned, and GpuState.lock is never released, so every subsequent GPU attestation request queues indefinitely. nvattest doesn't fork today and the command line comes from the measured initrd, so not blocking, but worth either bounding the reads or noting the invariant (the collector must not leak pipe holders).

rust/crates/aleph-tee/src/report_data.rs (line 34): With the third shape added, the closing sentence "Both schemes write SHA-384(...) into the first 48 bytes of the 64-byte field" reads as stale and now ambiguous: the GPU-nonce scheme writes a 32-byte SHA-256 and is not a report_data at all. Consider "Both report_data schemes write ..." (and the "two namespaces" on line 28 could become "the schemes").

rust/crates/aleph-attest-agent/src/gpu.rs (line 136): The timeout branch (kill + bail) is the one piece of collect() logic with no test: the fake-collector tests all exercise collectors that exit on their own. Consider making the timeout injectable (e.g. a collect_with_timeout(duration) used by collect()) so a test can pass a short bound against a sleeping script and assert the kill path, rather than waiting 60 s.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants