feat(tee, attest-agent): NVIDIA CC part A, GPU evidence route - #1196
feat(tee, attest-agent): NVIDIA CC part A, GPU evidence route#1196odesenfans wants to merge 7 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev-2.1 #1196 +/- ##
========================================
Coverage 78.85% 78.85%
========================================
Files 146 146
Lines 15884 15884
Branches 1000 1000
========================================
Hits 12525 12525
Misses 3185 3185
Partials 174 174 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
foxpatch-aleph
left a comment
There was a problem hiding this comment.
Solid, fail-closed first slice of the NVIDIA CC stack. I independently verified the pinned gpu_nonce vectors (all three match a Python recomputation), confirmed the wait()-after-try_wait() pattern at gpu.rs:152 is correct on Unix (std caches the exit status; try_wait stores it and wait returns the cached value, so no ECHILD), and confirmed the pipe-drain fix is sound with the 200 KiB regression test. The route is 404 when the GPU feature is off, nonce handling is shared with the SNP route, error detail stays in the guest log, and the CLI flags are mutually required. The dev-only Cargo.lock additions (tempfile) do not enter the measured binary. Remaining observations are minor robustness notes below; none block.
rust/crates/aleph-attest-agent/src/gpu.rs (line 137): On the timeout path the two reader-thread JoinHandles are dropped without being joined: the threads detach and normally exit once the pipes close. That is the right call for the grandchild-holds-the-pipe case (joining could block forever), but it means a collector that spawns a surviving grandchild would leak two threads per timeout. Worth a comment noting the detach is deliberate, since joining would be the worse option here.
rust/crates/aleph-attest-agent/src/proxy.rs (line 150): The lock is held across the entire web::block collect (up to COLLECT_TIMEOUT, 60 s) with no bound on how many callers queue and no timeout on acquisition, so N concurrent callers can each wait up to N*60 s with no backpressure. Callers are attested clients, so this is low risk, but a try_lock-based 503 or a bounded wait would turn pile-ups into fast failures instead of long hangs.
rust/crates/aleph-attest-agent/src/gpu.rs (line 40): The timeout/kill branch has no test coverage (the other collect() paths all do). A collector script that sleeps forever plus a shorter timeout parameterized for tests (or #[cfg(test)] override) would cover the kill-and-bail path, including the detach behavior above.
Approved design for single-GPU NVIDIA confidential computing inside a measured SEV-SNP guest: schema slot on the V-PROGRAM, CC-mode probe and passthrough on the CRN, driver plus NVIDIA's local verifier in a new measured image flavor, and pure-Rust binding checks in the aleph-rs client over evidence nonce-bound to the attested TLS key.
…xtra sidecar Seventeen tasks in five stacked PRs: attest-agent GPU route, daemon CC-mode probe and advertisement, SNP passthrough argv and gate, Nix gpuImage flavor with in-guest verification, manifest gpu block and agent resolution. The spec gains the cmdline_extra sidecar: the daemon derives the measured cmdline from sidecars, so the runtime's fixed swiotlb token needs one, validated against a closed allowlist before it reaches -append.
…idia-smi The attest-agent is a static musl binary inside a content-only measured initrd, so it cannot dlopen NVIDIA's glibc NVML. Tasks 2 and 3 now run NVIDIA's own collect-evidence as a chrooted child process with the derived nonce appended, and init sets the ready state with nvidia-smi. The GPU step also prepares the rootfs chroot before verification, since nvattest needs /proc, /sys and /dev bound there.
…with the derived nonce
… evidence cannot deadlock
…evidence over the attested channel
ee9d1a3 to
46bbbb1
Compare
foxpatch-aleph
left a comment
There was a problem hiding this comment.
Part A of the NVIDIA CC stack is solid: the gpu_nonce scheme is correctly domain-separated and all three pinned vectors independently verify, the collector's pipe-draining fix is correct (and regression-tested at 200 KiB), the nonce-refusal logic in parse_output is fail-closed, the decode_nonce refactor keeps the SNP route byte-identical, and the CLI gates are fail-closed. The findings below are non-blocking: a spec/impl inconsistency on the 503 error contract, an unbounded join() after the collector child exits, a stale sentence in the report_data module doc, and the timeout branch being the one piece of collect() with no test coverage.
docs/superpowers/specs/2026-09-04-nvidia-cc-design.md (line 248): The spec added in this PR documents a 503 with {"error": "gpu not attested"} for "boot-time verification did not complete", but the implementation never emits it: gpu_attestation_endpoint (proxy.rs:141) returns 404 when GpuState is absent and 500 on collection failure, and init powers off on boot-verification failure, so the 503 state cannot occur. Either drop the 503 from the spec or serve it defensively, so the aleph-rs verifier authors don't wire up handling for an unreachable code.
rust/crates/aleph-attest-agent/src/gpu.rs (line 145): COLLECT_TIMEOUT only bounds the direct child: after try_wait() reports exit, stdout_handle.join() has no deadline. If the collector ever spawns a helper that inherits stdout and outlives the direct child (or a grandchild survives the kill() on the timeout path), the join blocks forever, the web::block worker stays pinned, and GpuState.lock is never released, so every subsequent GPU attestation request queues indefinitely. nvattest doesn't fork today and the command line comes from the measured initrd, so not blocking, but worth either bounding the reads or noting the invariant (the collector must not leak pipe holders).
rust/crates/aleph-tee/src/report_data.rs (line 34): With the third shape added, the closing sentence "Both schemes write SHA-384(...) into the first 48 bytes of the 64-byte field" reads as stale and now ambiguous: the GPU-nonce scheme writes a 32-byte SHA-256 and is not a report_data at all. Consider "Both report_data schemes write ..." (and the "two namespaces" on line 28 could become "the schemes").
rust/crates/aleph-attest-agent/src/gpu.rs (line 136): The timeout branch (kill + bail) is the one piece of collect() logic with no test: the fake-collector tests all exercise collectors that exit on their own. Consider making the timeout injectable (e.g. a collect_with_timeout(duration) used by collect()) so a test can pass a short bound against a sleeping script and assert the kill path, rather than waiting 60 s.
NVIDIA CC on SEV-SNP, part A of 5: attest-agent GPU evidence route
First PR of the confidential-GPU stack. Design:
docs/superpowers/specs/2026-09-04-nvidia-cc-design.md, plan:docs/superpowers/plans/2026-09-04-nvidia-cc-aleph-vm.md(both in this PR).What
aleph-tee:gpu_nonce(served_key, client_nonce), the key-bound, domain-separated SHA-256 SPDM nonce (aleph-gpu-nonce-v1), with a pinned vector shared with the aleph-rs mirror.aleph-attest-agent: aGpuEvidenceSourcetrait whose production implementation runs NVIDIA'snvattest collect-evidenceas a child process (chrooted into the verity rootfs by init) with the derived nonce as the last argument, parses its JSON, and refuses evidence answering any other nonce. The agent is a static musl binary inside a content-only initrd, so it cannot load NVIDIA's glibc NVML itself.GET /.well-known/attestation/gpu?nonce=<hex>returning{tee_type:"nvidia-cc", client_nonce, gpus:[{arch,nonce,evidence,certificate}], boot_claims}over the attested channel, enabled by--gpu-claimsand--gpu-collector, 404 otherwise. The existing/.well-known/attestationresponse is byte-identical.Golden measurements
The agent binary is baked into the measured initrd, so this PR moves every base launch measurement. The re-seed lands in part D (nix); expect the golden-measurements CI job to be red on A, B and C and green from D on.
Testing
cargo testinrust/and in the agent's own workspace; the pipe-draining collector has a 200 KiB output test (a poll-then-read loop deadlocked on the 64 KiB pipe buffer during review and was fixed).Stack
A (this) -> B
od/nvidia-cc-b-cc-probe-> Cod/nvidia-cc-c-snp-argv-> Dod/nvidia-cc-d-nix-gpu-image-> Eod/nvidia-cc-e-agent.🤖 Generated with Claude Code
https://claude.ai/code/session_01QYHri2Q9pToxL4yVWkVJ2x