What is left undone
On a host-fabric transport (RDMA or EFA), the operator renders everything the fabric needs —
hostNetwork, /dev/infiniband, IPC_LOCK/SYS_RESOURCE, and on EFA the host's
/opt/amazon/efa tree plus LD_LIBRARY_PATH — into the backend's member Pods only. The
engine Pods (vLLM / SGLang) that consume the pool get none of it: the injection webhook renders
the transport name (MOONCAKE_PROTOCOL / mooncake.json protocol) and nothing the fabric
needs.
On TCP that distinction costs nothing — it is plain pod networking. On RDMA or EFA, an
engine that is to move bytes over the fabric needs the same device and library access on its own
Pod, and today its transfer engine either fails to install the transport or silently falls back.
This gap predates EFA — RDMA has always had it; EFA inherits it. The enum's own bar ("only
TCP has been exercised end to end") is downstream of exactly this.
What deciding this looks like
- Whether engine Pods should get fabric access at all: it grants
hostNetwork and
capabilities to tenant-adjacent workloads, a much wider blast radius than the cluster-scoped
member DaemonSet. The member-side rule "a privilege is requested, never inferred" has no
obvious engine-side spelling — there is no field on the engine's owning object that asks for
it.
- If yes, how: a per-pool or per-backend opt-in the injection webhook can see at mutate time,
rendering the same host-fabric base (and the EFA extras) into the engine container; or a
documented node-level answer (device plugin / CNI) the operator stays out of.
- What the honest interim is: the docs now state the gap on both pages a reader would look
for it (docs/kv-cache/backend.md, docs/reference/kv-cache-injection.md). If the decision is
"members only, engines stay on TCP", that should be said as a rule, not left as an absence.
References
- Member-side rendering (the half that exists):
applyMemberFabric in
pkg/worker/kvcache/mooncake/member_workload.go.
- Engine-side injection (the half that only names the transport):
pkg/worker/kvcache/inject/engine.go, pkg/worker/webhooks/worker/pod_kv_cache_resolve.go.
What is left undone
On a host-fabric transport (
RDMAorEFA), the operator renders everything the fabric needs —hostNetwork,/dev/infiniband,IPC_LOCK/SYS_RESOURCE, and on EFA the host's/opt/amazon/efatree plusLD_LIBRARY_PATH— into the backend's member Pods only. Theengine Pods (vLLM / SGLang) that consume the pool get none of it: the injection webhook renders
the transport name (
MOONCAKE_PROTOCOL/ mooncake.jsonprotocol) and nothing the fabricneeds.
On
TCPthat distinction costs nothing — it is plain pod networking. OnRDMAorEFA, anengine that is to move bytes over the fabric needs the same device and library access on its own
Pod, and today its transfer engine either fails to install the transport or silently falls back.
This gap predates
EFA—RDMAhas always had it;EFAinherits it. The enum's own bar ("onlyTCP has been exercised end to end") is downstream of exactly this.
What deciding this looks like
hostNetworkandcapabilities to tenant-adjacent workloads, a much wider blast radius than the cluster-scoped
member DaemonSet. The member-side rule "a privilege is requested, never inferred" has no
obvious engine-side spelling — there is no field on the engine's owning object that asks for
it.
rendering the same host-fabric base (and the EFA extras) into the engine container; or a
documented node-level answer (device plugin / CNI) the operator stays out of.
for it (
docs/kv-cache/backend.md,docs/reference/kv-cache-injection.md). If the decision is"members only, engines stay on TCP", that should be said as a rule, not left as an absence.
References
applyMemberFabricinpkg/worker/kvcache/mooncake/member_workload.go.pkg/worker/kvcache/inject/engine.go,pkg/worker/webhooks/worker/pod_kv_cache_resolve.go.