Shim failing to run on docker v28+ #378
Description
The shim is currently failing to run Spin apps when directly executed by Docker. It works with Docker v27 but hangs for all greater versions.
This points out a gap in our tests since we only test the shim on Kubernetes (k3d). This PR adds a script to test on docker using docker-in-docker: #377.
Issue
The Spin app never starts:
$ docker run --name spin --runtime io.containerd.spin.v2 --platform wasi/wasm --publish 8080:80 ghcr.io/spinframework/containerd-shim-spin/examples
/spin-rust-hello:v0.22.0 /
Unable to find image 'ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.22.0' locally
v0.22.0: Pulling from spinframework/containerd-shim-spin/examples/spin-rust-hello
25d3662ed73c: Pull complete
891ca7c43ac7: Pull complete
25b36397191d: Pull complete
Digest: sha256:ad8874b8560cc5e3bf17367bb534e404b16dc7fe097975f34cf43b1f96b9e015
Status: Downloaded newer image for ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.22.0
# HANGING HEREIt should instead run the app:
$ docker run --name spin --runtime io.containerd.spin.v2 --platform wasi/wasm --publish 8080:80 ghcr.io/spinframework/containerd-shim-spin/examples
/spin-rust-hello:v0.20.0 /
Unable to find image 'ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.20.0' locally
v0.20.0: Pulling from spinframework/containerd-shim-spin/examples/spin-rust-hello
a8350dc9cb8c: Download complete
53dcbf4e5934: Download complete
f012b1ca8a19: Download complete
Digest: sha256:1c3bcefb614e433c0c0548aa6b36b6d70eb2c27ae438bc60b088d1b7d1440b34
Status: Downloaded newer image for ghcr.io/spinframework/containerd-shim-spin/examples/spin-rust-hello:v0.20.0
Serving [http://0.0.0.0:80](http://0.0.0.0/)
Available Routes:
hello: [http://0.0.0.0:80/hello](http://0.0.0.0/hello)
go-hello: [http://0.0.0.0:80/go-hello](http://0.0.0.0/go-hello)Reproduce
Checkout this PR #377 and then navigate to the test directory:
git clone -b docker-integration-test [email protected]:kate-goldenring/containerd-shim-spin.git
cd tests/docker-integration-test
# Test the latest shim on Docker v27 which succeeds
CONTAINERD_SPIN_SHIM_VERSION=v0.22.0 DOCKER_VERSION=27 ./tests/docker-integration-test/docker-integration-test.sh
# Test the latest (or older) shims on Docker v28 which fails
CONTAINERD_SPIN_SHIM_VERSION=v0.22.0 DOCKER_VERSION=28 ./tests/docker-integration-test/docker-integration-test.shLogs
Logging can be configured by adding the RUST_LOG env var to the systemctl config file:
systemctl edit containerdAdd the following to the file that pops up and then restart containerd:
[Service]
Environment="RUST_LOG=info"Nothing in the traces stood out to me
Execution in the shim gets to this point where "no WASM layers found in OCI image" is logged but then no further progress is being made. This log is expected since it is a docker image not a wasm oci image.
Nov 16 05:52:47 katedev containerd[864562]: time="2025-11-16T05:52:47.991787785Z" level=info msg="create_task_service; self=Shim { namespace: "moby", containerd_address: "/run/containerd/containerd.sock", _id: "ea20377eefe0f968dc87704a7723617b07876d1a9b4c7c598615da33c5eea0d2" }"
Nov 16 05:52:47 katedev containerd[864562]: time="2025-11-16T05:52:47.991908951Z" level=info msg="server listen started"
Nov 16 05:52:47 katedev containerd[864562]: time="2025-11-16T05:52:47.991917017Z" level=info msg="server started"
Nov 16 05:52:47 katedev containerd[864562]: time="2025-11-16T05:52:47.99192919Z" level=info msg="Shim successfully started, waiting for exit signal..."
Nov 16 05:52:47 katedev containerd[864562]: time="2025-11-16T05:52:47.991972655Z" level=info msg="wait; self=Shim { namespace: "moby", containerd_address: "/run/containerd/containerd.sock", _id: "ea20377eefe0f968dc87704a7723617b07876d1a9b4c7c598615da33c5eea0d2" }"
Nov 16 05:52:47 katedev containerd[864562]: time="2025-11-16T05:52:47.992357083Z" level=info msg="create; _ctx=TtrpcContext { fd: 14, cancel_rx: Receiver { .. }, mh: MessageHeader { length: 587, stream_id: 1, type_: 1, flags: 0 }, res_tx: Sender { .. }, metadata: {"containerd-namespace-ttrpc": ["moby"]}, timeout_nano: 0 } req=CreateTaskRequest { id: "ea20377eefe0f968dc87704a7723617b07876d1a9b4c7c598615da33c5eea0d2", bundle: "/run/containerd/io.containerd.runtime.v2.task/moby/ea20377eefe0f968dc87704a7723617b07876d1a9b4c7c598615da33c5eea0d2", rootfs: [], terminal: false, stdin: "", stdout: "/var/run/docker/containerd/ea20377eefe0f968dc87704a7723617b07876d1a9b4c7c598615da33c5eea0d2/ea20377eefe0f968dc87704a7723617b07876d1a9b4c7c598615da33c5eea0d2-stdout", stderr: "/var/run/docker/containerd/ea20377eefe0f968dc87704a7723617b07876d1a9b4c7c598615da33c5eea0d2/ea20377eefe0f968dc87704a7723617b07876d1a9b4c7c598615da33c5eea0d2-stderr", checkpoint: "", parent_checkpoint: "", options: MessageField(None), special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }"
Nov 16 05:52:47 katedev containerd[864562]: time="2025-11-16T05:52:47.994602343Z" level=info msg="found manifest with WASM OCI image format"
Nov 16 05:52:47 katedev containerd[864562]: time="2025-11-16T05:52:47.99461632Z" level=info msg="no WASM layers found in OCI image"
Nov 16 05:52:47 katedev containerd[864562]: time="2025-11-16T05:52:47.995891472Z" level=info instance="ea20377eefe0f968dc87704a7723617b07876d1a9b4c7c598615da33c5eea0d2" msg="cgroup manager V2 will be used"
Nov 16 05:52:47 katedev containerd[864562]: time="2025-11-16T05:52:47.996013249Z" level=warn
Traces
Tracing can be configured by running jaeger and then adding the otel exporter address as an env var to the systemctl config file:
systemctl edit containerdAdd the following to the file that pops up and then restart containerd:
[Service]
Environment="OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318/"Nothing in the traces stood out to me