Skip to content

bug: framework.docker fails with non-root images #92

@aefhm

Description

@aefhm

Problem

When using framework.docker with mount form, amber-helper crashes with permission denied:

ERROR failed to bind docker socket /var/run/docker.sock: Permission denied (os error 13)

Root Cause

  1. amber-helper needs to create /var/run/docker.sock inside the agent container
  2. /var/run/ is owned by root with mode 755 (not writable by non-root users)
  3. The compiler does not set user: on agent services in compose.yaml
  4. Agents inherit the USER from their base image:
    • Base image with no USER → runs as root → works ✅
    • Base image with USER directive → runs as non-root → fails ❌

Reproduction

Create these files:

Dockerfile:

# Minimal non-root image to reproduce docker socket bug
FROM docker:27.3.1-cli

# Create non-root user
RUN adduser -D testuser

# Switch to non-root user
USER testuser

WORKDIR /home/testuser

ENTRYPOINT ["sh", "-c"]
CMD ["docker version"]

scenario.json5:

{
  manifest_version: "0.1.0",
  experimental_features: ["docker"],
  components: {
    worker: {
      manifest: "./worker.json5",
    },
  },
  bindings: [
    { to: "#worker.docker", from: "framework.docker" },
  ],
  exports: {
    status: "#worker.status",
  },
}

worker.json5:

{
  manifest_version: "0.1.0",
  experimental_features: ["docker"],
  program: {
    image: "test-docker-socket-bug:latest",
    entrypoint: ["sh", "-c", "test -S /var/run/docker.sock && docker version"],
    mounts: [
      { path: "/var/run/docker.sock", from: "framework.docker" },
    ],
    env: {
      DOCKER_HOST: "${slots.docker.url}",
    },
    network: {
      endpoints: [
        { name: "status", port: 8080, protocol: "http" },
      ],
    },
  },
  slots: {
    docker: { kind: "docker" },
  },
  provides: {
    status: { kind: "http", endpoint: "status" },
  },
  exports: {
    status: "status",
  },
}

Then run:

docker build -t test-docker-socket-bug:latest .
amber compile scenario.json5 --docker-compose /tmp/test-bug
export AMBER_DOCKER_SOCK="$HOME/.docker/run/docker.sock"  # macOS
docker compose -f /tmp/test-bug/compose.yaml up

Expected error:

c1-worker-1 | ERROR failed to bind docker socket /var/run/docker.sock: Permission denied (os error 13)
c1-worker-1 exited with code 1

Why examples/framework-docker works

It uses docker:27.3.1-cli which has no USER directive (runs as root).

Solution Options

1. Compiler sets user: "0:0" when framework.docker mount is present

Pros: Simple, guarantees socket creation works
Cons: Forces entire agent to run as root

Implementation: In compiler/src/targets/mesh/docker_compose/mod.rs, when building agent service, check if has_docker_mount is true and set program_service.user = Some("0:0".to_string())

2. amber-helper uses /tmp/docker.sock instead

Pros: No privilege escalation needed
Cons: Requires setting DOCKER_HOST env var, some tools may ignore it

Implementation: Change socket path in amber-helper when creating proxy socket, also set DOCKER_HOST automatically

3. Fix in agent Dockerfiles

Pros: No Amber code changes, keeps containers running as non-root
Cons: Requires documentation

Implementation: Document that agent Dockerfiles using framework.docker must make /var/run writable:

RUN chmod 777 /var/run
USER agent

Recommendation

Option 3 - Document the requirement for agent Dockerfiles. This keeps agent containers running as non-root while allowing amber-helper to create the socket. The security boundary remains the Docker socket itself, not filesystem permissions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions