Is there a way to attach buildx nodes with different drivers to the same builder?

[vkhobor]

Hi, I am trying to setup my raspberry so that ARM64 platform images are built on it automatically.

• I have set up a moby/buildkit container on my pi, that I exposed on a tcp port.
• I have added a local amd64 builder instance with docker buildx create

docker buildx create \
  --name local_remote_builder \
  --node local_remote_builder \
  --platform linux/amd64
local_remote_builder

• Then when I want to add the remote instance and append it to the builder I get this error:

docker buildx create \
    --name local_remote_builder \
    --append \
    --node raspberry \
    --platform linux/arm64 \
    --driver remote \
     tcp://raspi.home:7571
ERROR: existing instance for "local_remote_builder" but has mismatched driver "docker-container"

The local node of the builder uses the docker instance on my local dev machine. So there is the driver mismatch.

Can I set this up somehow, or what are my options?

[silverwind]

Facing the same issue. It seems that --driver must match for all nodes in the builder instance, otherwise it fails with mismatched driver, e.g. can't have docker-container and remote drivers on the same builder instance for some reason, maybe it's a bug in buildx parameter validation?

Only solution so far that I've found is to also run a second buildkit locally on unix socket and use the remote driver with that, but I'd prefer if I could just use docker-container for the local build. Here is my current working solution, but I'm not happy having to run this unnecessary buildkitd locally:

docker buildx create \
  --name multi \
  --node linux-x86 \
  --platform linux/amd64 \
  --driver remote \
  unix:///run/buildkit/buildkitd.sock

docker buildx create \
  --name multi \
  --node linux-arm \
  --append \
  --platform linux/arm64/v8 \
  --driver remote \
  tcp://foo:5000