Registry credential does not work #6409
Description
Contributing guidelines and issue reporting guide
- I've read the contributing guidelines and wholeheartedly agree. I've also read the issue reporting guide.
Well-formed report checklist
- I have found a bug that the documentation does not mention anything about my problem
- I have found a bug that there are no open or closed issues that are related to my problem
- I have provided version/information about my environment and done my best to provide a reproducer
Description of bug
Bug description
Pull failed even registry credential is provided, the same credential works with nerdctl on the buildkitd server. It seems the credentials was not included in the request?
Reproduction
# buildctl --addr tcp://192.168.64.2:1234 build --frontend dockerfile.v0 --local context=. --local dockerfile=. \
--output type=image,name=crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/cloud-chain:1.0
[+] Building 0.4s (3/3) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 193B 0.0s
=> ERROR [internal] load metadata for crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim 0.2s
=> [auth] ttc0419/nginx:pull token for crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com 0.0s
------
> [internal] load metadata for crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim:
------
Dockerfile:1
--------------------
1 | >>> FROM crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim
--------------------
error: failed to solve: crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim: failed to resolve source metadata for crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
time="2025-12-11T14:31:17+08:00" level=fatal msg="image \"crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/cloud-chain:1.0\": not found"
Server buildkitd config:
[worker.oci]
enabled = false
[worker.containerd]
enabled = true
namespace = "k8s.io"
[grpc]
address = [ "tcp://0.0.0.0:1234" ]
buildctl config.json
{
"auths": {
"crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com": {
"auth": "xxx"
}
}
}buildkitd log:
DEBU[2025-12-11T14:51:34+08:00] resolving span="resolving crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim" spanID=982c2385695dbae4 traceID=ee69ca321376d3c067d888fae8dfdcc9
DEBU[2025-12-11T14:51:34+08:00] do request request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=HEAD span="resolving crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim" spanID=982c2385695dbae4 traceID=ee69ca321376d3c067d888fae8dfdcc9
DEBU[2025-12-11T14:51:34+08:00] fetch response received response.header.connection=keep-alive response.header.content-length=156 response.header.content-type="application/json; charset=utf-8" response.header.date="Thu, 11 Dec 2025 06:51:34 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.www-authenticate="Bearer realm=\"https://dockerauth.cn-hangzhou.aliyuncs.com/auth\",service=\"registry.aliyuncs.com:cn-shanghai:26842\",scope=\"repository:ttc0419/nginx:pull\"" response.status="401 Unauthorized" span="resolving crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim" spanID=982c2385695dbae4 traceID=ee69ca321376d3c067d888fae8dfdcc9
DEBU[2025-12-11T14:51:34+08:00] Unauthorized header="Bearer realm=\"https://dockerauth.cn-hangzhou.aliyuncs.com/auth\",service=\"registry.aliyuncs.com:cn-shanghai:26842\",scope=\"repository:ttc0419/nginx:pull\"" span="resolving crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim" spanID=982c2385695dbae4 traceID=ee69ca321376d3c067d888fae8dfdcc9
DEBU[2025-12-11T14:51:34+08:00] do request request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=HEAD span="resolving crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim" spanID=982c2385695dbae4 traceID=ee69ca321376d3c067d888fae8dfdcc9
DEBU[2025-12-11T14:51:35+08:00] fetch response received response.header.connection=keep-alive response.header.content-length=156 response.header.content-type="application/json; charset=utf-8" response.header.date="Thu, 11 Dec 2025 06:51:35 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.www-authenticate="Bearer realm=\"https://dockerauth.cn-hangzhou.aliyuncs.com/auth\",service=\"registry.aliyuncs.com:cn-shanghai:26842\",scope=\"repository:ttc0419/nginx:pull\",error=\"insufficient_scope\"" response.status="401 Unauthorized" span="resolving crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim" spanID=982c2385695dbae4 traceID=ee69ca321376d3c067d888fae8dfdcc9
DEBU[2025-12-11T14:51:35+08:00] Unauthorized header="Bearer realm=\"https://dockerauth.cn-hangzhou.aliyuncs.com/auth\",service=\"registry.aliyuncs.com:cn-shanghai:26842\",scope=\"repository:ttc0419/nginx:pull\",error=\"insufficient_scope\"" span="resolving crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim" spanID=982c2385695dbae4 traceID=ee69ca321376d3c067d888fae8dfdcc9
INFO[2025-12-11T14:51:35+08:00] fetch failed error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" span="resolving crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim" spanID=982c2385695dbae4 traceID=ee69ca321376d3c067d888fae8dfdcc9
ERRO[2025-12-11T14:51:35+08:00] /moby.buildkit.v1.frontend.LLBBridge/Solve returned error: rpc error: code = Unknown desc = crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim: failed to resolve source metadata for crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed spanID=c17e7e98badd2ee0 traceID=ee69ca321376d3c067d888fae8dfdcc9
crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim: failed to resolve source metadata for crpi-9ogjbhecfaihmimu.cn-shanghai.personal.cr.aliyuncs.com/ttc0419/nginx:mainline-alpine-slim: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
2281 0.25.1 buildkitd --trace
github.com/moby/buildkit/util/resolver.invalidAuthorization
/build/buildkit/src/buildkit/util/resolver/authorizer.go:433
github.com/moby/buildkit/util/resolver.(*dockerAuthorizer).AddResponses
/build/buildkit/src/buildkit/util/resolver/authorizer.go:160
github.com/containerd/containerd/v2/core/remotes/docker.(*request).retryRequest
/build/buildkit/src/pkg/mod/github.com/containerd/containerd/[email protected]/core/remotes/docker/resolver.go:740
github.com/containerd/containerd/v2/core/remotes/docker.(*request).doWithRetriesInner
/build/buildkit/src/pkg/mod/github.com/containerd/containerd/[email protected]/core/remotes/docker/resolver.go:719
github.com/containerd/containerd/v2/core/remotes/docker.(*request).doWithRetriesInner
/build/buildkit/src/pkg/mod/github.com/containerd/containerd/[email protected]/core/remotes/docker/resolver.go:726
github.com/containerd/containerd/v2/core/remotes/docker.(*request).doWithRetries
/build/buildkit/src/pkg/mod/github.com/containerd/containerd/[email protected]/core/remotes/docker/resolver.go:694
github.com/containerd/containerd/v2/core/remotes/docker.(*dockerResolver).Resolve
/build/buildkit/src/pkg/mod/github.com/containerd/containerd/[email protected]/core/remotes/docker/resolver.go:308
github.com/moby/buildkit/util/resolver.(*Resolver).Resolve
/build/buildkit/src/buildkit/util/resolver/pool.go:234
github.com/moby/buildkit/util/imageutil.Config
/build/buildkit/src/buildkit/util/imageutil/config.go:107
github.com/moby/buildkit/source/containerimage.(*Source).ResolveImageConfig.func2
/build/buildkit/src/buildkit/source/containerimage/source.go:193
github.com/moby/buildkit/util/flightcontrol.(*call[...]).run
/build/buildkit/src/buildkit/util/flightcontrol/flightcontrol.go:122
sync.(*Once).doSlow
/usr/lib/go/src/sync/once.go:78
sync.(*Once).Do
/usr/lib/go/src/sync/once.go:69
runtime.goexit
/usr/lib/go/src/runtime/asm_arm64.s:1268