Skip to content

invalid content length error when used chunked transfer #4573

@lixiv

Description

@lixiv

Bug report:

when harbor repo use chunked as transfer-encoding, pull image failed with error "invalid content length "

Expected behavior:

pull image success without the error

How to reproduce it:

  1. containerd use http_proxy (mirror also have same error)
  /etc/systemd/system/containerd.service.d/http-proxy.conf
  [Service]
  Environment="HTTP_PROXY=http://127.0.0.1:4001/"
  Environment="HTTPS_PROXY=http://127.0.0.1:4001/"
  1. use curl to confirm chuncked
curl -k -vv -H "Authorization: Basic x" https://xx.com/v2/keda/prometheus/blobs/sha256:a683da7699127231b140f7dbd7d60e9741d51543ba91c656bf3287860a39428f
< Content-Type: text/plain; charset=utf-8
< Connection: close
< Transfer-Encoding: chunked
  1. crictl pull or nerdctl -n k8s.io pull
crictl pull xx.com/keda/prometheus:v3.7.0
  1. the dfdaemon logs show errors:
  2026-01-20T10:34:26.483499170+00:00 ERROR  download started failed: invalid content length
    at dragonfly-client/src/grpc/dfdaemon_download.rs:312
    in download_task with host_id: "ip-cluster-worker-s547t", task_id: "2eaddecde4a9a7f0823350f6b7f16d35973bf0e829cadd874d4a31b28c8d9925", peer_id: "ip-cluster-worker-s547t-a59949bf-7fe0-458d-9ea7-629625091b21", url: "http://xxx.com/v2/keda/prometheus/blobs/sha256:a683da7699127231b140f7dbd7d60e9741d51543ba91c656bf3287860a39428f"

  2026-01-20T10:34:26.483646474+00:00 ERROR  download task failed: status: Internal, message: "invalid content length", details: [], metadata: MetadataMap { headers: {"content-type": "application/grpc", "date": "Tue, 20 Jan 2026 10:34:26 GMT", "content-length": "0"} }
    at dragonfly-client/src/proxy/mod.rs:672
    in proxy_via_dfdaemon
    in http_handler
    in handler with uri: "http://xxx.com/v2/keda/prometheus/blobs/sha256:a683da7699127231b140f7dbd7d60e9741d51543ba91c656bf3287860a39428f", method: "GET"
  1. related issues: ERROR download task failed: status: Internal, messagge: "invalid content length" #4366 (comment)
  2. cherry-pick this pr feat: add Range header to ensure Content-Length is returned in response headers client#1263 ,not work

Environment:

  • Dragonfly version: dfdaemon: v0.2.36; manager: v2.2.2

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions