Unable to access agents via Caddy reverse proxy.

[RevenantOne]

🔍 Check for existing issues

• Completed

How is Dozzle deployed?

Agents

📦 Dozzle version

v8.14.10

✅ Command used to run Dozzle

docker compose up -d

Host docker-compose.yml:

services:
  dozzle:
    container_name: dozzle
    image: amir20/dozzle:latest
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /opt/Docker/Dozzle/data:/data
    restart: unless-stopped
    environment:
      - PUID=1001
      - PGID=1001
      - TZ=Europe/London
      - DOZZLE_NO_ANALYTICS=true
      - DOZZLE_AUTH_PROVIDER=simple
      - DOZZLE_HOSTNAME=DozzleHost
      - DOZZLE_ENABLE_ACTIONS=true
      - DOZZLE_REMOTE_AGENT=dozzle.domain.tld
    ports:
      - 3023:8080
    deploy:
      resources:
        limits:
          memory: 2G
        reservations:
          memory: 1G

Agent docker-compose.yml:

services:
  dozzle:
    container_name: dozzle
    image: amir20/dozzle:latest
    command: agent
    healthcheck:
      test: ["CMD", "/dozzle", "healthcheck"]
      interval: 5s
      retries: 5
      start_period: 5s
      start_interval: 5s
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    ports:
      - 7007:7007

Caddyfile:

dozzle.domain.tld {
        reverse_proxy 127.0.0.1:7007

         log {
                output file /var/log/caddy/dozzle.log {
                }
}

🐛 Describe the bug / provide steps to reproduce it

I've set up Dozzle on two seperate networks, one as host and one as an agent. I've followed the agent guide to no avail.

They both work fine as stand alone instances, but if I attempt to set one as an agent and use caddy (v2.6.2) as a reverse proxy it fails. I've tried to troubleshoot with ChatGPT and it suggests it's due to caddy sending a html page rather than the expected JSON. I've searched on line and tried the caddyfile config located here and more than a dozen suggestions from ChatGPT, as well as the flush_interval -1 that's been suggested here.

I can confirm it works if I bypass Caddy and use http://IP-ADDRESS:PORT as the DOZZLE_REMOTE_AGENT value , but I'd rather not go that way for security reasons.

Am I missing a config for Dozzle or is this purely down to Caddy misbehaving?

💻 Environment

Host Environment:

Client:
 Version:    28.2.2
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc.)
    Version:  2.37.1+ds1-0ubuntu2~24.04.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 40
  Running: 32
  Paused: 0
  Stopped: 8
 Images: 65
 Server Version: 28.2.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version:
 runc version:
 init version:
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-88-generic
 Operating System: Ubuntu 24.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 62.69GiB
 Name:  **REDACTED**
 ID: df2e2716-7e2b-4918-9244-08a2f7119559
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

Agent Environment:

Client:
 Version:    27.5.1
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc.)
    Version:  2.37.1+ds1-0ubuntu2~24.04.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 41
  Running: 40
  Paused: 0
  Stopped: 1
 Images: 42
 Server Version: 27.5.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version:
 runc version:
 init version:
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.14.0-36-generic
 Operating System: Ubuntu 24.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 15.54GiB
 Name: **REDACTED**
 ID: ed87f4d7-9b02-45d2-9fb1-c09e2e44c651
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: **REDACTED**
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

📜 Dozzle logs are required for debugging purposes. You may need to enable debug mode. See https://dozzle.dev/guide/debugging.

Host Logs:

{"level":"warn","version":"v8.14.10","error":"rpc error: code = PermissionDenied desc = unexpected HTTP status code received from server: 403 (Forbidden); transport: received unexpected content-type \"text/html\"","endpoint":"dozzle.domain.tld","time":"2025-12-09T23:50:53Z","message":"error fetching host info for agent"}
{"level":"warn","version":"v8.14.10","error":"rpc error: code = PermissionDenied desc = unexpected HTTP status code received from server: 403 (Forbidden); transport: received unexpected content-type \"text/html\"","time":"2025-12-09T23:50:53Z","message":"error while listing containers"}

📸 If applicable, add screenshots to help explain your bug

No response

[amir20]

This is not a bug so moving to discussion.

[amir20]

Dozzle uses gRPC. So a reverse transport wouldn't work as it is designed for HTTP. However, according to Claude, you should be able to use reverse_proxy h2c://localhost:50051 which tells Caddy to use HTTP2. Dozzle does handle it's own SSL handshake so you might want to do:

reverse_proxy h2c://localhost:50051 {
       flush_interval -1
       transport http {
           read_timeout 0
           write_timeout 0
       }
   }

You'll also want to handle flush intervals and timeouts. This is not tested and I am not sure if it will work. I would recommend searching for "reverse proxy with Caddy for gRPC" and seeing if that yields better results.

Let me know how it goes. I can probably do a quick test later this week if I get time.

[RevenantOne]

I've spent a several hours today trying to troubleshoot this issue and as far as I can tell I'm no closer to getting it working. I just can't figure out why Caddy isn't sending the expected response.

I can get agents working locally using internal IPs, e.g. 192.168.x.x.
I can get agents working remotely using my external IP and forwarding port 7007 on my router.

However, getting them working through cloudflare and caddy so I don't need to expose/open any ports just doesn't work.
Cloudflare is set to DNS only and the latest version of my caddy file specifies to use gRPC.

https://dozzle.domain.tld {
    @grpc {
        header Content-Type application/grpc
    }
    reverse_proxy @grpc h2c://192.168.6.5:7007 {
       flush_interval -1
        transport http {
                versions h2c
                read_timeout 0
                write_timeout 0
        }
    }
}

I'm not sure what to try next so any suggestions when you have time would be greatly appreciated.

[amir20]

I figured it out. However, it might not be the solution you want. Caddy is treating the gRPC call as HTTP, so it's trying to decrypt it before sending it to Dozzle. It can't do that because Dozzle's certificates are private. When Caddy sees the SSL request, it fails because it can't perform the proper handshake.

The right solution is to enable layer 4 proxy which just forwards raw TCP.

I was able to test it with:

{
    layer4 {
        localhost:2019 {
            route {
                proxy {
                    upstream localhost:7007
                }
            }
        }
    }
}

Layer 4 is not built in the default binary. You would have to build a custom binary at https://caddyserver.com/download?package=github.com/mholt/caddy-l4 which is probably more complicated.

I don't have a better solution. Perhaps using your own certs is an option and telling Caddy and Dozzle to use those certs would work.

[amir20]

Did this work @RevenantOne ?

[RevenantOne]

I've not had a chance to try this yet but it does look a little too involved for myself. I'll look into it more and report back if I make any progess.