Network problem with varnish containers - does not bind 0.0.0.0:6081 - Debian 13

[valicm]

Codebase
Built-in vanilla Drupal or mounted codebase

Describe your issue
Made a setup for Drupal app which goes like this:
Traefik (80,443) => Varnish (6081) => Nginx (80) => PHP-FPM

Upon running containers varnish is non-responsive in terms that Traefik can't reach it. The Varnish container is running. But varnishd is taking 100% cpu (tried even to limit to half cpu), but never crashes.

Running curl -v nginx inside varnish container returns properly drupal site.

After some digging, inside varnish container I got:

tcp        0      0 127.0.0.11:37417        0.0.0.0:*               LISTEN       
udp        0      0 127.0.0.11:54745        0.0.0.0:*       

The funny thing is after hour or two running, the varnish container starts listening properly on 0.0.0.0.

Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 127.0.0.11:37417        0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:6082            0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:6081            0.0.0.0:*               LISTEN      
tcp        0      0 :::6082                 :::*                    LISTEN      
tcp        0      0 :::6081                 :::*                    LISTEN      
udp        0      0 127.0.0.11:54745        0.0.0.0:*                      

So yesterday I am running Github action, deploy code, starts docker containers - site is unresponsive - Varnish
Wake up next morning - site is running normal

The VCL linked is customized VCL based of woodby VCL. But same problem persist with the stock one provided by wodby/varnish

As well did try to create new docker network, nothing changed

Output of docker info

Client: Docker Engine - Community
 Version:    28.3.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.26.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.39.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 10
  Running: 10
  Paused: 0
  Stopped: 0
 Images: 12
 Server Version: 28.3.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.12.41+deb13-cloud-amd64
 Operating System: Debian GNU/Linux 13 (trixie)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.737GiB
 Name: debian-2gb-hel1-1
 ID: 63a7e9e0-44a4-4285-87fc-e0957473cf9e
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

Contents of your compose.yml

services:
  traefik:
    image: traefik:$TRAEFIK_TAG
    container_name: "${PROJECT_NAME}_traefik"
    command:
      - "--providers.docker=true"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
      - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge=true"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web"
      - "--certificatesresolvers.letsencrypt.acme.email=valentino@vallic.com"
      - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
      - "--api.debug=true"
      - "--log.level=ERROR"
      - "--log.filePath=/logs/traefik.log"
      - "--accesslog=true"
      - "--accesslog.filePath=/logs/access.log"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ~/letsencrypt:/letsencrypt
      - ~/logs/:/logs/

  mariadb:
    image: wodby/mariadb:$MARIADB_TAG
    container_name: "${PROJECT_NAME}_mariadb"
    stop_grace_period: 30s
    environment:
      MYSQL_ROOT_PASSWORD: $DB_ROOT_PASSWORD
      MYSQL_DATABASE: $DB_NAME
      MYSQL_USER: $DB_USER
      MYSQL_PASSWORD: $DB_PASSWORD
      MYSQL_TRANSACTION_ISOLATION: READ-COMMITTED
    volumes:
      - ~/database/:/var/lib/mysql

  php:
    image: wodby/drupal-php:$PHP_TAG
    container_name: "${PROJECT_NAME}_php"
    environment:
      PHP_EXTENSIONS_DISABLE: xhprof,spx
      PHP_MAIL_MIXED_LF_AND_CRLF: 'On'
      DB_HOST: $DB_HOST
      DB_PORT: $DB_PORT
      DB_USER: $DB_USER
      DB_PASSWORD: $DB_PASSWORD
      DB_NAME: $DB_NAME
      DB_DRIVER: $DB_DRIVER
      DRUSH_OPTIONS_URI: "https://${PROJECT_BASE_URL}:${PROJECT_PORT}"
      PHP_FPM_CLEAR_ENV: no
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - ./:/var/www/html:cached
      - ~/public:/mnt/files
      - ~/private:/mnt/private

  crond:
    init: true
    image: wodby/drupal-php:$PHP_TAG
    container_name: "${PROJECT_NAME}_crond"
    depends_on:
      - php
    environment:
      CRONTAB: "0 * * * * drush -r /var/www/html/web cron"
    command: sudo -E crond -f -d 0
    volumes:
      - ./:/var/www/html:cached
      - ~/public:/mnt/files
      - ~/private:/mnt/private

  nginx:
    image: wodby/nginx:$NGINX_TAG
    container_name: "${PROJECT_NAME}_nginx"
    depends_on:
      - php
    environment:
      NGINX_STATIC_OPEN_FILE_CACHE: "off"
      NGINX_ERROR_LOG_LEVEL: debug
      NGINX_BACKEND_HOST: php
      NGINX_SET_REAL_IP_FROM: 172.17.0.0/16
      NGINX_HEADERS_CONTENT_SECURITY_POLICY: frame-ancestors 'self'
      NGINX_REAL_IP_HEADER: CF-CONNECTING-IP
      NGINX_SERVER_ROOT: /var/www/html/web
      NGINX_VHOST_PRESET: $NGINX_VHOST_PRESET

  varnish:
    image: wodby/varnish:$VARNISH_TAG
    container_name: "${PROJECT_NAME}_varnish"
    depends_on:
      - nginx
    environment:
      VARNISH_SECRET: iphsApvsmoAfzDkuekj9UHc8fptcYyAC
      VARNISH_BACKEND_HOST: nginx
      VARNISH_BACKEND_PORT: 80
      VARNISH_CONFIG_PRESET: drupal
      VARNISH_ALLOW_UNRESTRICTED_PURGE: 1
      VARNISH_PURGE_EXTERNAL_REQUEST_HEADER: X-Real-IP
      VARNISH_STRIP_PARAMS: utm_[a-z]+|gclid|cx|ie|cof|siteurl|fbclid
      VARNISH_STATIC_FILES: 1
      VARNISHD_VCL_SCRIPT: /var/www/html/.ddev/varnish/default.vcl
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.${PROJECT_NAME}_varnish.rule=Host(`${PROJECT_BASE_URL}`)"
      - "traefik.http.routers.${PROJECT_NAME}_varnish.entrypoints=websecure"
      - "traefik.http.services.${PROJECT_NAME}_varnish.loadbalancer.server.port=6081"
      - "traefik.http.routers.${PROJECT_NAME}_varnish.tls.certresolver=letsencrypt"
    volumes:
      - ./.ddev/varnish/default.vcl:/var/www/html/.ddev/varnish/default.vcl
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 256M
    extra_hosts:
      - "host.docker.internal:host-gateway"

  valkey:
    container_name: "${PROJECT_NAME}_valkey"
    image: wodby/valkey:$VALKEY_TAG

  solr:
    image: wodby/solr:$SOLR_TAG
    container_name: "${PROJECT_NAME}_solr"
    environment:
      SOLR_OPTS: "-Dsolr.config.lib.enabled=true"
      SOLR_MODULES: extraction,langid,ltr,analysis-extras
      ZK_HOST: zookeeper:2181
      SOLR_HEAP: 512m
    depends_on:
      - zookeeper
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 1024M

  zookeeper:
    image: zookeeper:$ZOOKEEPER_TAG
    container_name: "${PROJECT_NAME}_zookeeper"
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zookeeper:2888:3888;2181
      ZOO_4LW_COMMANDS_WHITELIST: mntr, conf, ruok

  rsyslog:
    container_name: "${PROJECT_NAME}_rsyslog"
    image: wodby/rsyslog:$RSYSLOG_TAG

Contents of your .env

### Documentation available at https://wodby.com/docs/stacks/drupal/local
### PROJECT SETTINGS

PROJECT_NAME=my_project
PROJECT_BASE_URL=mydomain.com
PROJECT_PORT=80

DB_NAME=drupal
DB_USER=drupal
DB_PASSWORD=drupal
DB_ROOT_PASSWORD=password
DB_HOST=mariadb
DB_PORT=3306
DB_DRIVER=mysql

NGINX_VHOST_PRESET=drupal11

### --- MARIADB ----
TRAEFIK_TAG=v2.9
MARIADB_TAG=11.4-3.32.3
DRUPAL_TAG=11-4.86.1
PHP_TAG=8.3-dev
NGINX_TAG=1.29-5.44.3
SOLR_TAG=9-5.3.0
VALKEY_TAG=8-1.1.1
NODE_TAG=22-dev-1.53.0
VARNISH_TAG=6.0
OPENSMTPD_TAG=7-1.26.2
RSYSLOG_TAG=latest
ZOOKEEPER_TAG=3.8

Logs output docker compose logs

Paste here

[valicm]

Provisioned new VPS, run entire setup. And Varnish behaves the same. Something has either with my VPS setup or something permission related?!

[valicm]

When it's broken, the process is under root for varnishd

    1     0 root     R     2796   0%   1  25% varnishd -j unix,user=varnish -F -a :6081 -T :6082 -f /var/www/html/.ddev/varnish/default.vcl -S /etc/varnish/sec
   88     0 root     S     1696   0%   1   0% sh
   94    88 root     R     1624   0%   0   0% top

But when it starts to work it is under varnish user

  108     1 varnish  S     105m   3%   1   0% {cache-main} varnishd -j unix,user=varnish -F -a :6081 -T :6082 -f /var/www/html/.ddev/varnish/default.vcl -S /et
    1     0 varnish  S     3432   0%   0   0% varnishd -j unix,user=varnish -F -a :6081 -T :6082 -f /var/www/html/.ddev/varnish/default.vcl -S /etc/varnish/sec
  354     0 root     S     1696   0%   1   0% sh
  362   354 root     R     1624   0%   1   0% top

[valicm]

Now going crazy :-D Started containers as root user, my user, with more hardware, less.

Nothing. What I am missing?

[valicm]

Finally! Problem is in Debian 13 as host OS.
On Debian 12 it works perfect.

cc @csandanov I am not sure do you want to tag it as potential Debian 13 problem, because Debian 13 is out for a week or two, maybe it is easier that I close it