Docker container in host-mode networking with domain

[Axodouble]

I am trying to figure out how to get a docker container to run with host-mode networking but also have a domain attached to it, currently struggling with the current config:

services:
  wg-easy:
    image: ghcr.io/wg-easy/wg-easy
    network_mode: "host"
    environment:
      - LANG=en
      - WG_HOST=${HOSTNAME}
      - PASSWORD_HASH=${WG_HASH}
      - PORT=51821
      - WG_PORT=51820
    volumes:
      - /server/wg-easy:/etc/wireguard
    ports:
      - "51821:51821/tcp"
      - "51821:51821/udp"
      - "51820:51820/udp"
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    restart: unless-stopped
    labels:
      - traefik.http.routers.wg-easy-web.rule=Host(`wg.example.com`)
      - traefik.http.routers.wg-easy-web.entrypoints=web
      - traefik.http.services.wg-easy-web.loadbalancer.server.port=51821
      - traefik.http.routers.wg-easy-web.service=wg-easy-web
      - traefik.http.routers.wg-easy-web.middlewares=redirect-to-https@file
      - traefik.http.routers.wg-easy-websecure.rule=Host(`wg.example.com`)
      - traefik.http.routers.wg-easy-websecure.entrypoints=websecure
      - traefik.http.services.wg-easy-websecure.loadbalancer.server.port=51821
      - traefik.http.routers.wg-easy-websecure.service=wg-easy-websecure
      - traefik.http.routers.wg-easy-websecure.tls.certresolver=letsencrypt
      - traefik.enable=true

But it appears that Traefik is unable to reach the service itself, as I get Bad Gateway when visiting the domain.
Any ideas?