macvlan network cannot resolve container names

[freva]

I'm trying to migrate a docker-compose template I've been running (with docker) for years to podman-compose/podman. In it I run several containers with a macvlan network so that they are reachable from other machines on my network.

The network was defined like this:

networks:
  mynet:
    name: mynet
    driver: macvlan
    driver_opts:
      macvlan_mode: "bridge"
      parent: "enp2s0"
    ipam:
      config:
        - subnet: 192.168.0.0/24
          gateway: 192.168.0.1

but this fails when I try to apply it with podman-compose:

subprocess.CalledProcessError: Command 'podman network create --label io.podman.compose.project=myuser --label com.docker.compose.project=myuser --driver macvlan --opt macvlan_mode=bridge --opt parent=enp2s0 --subnet 192.168.0.0/24 --gateway 192.168.0.1 mynet' returned non-zero exit status 125.

Running that command directly gives the following error message:

Error: unsupported macvlan network option macvlan_mode

If I remove the macvlan_mode: "bridge" option, everything seems to work except that containers nor the host can resolve any of the container names.

I'm guessing the bridge option is what made the resolution possible on docker? Is this possible with podman?

$ podman system info
host:
  arch: amd64
  buildahVersion: 1.33.8
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-1.el9.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: 574ce145d4fde456322f648afc2cb9dc2141ee16'
  cpuUtilization:
    idlePercent: 98.55
    systemPercent: 0.54
    userPercent: 0.92
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: almalinux
    version: "9.4"
  eventLogger: journald
  freeLocks: 2029
  hostname: myhost
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.14.0-427.37.1.el9_4.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 6900838400
  memTotal: 7909052416
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.10.0-3.el9_4.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.10.0
    package: netavark-1.10.3-1.el9.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: crun-1.14.3-1.el9.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.14.3
      commit: 1961d211ba98f532ea52d2e80f4c20359f241a98
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  pasta:
    executable: ""
    package: ""
    version: ""
  remoteSocket:
    exists: false
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.3-1.el9.x86_64
    version: |-
      slirp4netns version 1.2.3
      commit: c22fde291bb35b354e6ca44d13be181c76a0a432
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 0
  swapTotal: 0
  uptime: 0h 26m 9.00s
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /home/myuser/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/myuser/.local/share/containers/storage
  graphRootAllocated: 75094818816
  graphRootUsed: 58250186752
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 5
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/myuser/.local/share/containers/storage/volumes
version:
  APIVersion: 4.9.4-rhel
  Built: 1725363691
  BuiltTime: Tue Sep  3 13:41:31 2024
  GitCommit: ""
  GoVersion: go1.21.11 (Red Hat 1.21.11-1.el9_4)
  Os: linux
  OsArch: linux/amd64
  Version: 4.9.4-rhel

[Luap99]

First we just call the option mode not macvlan_mode.

But more importantly there is no way to have working container name resolution with macvlan networks because our dns server design does not allow for that. The dns server runs on the host namespace and there is no way for container host communication with macvlan so this will never work with podman I would say.
The easiest thing to do would be to add a second (maybe internal) bridge network for the containers that need to talk to each other which would make it work.

[freva]

The easiest thing to do would be to add a second (maybe internal) bridge network for the containers that need to talk to each other which would make it work.

Not sure if I understood you correctly, I tried the following:

version: '2.4'

services:
  c1:
    container_name: c1
    image: alpine
    entrypoint: sleep 9999
    networks:
      podman:
      mynet:
        ipv4_address: 192.168.0.200

  c2:
    container_name: c2
    image: alpine
    entrypoint: sleep 9999
    networks:
      podman:
      mynet:
        ipv4_address: 192.168.0.201

networks:
  podman:
    name: podman
    driver: bridge

  mynet:
    name: mynet
    driver: macvlan
    driver_opts:
      parent: "enp2s0"
    ipam:
      config:
        - subnet: 192.168.0.0/24
          gateway: 192.168.0.1

So both c1 and c2 are in the macvlan network and the podman bridge network, their resolv.conf seems to be that of the host, so they are unable to resolve each other.

My goal is to somehow expose the containers to the wider network outside the host, they don't need to have separate IPs, I could expose specific ports. Initially I tried that with rootless containers, but I guess due to some network isolation, I could not reach [Host IP]:[exposed container port] from other machines on the network :/