[quadlet] [rootless] public network needed

[podhorsky-ksj]

Hi, I have an issue and I cannot find out solution.

I have quadlet file containers, which I need to connect each other and into internet.
It would be easier use pod for this, but I cannot use it, because I need to use UserNS=keep-id:uid=x,gid=y, where x,y is different for every container, and pod don't allow this.

So, I created network:

[Unit]
Description=immich
Requires=network-online.service
After=network-online.service

[Network]
NetworkName=immich
Driver=ipvlan
Subnet=10.0.0.0/8
Internal=false
Gateway=10.0.0.1

[Install]
WantedBy=default.target

postgres

[Unit]
Description=Immich
Requires=immich-network.service
After=immich-network.service
StartLimitIntervalSec=90
StartLimitBurst=2

[Container]
ContainerName=immich_postgres
AutoUpdate=registry
Environment=POSTGRES_INITDB_ARGS=--data-checksums POSTGRES_DB=immich POSTGRES_PASSWORD=postgres POSTGRES_USER=postgres
Exec=postgres -c 'shared_preload_libraries=vectors.so' -c 'search_path="$user", public, vectors' -c 'logging_collector=on' -c 'max_wal_size=2GB' -c 'shared_buffers=512MB' -c 'wal_compression=on'
Image=docker.io/tensorchord/pgvecto-rs:pg16-v0.2.1
Volume=%h/postgres:/var/lib/postgresql/data:Z
#HealthCmd=pg_isready --dbname='immich' || exit 1; Chksum="$$(psql --dbname='immich' --username='postgres' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
HealthInterval=5m
HealthStartupInterval=30m
HealthStartPeriod=5m
IP=10.0.0.3
UserNS=keep-id:uid=999,gid=999
Network=immich
#SecurityLabelType=immich-postgres.process

[Service]
Restart=always
RestartSec=5

[Install]
WantedBy=default.target

and main server

[Unit]
Description=Immich
Requires=immich_postgres.service immich_valkey.service
After=immich_postgres.service immich_valkey.service
StartLimitIntervalSec=90
StartLimitBurst=2

[Container]
ContainerName=immich_server
AutoUpdate=registry
Environment=DB_HOSTNAME=10.0.0.3 REDIS_HOSTNAME=10.0.0.4 DB_USERNAME=postgres DB_DATABASE_NAME=immich DB_PASSWORD=postgres
Image=ghcr.io/immich-app/immich-server:release
Volume=%h/library:/usr/src/app/upload:Z
AddDevice=/dev/dri/renderD128:/dev/dri/renderD128
PodmanArgs=--group-add=keep-groups
Volume=/etc/localtime:/etc/localtime:ro
PublishPort=3001:3001
UserNS=keep-id:uid=1000,gid=1000
Network=immich
IP=10.0.0.2
#SecurityLabelType=immich-server.process

[Service]
Restart=always
RestartSec=5

[Install]
WantedBy=default.target

This is not whole structure, but it can be used as example. I need predictable ip addresses, because server needs to be connected to postgres. It cannot be just any address.

The network is working and containers can communicate each other. But I found out one container needs internet access and I don't know how to provide it. I think maybe network Driver bridge would do the thing, But I remember I selected this, because bridge returns the subnet is already used, but I haven't seen where.

default podman network is:

[
     {
          "name": "podman",
          "id": "2f259bab93aaaaa2542ba43ef33eb990d0999ee1b9924b557b7be53c0b7a1bb9",
          "driver": "bridge",
          "network_interface": "podman0",
          "created": "2024-07-17T12:22:08.734802928+02:00",
          "subnets": [
               {
                    "subnet": "10.88.0.0/16",
                    "gateway": "10.88.0.1"
               }
          ],
          "ipv6_enabled": false,
          "internal": false,
          "dns_enabled": false,
          "ipam_options": {
               "driver": "host-local"
          },
          "containers": {}
     }
]

And It doesn't seems to have address collision with my network.

How can I solve this issue?

[Luap99]

The bridge driver should have internet connectivity by default unless the network is internal (created with --internal)
Looks like you created the network with ipvlan, not sure why you did this but if you remove it and use bridge it will just work.
Ipvlan/macvlan are not really useful in rootless mode as the parent is in the rootless netns and not the host (#22943 (comment))

[podhorsky-ksj]

I already explained it in this ticket, why I have done it like that.
When I switch to bridge with in network quadlet, it will failed with:

Jul 17 13:26:15 server-ksj immich-network[150149]: Error: subnet 10.0.0.0/8 is already used on the host or by another config
Jul 17 13:26:15 server-ksj systemd[1398]: immich-network.service: Main process exited, code=exited, status=125/n/a

And I haven't found out, where exactly it is used.

[Luap99]

it is used by the default podman config because podman uses 10.0.88.0/16 so it overlaps with your given subnet

Why are you trying to use a /8 network? Change this to /24 and it will work or use a different private range like 192.168...

[podhorsky-ksj]

Yes, 192.168.1.0/24 seems to provide connection and access to internet too. Thanks.