Quadlet gets shut down when an automount volume reaches idle timeout

[abompard]

Issue Description

Please consider this quadlet:

# /etc/containers/systemd/test.container
[Unit]
Description=Container for testing
Wants=network-online.target
After=network-online.target

[Container]
ContainerName=testing
Image=quay.io/fedora/fedora-minimal:latest
Exec=/usr/bin/sleep 1h
HealthCmd=none
Volume=/mnt/nas/Media/All:/media

[Service]
# Restart service when sleep finishes
Restart=always
# Extend Timeout to allow time to pull the image
TimeoutStartSec=180

[Install]
# Start by default on boot
WantedBy=multi-user.target default.target

With the volume defined in /etc/fstab as:

192.168.1.202:/volume1/Media /mnt/nas/Media	nfs4	x-systemd.automount,x-systemd.idle-timeout=300,async	0 0

When I start this quadlet, it gets shut down after 5 minutes, systematically. I'm getting this in the logs:

14:54:16 nuc.example.com systemd[1]: Started test.service - Container for testing.
14:54:16 nuc.example.com test[170559]: 0d70a516d097571fd83c3789b77fe7b4432073dd926fc7a474f9bd1153f17e82
14:59:16 nuc.example.com systemd[1]: Stopping test.service - Container for testing...
14:59:26 nuc.example.com test[170777]: time="2023-12-16T14:59:26+01:00" level=warning msg="StopSignal SIGTERM failed to stop container testing in 10 seconds, resorting to SIGKILL"
14:59:27 nuc.example.com podman[170777]: 2023-12-16 14:59:27.009535825 +0100 CET m=+10.132388112 container died 0d70a516d097571fd83c3789b77fe7b4432073dd926fc7a474f9bd1153f17e82 (image=quay.io/fedora/fedora-minimal:latest, name=testing, version=39, PODMAN_SYSTEMD_UNIT=test.service, license=MIT, name=fedora, vendor=Fedora Project)
14:59:27 nuc.example.com systemd[1]: 0d70a516d097571fd83c3789b77fe7b4432073dd926fc7a474f9bd1153f17e82.timer: Deactivated successfully.
14:59:27 nuc.example.com systemd[1]: Stopped 0d70a516d097571fd83c3789b77fe7b4432073dd926fc7a474f9bd1153f17e82.timer - /usr/bin/podman healthcheck run 0d70a516d097571fd83c3789b77fe7b4432073dd926fc7a474f9bd1153f17e82.
14:59:27 nuc.example.com kernel: cni-podman0: port 1(veth7a1af3bc) entered disabled state
14:59:27 nuc.example.com systemd[1]: run-netns-netns\x2dfe8075ea\x2de3a9\x2d3626\x2d4e57\x2df4cde79b399a.mount: Deactivated successfully.
14:59:27 nuc.example.com systemd[1]: var-lib-containers-storage-overlay\x2dcontainers-0d70a516d097571fd83c3789b77fe7b4432073dd926fc7a474f9bd1153f17e82-userdata-shm.mount: Deactivated successfully.
14:59:27 nuc.example.com podman[170777]: 2023-12-16 14:59:27.366781975 +0100 CET m=+10.489634302 container remove 0d70a516d097571fd83c3789b77fe7b4432073dd926fc7a474f9bd1153f17e82 (image=quay.io/fedora/fedora-minimal:latest, name=testing, license=MIT, name=fedora, vendor=Fedora Project, version=39, PODMAN_SYSTEMD_UNIT=test.service)
14:59:27 nuc.example.com test[170777]: 0d70a516d097571fd83c3789b77fe7b4432073dd926fc7a474f9bd1153f17e82
14:59:27 nuc.example.com systemd[1]: test.service: Main process exited, code=exited, status=137/n/a
14:59:27 nuc.example.com systemd[1]: test.service: Failed with result 'exit-code'.
14:59:27 nuc.example.com systemd[1]: Stopped test.service - Container for testing.
14:59:27 nuc.example.com systemd[1]: test.service: Consumed 1.287s CPU time.
14:59:27 nuc.example.com systemd[1]: Unmounting mnt-nas-Media.mount - /mnt/nas/Media...
14:59:27 nuc.example.com systemd[1]: var-lib-containers-storage-overlay-7798aeea75b2376de27f0e017a3174d75fd28d1a88ec4ee751ac088b0d23244d-merged.mount: Deactivated successfully.
14:59:27 nuc.example.com systemd[1]: var-lib-containers-storage-overlay.mount: Deactivated successfully.
14:59:27 nuc.example.com systemd[1]: mnt-nas-Media.mount: Deactivated successfully.
14:59:27 nuc.example.com systemd[1]: Unmounted mnt-nas-Media.mount - /mnt/nas/Media.

The service receives a stop command exactly 5 minutes after its start, and since /mnt/nas/Media gets unmounted right after the stop and the automount timeout is 5 minutes, I think that's likely the culprit. If I remove the Volume line in the quadlet file, the service doesn't get stopped.

Is this a bug in Podman or is there something I must add to the .container file? (I didn't find anything about that in the docs)

I can easily reproduce it, so I'm happy to help debug it if needed.

Steps to reproduce the issue

Steps to reproduce the issue

1. Setup the provided quadlet
2. Setup the automount mount point
3. systemctl daemon-reload and start the automount mount point and the container
4. wait 5 minutes.

Describe the results you received

The service gets stopped.

Describe the results you expected

The automount mount point gets pinged (or however its timeout should get cancelled) and isn't unmounted, or it gets unmounted but is remounted transparently when the container accesses the volume.

podman info output

This is on Fedora 39.
# podman info
host:
  arch: amd64
  buildahVersion: 1.33.2
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.8-2.fc39.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.8, commit: '
  cpuUtilization:
    idlePercent: 97.77
    systemPercent: 0.79
    userPercent: 1.44
  cpus: 2
  databaseBackend: boltdb
  distribution:
    distribution: fedora
    variant: server
    version: "39"
  eventLogger: journald
  freeLocks: 1995
  hostname: nuc.home.bompard.org
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.6.4-200.fc39.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 178708480
  memTotal: 3533619200
  networkBackend: cni
  networkBackendInfo:
    backend: cni
    dns:
      package: podman-plugins-4.8.1-1.fc39.x86_64
      path: /usr/libexec/cni/dnsname
      version: |-
        CNI dnsname plugin
        version: 1.3.1
        commit: unknown
    package: |-
      containernetworking-plugins-1.3.0-3.fc39.x86_64
      podman-plugins-4.8.1-1.fc39.x86_64
    path: /usr/libexec/cni
  ociRuntime:
    name: crun
    package: crun-1.12-1.fc39.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.12
      commit: ce429cb2e277d001c2179df1ac66a470f00802ae
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20231119.g4f1709d-1.fc39.x86_64
    version: |
      pasta 0^20231119.g4f1709d-1.fc39.x86_64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: false
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-1.fc39.x86_64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 7227301888
  swapTotal: 7227826176
  uptime: 92h 41m 35.00s (Approximately 3.83 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 16095641600
  graphRootUsed: 9517817856
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 3
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.8.1
  Built: 1701777650
  BuiltTime: Tue Dec  5 13:00:50 2023
  GitCommit: ""
  GoVersion: go1.21.4
  Os: linux
  OsArch: linux/amd64
  Version: 4.8.1

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

No

Additional environment details

Fedora 39 freshly upgraded and updated.

Additional information

No response

[rhatdan]

Podman is not shutting down the service every 5 minutes. Podman is not even running, so most likely something else is telling systemd to shut down the service.

[abompard]

Right, this is probably a systemd thing. The service file generated in /var/run/systemd/generator/test.service has RequiresMountsFor=/mnt/nas/Media and I guess that systemd stops all dependant services when the automount point gets unmounted.
Sorry for the bogus report.