Why do some of my containers require podman rm --storage when I reboot?

[poperigby]

I have a collection of containers that are orchestrated with .kube files. Seemingly at random, a service will fail to start on reboot. When I go to start it manually with podman kube play service.yaml, it will say something like this:

Error: creating container storage: the container name "authelia-pod-authelia" is already in use by b8bf71cb1c5029f2dfee4f48e2be50c4bdeff7af9c99d5cd09d48b25eb8cbe7d. You have to remove that container to be able to reuse that name: that name is already in use

I then have to run something like this to fix it:

podman rm --storage b8bf71cb1c5029f2dfee4f48e2be50c4bdeff7af9c99d5cd09d48b25eb8cbe7d

It just recently (about a month ago), started doing this, so I'm not sure what happened.

Here's one of my .kube files:

[Install]
WantedBy=default.target

[Unit]
Requires=caddy.service
After=caddy.service

[Service]
TimeoutSec=900

[Kube]
Yaml=services/authelia/service.yaml
Network=reverse-proxy.network

Here's one of my Kubernetes style YAML files:

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: authelia-pod
  name: authelia-pod
spec:
  hostname: authelia-pod
  securityContext:
    seLinuxOptions:
      type: spc_t
  containers:
  - name: authelia
    image: ghcr.io/authelia/authelia:latest
    imagePullPolicy: Never
    #args:
    #- --config
    #- /config/configuration.yml
    env:
    - name: AUTHELIA_JWT_SECRET
      value: REDACTED
    - name: AUTHELIA_STORAGE_ENCRYPTION_KEY
      value: REDACTED
    - name: AUTHELIA_NOTIFIER_SMTP_PASSWORD
      value: REDACTED
    - name: AUTHELIA_IDENTITY_PROVIDERS_OIDC_HMAC_SECRET
      value: REDACTED
    - name: AUTHELIA_IDENTITY_PROVIDERS_OIDC_ISSUER_PRIVATE_KEY
      value: REDACTED
    - name: AUTHELIA_SESSION_SECRET
      value: REDACTED
    volumeMounts:
    - mountPath: /config
      name: authelia-config
  - name: redis
    image: docker.io/library/redis:latest
    #args:
    #- redis-server
    volumeMounts:
    - mountPath: /data
      name: redis-data
  volumes:
  - name: redis-data
    persistentVolumeClaim:
      claimName: redis
  - hostPath:
      path: /home/cassidy/.config/containers/systemd/services/authelia/authelia
      type: Directory
    name: authelia-config

[rhatdan]

@mheon PTAL

[mheon]

@vrothberg I assume some sort of removal of the pod and associated containers from the Kube YAML should be happening on unit start?

[mheon]

(Because I'd expect that to catch this kind of thing)

[vrothberg]

The generated systemd units have a ExecStart=/usr/bin/podman kube play --replace (note the --replace). So to me it looks like the unit got shot down on stop (i.e., ExecStopPost=/usr/bin/podman kube down).

Without a reproducer we cannot be sure but I think it makes sense to have kube down and --replace remove "external" storage containers to account for such cases.

[vrothberg]

I created issue #20025 to track work on it.

[poperigby]

Thank you!