Podman in rootless Nixos #25813

yaroslavkasatikov · 2025-04-04T16:45:59Z

yaroslavkasatikov
Apr 4, 2025

Issue Description

Podman can not pull image in rootless mode.

I got this error when buildah works fine:

$ podman build .           
STEP 1/2: FROM ubuntu:latest
Resolved "ubuntu" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/ubuntu:latest...
Getting image source signatures
Copying blob 5a7813e071bf done   | 
ERRO[0002] While applying layer: ApplyLayer stdout:  stderr: potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/gshadow): Check /etc/subuid and /etc/subgid if configured loca
Copying blob 5a7813e071bf done   | 
Error: creating build container: copying system image from manifest list: writing blob: adding layer with blob "sha256:5a7813e071bfadf18aaa6ca8318be4824a9b6297b3240f2cc84c1db6f4113040"/""/"sha256:4b7c01ed0534d4f9be9cf97d068da1598c6c20b26cb6134fad066defdb6d541d": ApplyLayer stdout:  stderr: potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/gshadow): Check /etc/subuid and /etc/subgid if configured locally and run "podman system migrate": lchown /etc/gshadow: invalid argument exit status 1

# 1000910000 @ workspace680c634c8aeb406d-74cb56cc58-wbclx in ~/platform-base-toolchain/example-for-build/docker/docker on git:unprivileged-build x [16:10:04] C:1
# 1000910000 @ workspace680c634c8aeb406d-74cb56cc58-wbclx in ~/platform-base-toolchain/example-for-build/docker/docker on git:unprivileged-build x [16:11:29] C:130
$ podman pull alpine
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob f18232174bc9 done   | 
ERRO[0001] While applying layer: ApplyLayer stdout:  stderr: potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/shadow): Check /etc/subuid and /etc/subgid if configured local
Copying blob f18232174bc9 done   | 
Error: copying system image from manifest list: writing blob: adding layer with blob "sha256:f18232174bc91741fdf3da96d85011092101a032a93a388b79e99e69c2d5c870"/""/"sha256:08000c18d16dadf9553d747a58cf44023423a9ab010aab96cf263d2216b8b350": ApplyLayer stdout:  stderr: potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/shadow): Check /etc/subuid and /etc/subgid if configured locally and run "podman system migrate": lchown /etc/shadow: invalid argument exit status 1

I tried many steps from different AI recomendation and stackoverflow-like.

export _CONTAINERS_USERNS_CONFIGURED=“1”

these lead to

$ podman pull alpine                                                                                                                                      
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1112f7a]

goroutine 1 [running]:
github.com/containers/common/libimage.(*Runtime).Pull(0x0, {0x2169420, 0x3008a60}, {0x7ffc3cba2d5a, 0x6}, 0x0, 0xc00061b860)
        github.com/containers/[email protected]/libimage/pull.go:57 +0x11a
github.com/containers/podman/v5/pkg/domain/infra/abi.(*ImageEngine).Pull(0x0?, {0x2169420?, 0x3008a60?}, {0x7ffc3cba2d5a?, _}, {0x0, {0x0, 0x0}, {0x0, 0x0}, ...})
        github.com/containers/podman/v5/pkg/domain/infra/abi/images.go:303 +0x2aa
github.com/containers/podman/v5/cmd/podman/images.imagePull(0x2ee7ee0, {0xc00012b9d0, 0x1, 0x1?})
        github.com/containers/podman/v5/cmd/podman/images/pull.go:211 +0x63c
github.com/spf13/cobra.(*Command).execute(0x2ee7ee0, {0xc00011a6b0, 0x1, 0x1})
        github.com/spf13/[email protected]/command.go:985 +0xaaa
github.com/spf13/cobra.(*Command).ExecuteC(0x2ee1480)
        github.com/spf13/[email protected]/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/[email protected]/command.go:1041
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        github.com/spf13/[email protected]/command.go:1034
main.Execute()
        github.com/containers/podman/v5/cmd/podman/root.go:119 +0x57
main.main()
        github.com/containers/podman/v5/cmd/podman/main.go:62 +0x4b2

added userns = “auto” into storage.conf (no results)

Tryied different images for pull (I understood that pull is blocked here)

Playing with isolation types

tryied to addd --userns=keep-id (podman/docs/tutorials/rootless_tutorial.md at main · containers/podman ·
GitHub)

Steps to reproduce the issue

Steps to reproduce the issue
run podman pull alpine

Preparation:
it's unprivileged container.

I have configured dockerfile:

FROM nixos/nix:2.26.3
# We install main packages which is necessary for managing with rootless. The nixpkgs.shadow is the package with newuidmap/newgidmap, libcap is the package with setcap/getcap for capability managing. 
ADD containers /etc/containers
RUN ls -la /etc/containers
RUN nix-env -iA nixpkgs.shadow  nixpkgs.libcap nixpkgs.gawk
# Next two commands set capability for two shadow binaries. It's the most important part - the reason of image rebuilding
RUN setcap cap_setuid+ep $(ls -la /root/.nix-profile/bin/newuidmap | awk -F"->" '{print $2}')
RUN setcap cap_setgid+ep $(ls -la /root/.nix-profile/bin/newgidmap | awk -F"->" '{print $2}')
# Copying these binaries into dedicated folder b/c installed with nix-config cm will be not marked as correct cap. We need SETFCAP capability for changing it in running pod. 
RUN mkdir /shadowbin/ && cp -av $(ls -la /root/.nix-profile/bin/newuidmap | awk -F"->" '{print $2}') /shadowbin/ && cp -av $(ls -la /root/.nix-profile/bin/newgidmap | awk -F"->" '{print $2}') /shadowbin/
# Removing symlink which are to new*idmap without caps
RUN rm -rfv /root/.nix-profile/bin/new*idmap  && rm -rfv /nix/var/nix/profiles/default/sbin/new*idmap
# Creating subgid, subuid, passwd and set g=u. This help us managing files when pod is running

RUN touch /etc/subgid /etc/subuid  && \
    chmod g=u /etc/subgid /etc/subuid /etc/passwd  && \
    echo user:10000:65536 > /etc/subuid  && \
    echo user:10000:65536 > /etc/subgid

Run it with storage driver VFS and isolation type "chroot"

Buildah works fine:

$ podman pull alpine                 
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob f18232174bc9 done   | 
ERRO[0001] While applying layer: ApplyLayer stdout:  stderr: potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/shadow): Check /etc/subuid and /etc/subgid if configured local
Copying blob f18232174bc9 done   | 
Error: copying system image from manifest list: writing blob: adding layer with blob "sha256:f18232174bc91741fdf3da96d85011092101a032a93a388b79e99e69c2d5c870"/""/"sha256:08000c18d16dadf9553d747a58cf44023423a9ab010aab96cf263d2216b8b350": ApplyLayer stdout:  stderr: potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/shadow): Check /etc/subuid and /etc/subgid if configured locally and run "podman system migrate": lchown /etc/shadow: invalid argument exit status 1

# user @ workspace680c634c8aeb406d-74cb56cc58-wbclx in ~/platform-base-toolchain/example-for-build/docker/docker on git:unprivileged-build x [16:37:39] C:125
$ buildah pull alpine
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob f18232174bc9 done   | 
Copying config aded1e1a5b done   | 
Writing manifest to image destination
aded1e1a5b3705116fa0a92ba074a5e0b0031647d9c315983ccba2ee5428ec8b

# user @ workspace680c634c8aeb406d-74cb56cc58-wbclx in ~/platform-base-toolchain/example-for-build/docker/docker on git:unprivileged-build x [16:41:26] 
$

setuid and setgit configured correct:

bash-5.2$ whoami
user
bash-5.2$ grep ^user /etc/passwd
user:x:1000910000:0:1000910000 user:/home/nix:/sbin/nologin
bash-5.2$ cat /etc/subuid 
user:10000:65536
bash-5.2$ cat /etc/subgid 
user:10000:65536

so don't know what is the issue :(

Describe the results you received

$ podman pull alpine
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob f18232174bc9 done |
ERRO[0001] While applying layer: ApplyLayer stdout: stderr: potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/shadow): Check /etc/subuid and /etc/subgid if configured local
Copying blob f18232174bc9 done |
Error: copying system image from manifest list: writing blob: adding layer with blob "sha256:f18232174bc91741fdf3da96d85011092101a032a93a388b79e99e69c2d5c870"/""/"sha256:08000c18d16dadf9553d747a58cf44023423a9ab010aab96cf263d2216b8b350": ApplyLayer stdout: stderr: potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/shadow): Check /etc/subuid and /etc/subgid if configured locally and run "podman system migrate": lchown /etc/shadow: invalid argument exit status 1

Describe the results you expected

Image should be pulled

podman info output

bash-5.2$ podman info
host:
  arch: amd64
  buildahVersion: 1.39.2
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: Unknown
    path: /nix/store/hrxgvkdr6i3wvqp6m1kfqjrq3h4jylkx-podman-helper-binary-wrapper/bin/conmon
    version: 'conmon version 2.1.13, commit: '
  cpuUtilization:
    idlePercent: 91.66
    systemPercent: 2.28
    userPercent: 6.06
  cpus: 16
  databaseBackend: sqlite
  distribution:
    distribution: unknown
    version: unknown
  eventLogger: file
  freeLocks: 2048
  hostname: workspace680c634c8aeb406d-74cb56cc58-wbclx
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 0
      size: 1
    uidmap:
    - container_id: 0
      host_id: 1000910000
      size: 1
  kernel: 6.5.5-200.fc38.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 1853849600
  memTotal: 32845611008
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: Unknown
      path: /nix/store/grsjjqaxzy9dgjw2zj55jarc94kdzdhz-podman-5.4.1/libexec/podman/aardvark-dns
      version: aardvark-dns 1.14.0
    package: Unknown
    path: /nix/store/grsjjqaxzy9dgjw2zj55jarc94kdzdhz-podman-5.4.1/libexec/podman/netavark
    version: netavark 1.14.1
  ociRuntime:
    name: crun
    package: Unknown
    path: /nix/store/hrxgvkdr6i3wvqp6m1kfqjrq3h4jylkx-podman-helper-binary-wrapper/bin/crun
    version: |-
      crun version 1.21
      commit: 1.21
      rundir: /tmp/storage-run-1000910000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  pasta:
    executable: /nix/store/grsjjqaxzy9dgjw2zj55jarc94kdzdhz-podman-5.4.1/libexec/podman/pasta
    package: Unknown
    version: ""
  remoteSocket:
    exists: true
    path: /tmp/storage-run-1000910000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: ""
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 2503h 53m 6.00s (Approximately 104.29 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  configFile: /home/nix/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: vfs
  graphOptions: {}
  graphRoot: /tmp/containers/storage
  graphRootAllocated: 245103374336
  graphRootUsed: 199493500928
  graphStatus: {}
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /tmp/storage-run-1000910000/containers
  transientStore: false
  volumePath: /tmp/containers/storage/volumes
version:
  APIVersion: 5.4.1
  Built: 315532800
  BuiltTime: Tue Jan  1 00:00:00 1980
  GitCommit: ""
  GoVersion: go1.24.1
  Os: linux
  OsArch: linux/amd64
  Version: 5.4.1

Podman in a container

Yes

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

giuseppe · 2025-04-07T07:03:06Z

giuseppe
Apr 7, 2025
Maintainer

it seems like you've not configured any additional UID/GID for the rootless user:

  idMappings:
    gidmap:
    - container_id: 0
      host_id: 0
      size: 1
    uidmap:
    - container_id: 0
      host_id: 1000910000
      size: 1

so the GID 42 is not mapped inside the user namespace. You need to change the configuration for your system and allocate more IDs to your user

4 replies

yaroslavkasatikov Apr 7, 2025
Author

Thank you for reply, but could you please help me with point where I can configure it?
I don't have id 42 in system. Also I have shared my /etc/subuid, /etc/subgid and /etc/passwd.
Can not find where to fix it.
Thanks in advance

giuseppe Apr 7, 2025
Maintainer

I don't know where ID 1000910000 is coming from.

You need to set something like:

1000910000:100000:65536

in your /etc/subuid and /etc/subgid files

yaroslavkasatikov Apr 7, 2025
Author

Thank you for your reply and sorry for confusing you..

1000910000 is UID/GID whih Openshift set for pod runtime

uid=1000910000(user) gid=0(root) groups=0(root),1000910000(user)

I have in /etc/passwd this line:

user:x:1000910000:1000910000:1000910000 user:/home/nix:/sbin/nologin

I have this in /etc/subuid and /etc/subgid

bash-5.2$ cat /etc/subuid
user:100000:65536

bash-5.2$ cat /etc/subgid
user:100000:65536

I trie to change it to 1000910000:100000:65536 but nothing changed. As I understood the first section in sub*id files is username/group name, so it is 'user' for both..

Is there the error?

giuseppe Apr 7, 2025
Maintainer

please refer to https://access.redhat.com/solutions/7045022 on how to run Podman on Openshift. There are some limitations unless you use user namespaces in Openshift

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Podman in rootless Nixos #25813

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment 4 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Podman in rootless Nixos #25813

Uh oh!

Uh oh!

yaroslavkasatikov Apr 4, 2025

Issue Description

Steps to reproduce the issue

Describe the results you received

Describe the results you expected

podman info output

Podman in a container

Privileged Or Rootless

Upstream Latest Release

Additional environment details

Additional information

Replies: 1 comment · 4 replies

Uh oh!

giuseppe Apr 7, 2025 Maintainer

Uh oh!

yaroslavkasatikov Apr 7, 2025 Author

Uh oh!

giuseppe Apr 7, 2025 Maintainer

Uh oh!

yaroslavkasatikov Apr 7, 2025 Author

Uh oh!

giuseppe Apr 7, 2025 Maintainer

yaroslavkasatikov
Apr 4, 2025

Replies: 1 comment 4 replies

giuseppe
Apr 7, 2025
Maintainer

yaroslavkasatikov Apr 7, 2025
Author

giuseppe Apr 7, 2025
Maintainer

yaroslavkasatikov Apr 7, 2025
Author

giuseppe Apr 7, 2025
Maintainer