Cannot bind local volume on macOS

[pheki]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

It fails to bind local directory on MacOS.

Steps to reproduce the issue:

1. On macOS, run podman machine init and podman machine start

2. Create directory (~/git/podman-test on my example)

3. Try to run podman binding that directory as a volume:

podman run -it --rm --name hewwo -v ~/git/podman-test:/usr/src/hewwo -w /usr/src/hewwo python:3 python hewwo.py

Describe the results you received:

$ podman run -it --rm --name hewwo -v ~/git/podman-test:/usr/src/hewwo -w /usr/src/hewwo python:3 python hewwo.py 
Error: statfs /Users/bilko/git/podman-test: no such file or directory

Describe the results you expected:

Binding works, such as with docker:

$ docker run -it --rm --name hewwo -v ~/git/podman-test:/usr/src/hewwo -w /usr/src/hewwo python:3 python hewwo.py 
Hewwo wowld

Additional information you deem important (e.g. issue happens only occasionally):

I have permissions to access this directory (my user is bilko):

$ ls -al ~/git/podman-test/
total 8
drwxr-xr-x   3 bilko  staff   96 Apr 20 23:40 .
drwxr-xr-x@ 22 bilko  staff  704 Apr 20 02:07 ..
-rw-r--r--   1 bilko  staff   21 Apr 20 22:43 hewwo.py

Output of podman version:

$ podman version
Client:       Podman Engine
Version:      4.0.3
API Version:  4.0.3
Go Version:   go1.18
Built:        Fri Apr  1 12:28:59 2022
OS/Arch:      darwin/amd64

Server:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.16.14
Built:        Thu Mar  3 11:56:56 2022
OS/Arch:      linux/amd64

Output of podman info --debug:

$ podman info --debug
host:
  arch: amd64
  buildahVersion: 1.24.1
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: '
  cpus: 1
  distribution:
    distribution: fedora
    variant: coreos
    version: "35"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 501
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 5.15.18-200.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 593727488
  memTotal: 2061381632
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.4.2-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.2
      commit: f6fbc8f840df1a414f31a60953ae514fa497c748
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/501/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 58m 35.12s
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/user/501/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 4.0.2
  Built: 1646319416
  BuiltTime: Thu Mar  3 11:56:56 2022
  GitCommit: ""
  GoVersion: go1.16.14
  OsArch: linux/amd64
  Version: 4.0.2

Package info (e.g. output of rpm -q podman or apt list podman):

$ brew info podman
podman: stable 4.0.3 (bottled), HEAD
Tool for managing OCI containers and pods
https://podman.io/
/usr/local/Cellar/podman/4.0.3 (172 files, 47.6MB) *
  Poured from bottle on 2022-04-19 at 12:07:28
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb
License: Apache-2.0
==> Dependencies
Build: go ✘, go-md2man ✘
Required: qemu ✔
==> Options
--HEAD
	Install HEAD version
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

Physical Intel Mac Mini running macOS Monterey 12.3.1.

qemu version:

$ brew info qemu
qemu: stable 6.2.0 (bottled), HEAD
Emulator for x86 and PowerPC
https://www.qemu.org/
/usr/local/Cellar/qemu/6.2.0_1 (162 files, 558.9MB) *
  Poured from bottle on 2022-04-19 at 12:07:21
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/qemu.rb
License: GPL-2.0-only
==> Dependencies
Build: libtool ✔, meson ✘, ninja ✘, pkg-config ✔
Required: glib ✔, gnutls ✔, jpeg ✔, libpng ✔, libslirp ✔, libssh ✔, libusb ✔, lzo ✔, ncurses ✔, nettle ✔, pixman ✔, snappy ✔, vde ✔
==> Options
--HEAD
	Install HEAD version

[touchardv]

With the current version (4.0.3), podman machine init is not enough to mount a local volume. You need to to specify some extra arguments e.g. podman machine init -v /Users/$USER:/Users/$USER.

[rhatdan]

The default in 4.1 will be -v $HOME:$HOME

[pheki]

Oh, thanks a lot! Somehow I missed that.

[ssbarnea]

While this was supposed to be fixed by newer versions of poadman that include $HOME in default machine mounts, for some reason with podman 4.4.2, it happens again, even if the machine was just recreated.

$ podman machine info                                              
Host:
  Arch: arm64
  CurrentMachine: podman-machine-default
  DefaultMachine: podman-machine-default
  EventsDir: /var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/podman-run--1/podman
  MachineConfigDir: /Users/ssbarnea/.config/containers/podman/machine/qemu
  MachineImageDir: /Users/ssbarnea/.config/containers/podman/machine/qemu
  MachineState: Running
  NumberOfMachines: 1
  OS: darwin
  VMType: qemu
Version:
  APIVersion: 4.4.2
  Built: 1677167961
  BuiltTime: Thu Feb 23 15:59:21 2023
  GitCommit: 74afe26887f814d1c39925a1624851ef3590e79c
  GoVersion: go1.20.1
  Os: darwin
  OsArch: darwin/arm64
  Version: 4.4.2

[heyvito]

4.4.1 amd64 (installed through Brew) has the same behaviour.

[jgresham]

Local directories are working for me. Podman does mount /Users:/Users by default. It shows when you create a new machine (I don't know the command to list mounts after a machine is created).

Isn't there a problem with the example where they are mounting ~/git/podman to a different location than Users/.../git/podman? Inside the container only /usr/src/hewwo would be seen, not git/podman-test... The example has -v ~/git/podman-test:/usr/src/hewwo

[ssbarnea]

Important aspect to check for, podman --version might report different version that podman info as the podman machine might be outdated and podman has no idea to warn the user about this.

In my case I had the 4.4.2 client with 4.4.1 machine and it was giving this error. I destroyed the machine and recreated it and it magically started to work. I fixed it by running init without any extra arguments, and we should ensure that no extra options are needed.

That is not the first time when I encounter a bug like this but I am not sure what we can do to improve the experience.

[v1k0d3n]

Not sure if this helps anyone, but if it's useful I'll share:

I downloaded the new Podman Desktop, and started a podman machine via the GUI. For the life of me, I couldn't find out why local mounts were not working correctly. Everything appeared as expected, but it just didn't click. It wasn't until I went into the CLI and remove the machine, then recreated the machine, did it start working correctly again. I didn't really expect it to work (or I would have taken more notes), but let me send what I can/had in front of me at the time:

❯ ls -asl /Users/bjozsa/Documents/Personal/Gaming/Nintendo/Amiibo 
## Yes, I'm converting backed up Amiibo bins to .nfc for my Flipper Zero
total 0
0 drwxr-xr-x 4 bjozsa staff 128 May 18 13:48 .
0 drwxr-xr-x 3 bjozsa staff  96 May 18 13:47 ..
0 -rw-r--r-- 1 bjozsa staff   0 May 18 13:48 file
0 drwxr-xr-x 3 bjozsa staff  96 May 18 13:48 tmp


❯ podman machine ls
NAME                     VM TYPE     CREATED      LAST UP            CPUS        MEMORY      DISK SIZE
podman-machine-default*  qemu        2 weeks ago  Currently running  1           2.147GB     107.4GB

❯ podman run --volume /Users/bjozsa/Documents/Personal/Gaming/Nintendo/Amiibo:/opt fedora ls /opt
Error: statfs /Users/bjozsa/Documents/Personal/Gaming/Nintendo/Amiibo: no such file or directory

❯ podman machine stop
Machine "podman-machine-default" stopped successfully
❯ podman machine rm

The following files will be deleted:

/Users/bjozsa/.ssh/podman-machine-default
/Users/bjozsa/.ssh/podman-machine-default.pub
/Users/bjozsa/.config/containers/podman/machine/qemu/podman-machine-default.ign
/Users/bjozsa/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-38.20230414.2.1-qemu.aarch64.qcow2
/Users/bjozsa/.local/share/containers/podman/machine/qemu/podman.sock
/Users/bjozsa/.local/share/containers/podman/machine/qemu/podman-machine-default_ovmf_vars.fd
/Users/bjozsa/.config/containers/podman/machine/qemu/podman-machine-default.json


Are you sure you want to continue? [y/N] y
❯ podman machine init --rootful --now
Downloading VM image: fedora-coreos-38.20230514.2.0-qemu.aarch64.qcow2.xz: done
Extracting compressed file
Image resized.
Machine init complete
Starting machine "podman-machine-default"
Waiting for VM ...
Mounting volume... /Users:/Users
Mounting volume... /private:/private
Mounting volume... /var/folders:/var/folders
API forwarding listening on: /var/run/docker.sock
Docker API clients default to this address. You do not need to set DOCKER_HOST.

Machine "podman-machine-default" started successfully

❯ podman run --volume /Users/bjozsa/Documents/Personal/Gaming/Nintendo/Amiibo:/opt fedora ls /opt
Resolved "fedora" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull registry.fedoraproject.org/fedora:latest...
Getting image source signatures
Copying blob sha256:d1270e6302d3f5b5321a1a5602be79eab9c11f9c2aaaaffa04cc5ac7454c91b1
Copying config sha256:211d750af63a4a075ac06311bf4457b66c3c33d136f4fc966f90bb1612011f8f
Writing manifest to image destination
Storing signatures
file
tmp

So all is good now. Again, not sure if any of this is helpful (it's definitely not new), but at least it's one more voice for "things just seemed to work, after first having some trouble."

[siathalysedI]

When updating podman desktop now to v1.19.2 I had this error. Had to delete my machine with all my containers and images in it. Worked finally thank you. But I think I should not need to delete all of my previous work.