containerd.sock not found when trying to use a kubeadm command

[marius-lupu]

Description

I installed containerd in rootless mode as described here.
Unfortunately, when I try to run the following command:
kubeadm config images pull
It throws the following error:

failed to pull image "k8s.gcr.io/kube-apiserver:v1.24.2": output: time="2022-06-23T13:20:32Z" level=fatal msg="unable to determine image API version: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial unix /var/run/containerd/containerd.sock: connect: no such file or directory\""
, error: exit status 1
To see the stack trace of this error execute with --v=5 or higher

Indeed I don't have that file although I can use contained with nerdctl.

Steps to reproduce the issue

1. Follow the instructions https://github.com/containerd/nerdctl/blob/master/docs/rootless.md#daemon-containerd
2. export CONTAINERD_SNAPSHOTTER=native
3. kubeadm config images pull

Describe the results you received and expected

failed to pull image "k8s.gcr.io/kube-apiserver:v1.24.2": output: time="2022-06-23T13:26:01Z" level=fatal msg="unable to determine image API version: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial unix /var/run/containerd/containerd.sock: connect: no such file or directory\""
, error: exit status 1
To see the stack trace of this error execute with --v=5 or higher

I'm expecting for the kubeadm command to work.

Additional info:

$ systemctl status --user containerd 
● containerd.service - containerd (Rootless)
   Loaded: loaded (/home/cloud-user/.config/systemd/user/containerd.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2022-06-23 12:24:33 UTC; 1h 2min ago
 Main PID: 9812 (rootlesskit)
    Tasks: 32
   Memory: 51.2M
   CGroup: /user.slice/user-1000.slice/[email protected]/containerd.service
           ├─9812 rootlesskit --state-dir=/run/user/1000/containerd-rootless --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-drive>
           ├─9823 /proc/self/exe --state-dir=/run/user/1000/containerd-rootless --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-dr>
           ├─9840 slirp4netns --mtu 65520 -r 3 --disable-host-loopback --enable-sandbox --enable-seccomp 9823 tap0
           └─9848 containerd

Jun 23 12:24:33 marius-tf-067.novalocal containerd-rootless.sh[9812]: time="2022-06-23T12:24:33.738936697Z" level=info msg=serving... address=/run/containerd/containerd.sock
Jun 23 12:24:33 marius-tf-067.novalocal containerd-rootless.sh[9812]: time="2022-06-23T12:24:33.738981949Z" level=info msg="containerd successfully booted in 0.041294s"
Jun 23 12:24:40 marius-tf-067.novalocal containerd-rootless.sh[9812]: time="2022-06-23T12:24:40.149061546Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.contai>
Jun 23 12:24:40 marius-tf-067.novalocal containerd-rootless.sh[9812]: time="2022-06-23T12:24:40.149206878Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.cont>
Jun 23 12:24:40 marius-tf-067.novalocal containerd-rootless.sh[9812]: time="2022-06-23T12:24:40.149278966Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.>
Jun 23 12:24:40 marius-tf-067.novalocal containerd-rootless.sh[9812]: time="2022-06-23T12:24:40.149646813Z" level=info msg="starting signal loop" namespace=default path=/run/containerd/io.container>
Jun 23 12:24:42 marius-tf-067.novalocal containerd-rootless.sh[9812]: time="2022-06-23T12:24:42.285123894Z" level=info msg="shim disconnected" id=d8823afe204fe8d982ed88cc547bd9d331e04a93bfa60a72c5d>
Jun 23 12:24:42 marius-tf-067.novalocal containerd-rootless.sh[9812]: time="2022-06-23T12:24:42.285283711Z" level=warning msg="cleaning up after shim disconnected" id=d8823afe204fe8d982ed88cc547bd9>
Jun 23 12:24:42 marius-tf-067.novalocal containerd-rootless.sh[9812]: time="2022-06-23T12:24:42.285328636Z" level=info msg="cleaning up dead shim"
Jun 23 12:24:42 marius-tf-067.novalocal containerd-rootless.sh[9812]: time="2022-06-23T12:24:42.299390462Z" level=warning msg="cleanup warnings time=\"2022-06-23T12:24:42Z\" level=info msg=\"start

$ nerdctl run --rm -it alpine
/ # 

$ ls /var/run/containerd/containerd.sock
ls: cannot access '/var/run/containerd/containerd.sock': No such file or directory

What version of nerdctl are you using?

Client:
 Version:	v0.20.0
 OS/Arch:	linux/amd64
 Git commit:	e77e05b5fd252274e3727e0439e9a2d45622ccb9

Server:
 containerd:
  Version:	v1.6.4
  GitCommit:	212e8b6fa2f44b9c21b2798135fc6fb7c53efc16

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

No response

Host information

Client:
 Namespace:	default
 Debug Mode:	false

Server:
 Server Version: v1.6.4
 Storage Driver: overlayfs
 Logging Driver: json-file
 Cgroup Driver: none
 Cgroup Version: 2
 Plugins:
  Log: json-file
  Storage: native overlayfs
 Security Options:
  seccomp
   Profile: default
  cgroupns
  rootless
 Kernel Version: 4.18.0-305.el8.x86_64
 Operating System: GNU/Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.46GiB
 Name: marius-tf-067.novalocal
 ID: 3af7189f-3700-4eff-8a4e-e081828646a8

WARNING: Running in rootless-mode without cgroups. Systemd is required to enable cgroups in rootless-mode.
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

[junnplus]

You should install rootful containerd for kubeadm config images pull.

[marius-lupu]

Thanks for your message, unfortunately, I need to install Kubernetes and its components (containerd in my case as container engine) in rootless mode.

[AkihiroSuda]

No rootless kubeadm yet.

See https://github.com/rootless-containers/usernetes for rootless Kubernetes in the
"hard way"

[marius-lupu]

Thanks for your message @AkihiroSuda. At this point, I'm trying to use Kubernetes (kubeadm, kubelet, kubectl) directly, not a variant of it. I understand if this is not related to containerd, but I will appreciate your advice/info.
I'm trying to follow this documentation. I also tried to start containerd in a user namespace using nohup containerd & with and without a config, then running kubeadm there but didn't work.
Do I have to install containerd in rootful mode inside a user namespace before running kubeadm?