microk8s fails to start on a PXE booted raspberry pi 5

[bigatl22]

Summary

After installing microk8s it fails to start.

Reproduction Steps

I'm running this on a raspberry pi that is pxe-booting raspbian os lite debian 12 (bookworm). I'm able to install microk8s on the same raspberry pi if I'm using the sd card for the OS. This is repeatable when pxe-booting.

1. PXE Boot Raspberry Pi 5 (lots of steps to get this working)
2. Enable cgroups
3. Run firmware update
4. Install snap
5. Snap install core
6. Snap install microks8 v1.33/stable (I've tried other versions but same issue)

More details below
Hi Everyone, Please Help!
I have been trying to get my Raspberry Pi 5 to successfully start microk8s. I’m able to install without issue but then the trouble starts. I have tried to get it working for a few days now and searched several forums for answers. Here is what I’m seeing.

Hardware: Raspberry Pi 5
OS: Raspberry Pi OS Lite (64bit)
Release date: May 13th 2025
System: 64-bit
Kernel version: 6.12
Debian version: 12 (bookworm)

snap services

Service                                              Startup  Current   Notes
microk8s.daemon-apiserver-kicker   enabled  active    -
microk8s.daemon-apiserver-proxy   enabled  inactive  -
microk8s.daemon-cluster-agent       enabled  active    -
microk8s.daemon-containerd           enabled  active    -
microk8s.daemon-etcd                     enabled  inactive  -
microk8s.daemon-flanneld               enabled  inactive  -
microk8s.daemon-k8s-dqlite            enabled  active    -
microk8s.daemon-kubelite               enabled  active    -

microk8s status
microk8s is not running. Use microk8s inspect for a deeper inspection.

microk8s inspect

Inspecting system
Inspecting Certificates
Inspecting services
  Service snap.microk8s.daemon-cluster-agent is running
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-kubelite is running
  Service snap.microk8s.daemon-k8s-dqlite is running
  Service snap.microk8s.daemon-apiserver-kicker is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
  Copy processes list to the final report tarball
  Copy disk usage information to the final report tarball
  Copy memory usage information to the final report tarball
  Copy server uptime to the final report tarball
  Copy openSSL information to the final report tarball
  Copy snap list to the final report tarball
  Copy VM name (or none) to the final report tarball
  Copy current linux distribution to the final report tarball
  Copy asnycio usage and limits to the final report tarball
  Copy inotify max_user_instances and max_user_watches to the final report tarball
  Copy network configuration to the final report tarball
Inspecting kubernetes cluster
  Inspect kubernetes cluster
Inspecting dqlite
  Inspect dqlite
cp: cannot stat '/var/snap/microk8s/8206/var/kubernetes/backend/localnode.yaml': No such file or directory

Building the report tarball
  Report tarball is at /var/snap/microk8s/8206/inspection-report-20250610_170754.tar.gz

inspection-report-20250610_170754.tar.gz

[berkayoz]

Hey @bigatl22,

Have you followed https://microk8s.io/docs/install-raspberry-pi ?

I see

Jun 10 17:06:57 pi-cluster-m0 microk8s.daemon-containerd[501]: time="2025-06-10T17:06:57.794688830+01:00" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-node-9q27t,Uid:e045225d-1bb6-4aa8-bb15-6b7e4654141d,Namespace:kube-system,Attempt:0,} failed, error" error="failed to create containerd task: failed to create shim task: failed to mount rootfs component: invalid argument: unknown"
Jun 10 17:07:12 pi-cluster-m0 microk8s.daemon-containerd[501]: time="2025-06-10T17:07:12.689266334+01:00" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-node-9q27t,Uid:e045225d-1bb6-4aa8-bb15-6b7e4654141d,Namespace:kube-system,Attempt:0,}"

Which indicates a containerd related issue. Do you have any storage/disks mounted?