Changing a shell in a guest VM makes it unbootable

[frkd-dev]

I never thought I would stuck with a such basic question but here we are: a simple shell change inside the VM makes it fail to boot.

I'm using the Alpine template and its default ash shell is incompatible with my project's zsh scripts, so I want to switch shell to zsh. This should be a trivial task as the lima is just a wrapper around the hypervisor running vm+ssh, so any basic/regular stuff in guest Linux should not break the VM, but it is.

I do:

sudo apk add zsh
sudo chsh -s /bin/zsh $USER
exit

I get:

# limactl stop myvm && limactl start myvm
INFO[0000] Using the existing instance "myvm"
INFO[0000] Starting the instance "myvm" with VM driver "vz"
INFO[0000] [hostagent] hostagent socket created at /Users/hostuser/.lima/myvm/ha.sock
INFO[0000] [hostagent] Starting VZ (hint: to watch the boot progress, see "/Users/hostuser/.lima/myvm/serial*.log")
INFO[0000] [hostagent] new connection from  to
INFO[0000] [hostagent] Setting up Rosetta share
INFO[0000] SSH Local Port: 51292
INFO[0000] [hostagent] [VZ] - vm state change: running
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] 2023/12/02 00:00:36 tcpproxy: for incoming conn 127.0.0.1:51308, error dialing "192.168.5.15:22": connect tcp 192.168.5.15:22: connection was refused
INFO[0020] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0030] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0040] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0050] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0061] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0071] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
etc…

What's broken and how to change the shell properly?

[jandubois]

You need to run your apk add zsh command in a system provisioning script:

lima/examples/default.yaml

Lines 185 to 195 in b453ded

	# Provisioning scripts need to be idempotent because they might be called
	# multiple times, e.g. when the host VM is being restarted.
	# 🟢 Builtin default: null
	# provision:
	# # `system` is executed with the root privilege
	# - mode: system
	# script: |
	# #!/bin/bash
	# set -eux -o pipefail
	# export DEBIAN_FRONTEND=noninteractive
	# apt-get install -y vim

Alpine is booted from an ISO and running from a RAM filesystem. Only a few directories like /etc and /usr/local are saved on a persistent volume. So you need to reinstall packages on each boot.

This will not redownload them; the apk package cache is also on the persisted volume.

[frkd-dev]

Ah, that was a missing piece of the puzzle which explains everything. Surprisingly Lima's documentation doesn't introduce this concept to a user and this can be learned only by reading templates.

Thank you!

[afbjorklund]

The templates don't have so much documentation either (not even a description or other metadata), possibly comments.

In this case, the implementation details would be found in the upstream distribution: https://github.com/lima-vm/alpine-lima