Adding Tailscale to Heimdall - Tailscale service doesn’t start.

[jazzmonkai]

Describe the issue
I used the (excellent) Heimdall script to create an lxc which I’d like to add to my tailscale network. Using the tailscale script with that CT appears to work, but on attempting to start the service with “tailscale up”, I get the following:

root@xxx:~# tailscale up failed to connect to local tailscaled; it doesn't appear to be running (sudo systemctl start tailscaled ?) root@xxx:~#

I have other lxc and VM’s that I have successfully installed tailscale on, both manually and using the script, but i cannot see what is wrong here. The suggested systemctl action does not resolve the issue either.

what am I missing here?

** edit - yes, I rebooted the lxc between installation and attempting to start tailscale! **

[jazzmonkai]

Ok - so I've looked a bit more closely at the results of a manual installation and it seems it's the creation of the tailscaled service that is failing.

The installation script suggests running systemctl status tailscaled.service to see the results of the following log:

* tailscaled.service - Tailscale node agent
     Loaded: loaded (/lib/systemd/system/tailscaled.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sat 2022-10-15 20:26:22 UTC; 2min 47s ago
       Docs: https://tailscale.com/kb/
    Process: 944 ExecStartPre=/usr/sbin/tailscaled --cleanup (code=exited, status=0/SUCCESS)
    Process: 948 ExecStart=/usr/sbin/tailscaled --state=/var/lib/tailscale/tailscaled.state --socket=/run/tailscale/tailscaled.sock --port=${PORT} $FLAGS (code=exited, st>
    Process: 954 ExecStopPost=/usr/sbin/tailscaled --cleanup (code=exited, status=0/SUCCESS)
   Main PID: 948 (code=exited, status=1/FAILURE)
        CPU: 77ms

Oct 15 20:26:22 heimdalldashboard systemd[1]: tailscaled.service: Failed with result 'exit-code'.
Oct 15 20:26:22 heimdalldashboard systemd[1]: Failed to start Tailscale node agent.
Oct 15 20:26:22 heimdalldashboard systemd[1]: tailscaled.service: Scheduled restart job, restart counter is at 5.
Oct 15 20:26:22 heimdalldashboard systemd[1]: Stopped Tailscale node agent.
Oct 15 20:26:22 heimdalldashboard systemd[1]: tailscaled.service: Start request repeated too quickly.
Oct 15 20:26:22 heimdalldashboard systemd[1]: tailscaled.service: Failed with result 'exit-code'.
Oct 15 20:26:22 heimdalldashboard systemd[1]: Failed to start Tailscale node agent.
...skipping...
* tailscaled.service - Tailscale node agent
     Loaded: loaded (/lib/systemd/system/tailscaled.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sat 2022-10-15 20:26:22 UTC; 2min 47s ago
       Docs: https://tailscale.com/kb/
    Process: 944 ExecStartPre=/usr/sbin/tailscaled --cleanup (code=exited, status=0/SUCCESS)
    Process: 948 ExecStart=/usr/sbin/tailscaled --state=/var/lib/tailscale/tailscaled.state --socket=/run/tailscale/tailscaled.sock --port=${PORT} $FLAGS (code=exited, st>
    Process: 954 ExecStopPost=/usr/sbin/tailscaled --cleanup (code=exited, status=0/SUCCESS)
   Main PID: 948 (code=exited, status=1/FAILURE)
        CPU: 77ms

Oct 15 20:26:22 heimdalldashboard systemd[1]: tailscaled.service: Failed with result 'exit-code'.
Oct 15 20:26:22 heimdalldashboard systemd[1]: Failed to start Tailscale node agent.
Oct 15 20:26:22 heimdalldashboard systemd[1]: tailscaled.service: Scheduled restart job, restart counter is at 5.
Oct 15 20:26:22 heimdalldashboard systemd[1]: Stopped Tailscale node agent.
Oct 15 20:26:22 heimdalldashboard systemd[1]: tailscaled.service: Start request repeated too quickly.
Oct 15 20:26:22 heimdalldashboard systemd[1]: tailscaled.service: Failed with result 'exit-code'.
Oct 15 20:26:22 heimdalldashboard systemd[1]: Failed to start Tailscale node agent.

I'm relatively new to Linux and Proxmox so all I can see from this is that

Process: 948 ExecStart=/usr/sbin/tailscaled --state=/var/lib/tailscale/tailscaled.state --socket=/run/tailscale/tailscaled.sock --port=${PORT} $FLAGS (code=exited, st>

seems to have failed but I don't know where to start looking to understand WHY!

[jcastro]

I'm having same issue on the docker LXC install. Did you find a solution?

[jazzmonkai]

I didn’t, at least not directly.

This week I enabled subnet routing on my Proxmox host Tailscale instance, so I can now access all the local domains remotely. This is actually a great option as there’s no port forwarding, and I’m not routing all my traffic through Tailscale like I would if I ran a VPN (which was my next option).

But I’m fairly dumb and new to running a home server so I’m sure there’s lots of reasons this approach is terrible!

[newmy-de]

Createing a subnetrouter will be easier than installing on every system inside a subnet.
@tteck has an own script for building this inside an own lxc.

[jazzmonkai]

Ok, after some more prodding and poking, I discovered I had the same issue with my Pi-hole LXC after an update, which I had previously had Tailscale installed and working on.

The Tailscale docs have guidance on installing in unprivileged containers (https://tailscale.com/kb/1130/lxc-unprivileged/) which I had used for Pi-hole first time around. After the update, it no longer worked.

The subnet option was not performing as well as when I had a dedicated install for Pi-hole, and so I was more motivated to fix it - and the same solution worked for Heimdall!

In the end, it was simple. Tailscale give entries to add to the LXC's .conf file for pre- and post- Proxmox v7 installations. What I discovered is that I had to add ALL the entries, not just the version appropriate ones.

And, snapshots seem to add info to the end of the LXC .conf file, which in my case hadn't transferred my manually added Tailscale entries to the end, which is why my update of Pi-hole broke it. I'd done a snapshot before the update.

So, follow the Tailscale link above, add all three lines to the LXC's .conf file as described, and Tailscale should be able to install and start correctly.

[tteck]

3 lines? The script adds 2 lines already

CTID=$1
CTID_CONFIG_PATH=/etc/pve/lxc/${CTID}.conf
cat <<EOF >>$CTID_CONFIG_PATH
lxc.cgroup2.devices.allow: c 10:200 rwm
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file
EOF

But, what's the third?

[jazzmonkai]

Ok, my reading skills have clearly been failing me.

so your script is using cgroup2, I’m using that and cgroup:

lxc.cgroup2.devices.allow: c 10:200 rwm

AND

lxc.cgroup.devices.allow: c 10:200 rwm

With the lxc.mount line too. Only seems to work with all three.

[tteck]

You must be using PVE 6

[jazzmonkai]

That would make sense, but it’s a fairly recent install and the package version list below looks to me like it’s v7.2? That’s also what the banner on the web management page says. 🤷🏼‍♂️

proxmox-ve: 7.2-1 (running kernel: 5.15.74-1-pve)
pve-manager: 7.2-14 (running version: 7.2-14/65898fbc)
pve-kernel-5.15: 7.2-14
pve-kernel-helper: 7.2-14
pve-kernel-5.15.74-1-pve: 5.15.74-1
pve-kernel-5.15.60-2-pve: 5.15.60-2
pve-kernel-5.15.30-2-pve: 5.15.30-3
ceph-fuse: 15.2.16-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-7
libpve-guest-common-perl: 4.2-2
libpve-http-server-perl: 4.1-5
libpve-storage-perl: 7.2-12
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.7-1
proxmox-backup-file-restore: 2.2.7-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.0-1
proxmox-widget-toolkit: 3.5.2
pve-cluster: 7.2-3
pve-container: 4.3-4
pve-docs: 7.2-3
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-7
pve-firmware: 3.5-6
pve-ha-manager: 3.4.0
pve-i18n: 2.7-2
pve-qemu-kvm: 7.1.0-3
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-10
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+2
vncterm: 1.7-1
zfsutils-linux: 2.1.6-pve1

[thezeagar]

I know this is a bit old, but I was just having the exact same issue installing TailScale on my AudiobookShelf LXC and wanted to confirm that @jazzmonkai's fix worked for me as well.

Currently on Proxmox 7.3-4. Tailscale service would not start.

Ran the script again, shut down the LXC, and added
lxc.cgroup.devices.allow: c 10:200 rwm
to the lxc.conf

Started the LXC back up and Tailscale worked perfectly.