Expected behavior:
Sessions with user provisioning enabled should not deadlock or crash.
Current behavior:
On older versions of Ubuntu (20.04) with libnss-extrausers 0.6-4 new sessions when create host user is enabled can hang indefinitely and crash on later versions (26.04).
Fundamentally I believe this is because this library is not thread safe: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831390.
This can be reproduced by randomly calling user.{Lookup,LookupGroup,LookupGroupId,GroupIds} with non existent users to traverse the full nss chain from multiple go routines.
I attached a thread dump from the reproducer:
threads.txt
libnss_extrausers keeps a module-global FILE* for its group database. _nss_extrausers_endgrent() takes no arguments it closes that shared stream.
- Two threads overwrite each other's stream pointer, then both
fclose the survivor Threads 7 and 11 on the identical FILE *0xffff280033c0 see above dump.
- The double unlink corrupts
_IO_list_all into a cylce.
- Thread 3 enters
_IO_un_link holding list_all_lock and never leaves.
- All remaining threads block on
list_all_lock
Notably on Ubuntu 26.04 this manifests as a double free crash instead.
Bug details:
- Teleport version: Any
- Recreation steps
Ubuntu with libnss-systemd libnss-extrausers
nsswitch.conf configured with at least:
passwd: compat systemd exec extrausers
group: files systemd extrausers
shadow: files extrausers
While this bug is not a bug in Teleport, this configuration can negatively impact users and lead to broken functionality by the Agent.
Expected behavior:
Sessions with user provisioning enabled should not deadlock or crash.
Current behavior:
On older versions of Ubuntu (20.04) with libnss-extrausers 0.6-4 new sessions when create host user is enabled can hang indefinitely and crash on later versions (26.04).
Fundamentally I believe this is because this library is not thread safe: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831390.
This can be reproduced by randomly calling
user.{Lookup,LookupGroup,LookupGroupId,GroupIds}with non existent users to traverse the full nss chain from multiple go routines.I attached a thread dump from the reproducer:
threads.txt
libnss_extrauserskeeps a module-globalFILE*for its group database._nss_extrausers_endgrent()takes no arguments it closes that shared stream.fclosethe survivor Threads 7 and 11 on the identicalFILE *0xffff280033c0see above dump._IO_list_allinto a cylce._IO_un_linkholdinglist_all_lockand never leaves.list_all_lockNotably on Ubuntu 26.04 this manifests as a double free crash instead.
Bug details:
Ubuntu with
libnss-systemd libnss-extrausersnsswitch.confconfigured with at least:While this bug is not a bug in Teleport, this configuration can negatively impact users and lead to broken functionality by the Agent.