Why do some environment variables not propagate to kernels (I.e. LD_LIBRARY_PATH)?

[ehvr20]

Hi all,

I’m running Kallichore inside a Singularity container and trying to use GPU/CUDA functionality via the cupy Python library. After a frustrating few hours of debugging, I discovered that $LD_LIBRARY_PATH—which the CUDA runtime relies on to locate the drivers—was not being propagated correctly to spawned kernels.

Specifically:

• $LD_LIBRARY_PATH inside the container differs from what kernels launched from the container see.
• I had to manually define it in .bashrc and enable Kernel Supervisor: Run in Shell to make things work.
• Interestingly, some environment variables, like $SINGULARITY_NAME, are propagated correctly, but $LD_LIBRARY_PATH is reset.

I’ve seen a few related issues but nothing directly addressing this, so I’m not opening a bug report straightaway.

Question: Why are certain environment variables propagated correctly while $LD_LIBRARY_PATH is not?

Thanks in advance for any insights!