Skip to content

Improve thread safety of ENV on UNIX#16448

Closed
ysbaddaden wants to merge 4 commits intocrystal-lang:masterfrom
ysbaddaden:feature/add-thread-safety-to-env
Closed

Improve thread safety of ENV on UNIX#16448
ysbaddaden wants to merge 4 commits intocrystal-lang:masterfrom
ysbaddaden:feature/add-thread-safety-to-env

Conversation

@ysbaddaden
Copy link
Collaborator

@ysbaddaden ysbaddaden commented Nov 28, 2025

The environ libc pointer is thread unsafe. The getenv, setenv and unsetenv libc calls are also thread unsafe. At best they libc implementations will leak memory and at worst will segfault on a mere getenv call.

I propose to parse environ once at program startup into an internal hash. This happens before we start threads that may mutate environ in parallel and is safe; then environ is never accessed again (and getenv never called). See comments for details.

I added a readers-writer lock to protect the internal hash and libc function calls. Any calls to read and mutate the environment should be safe...

...but it's still thread unsafe in practice because crystal code might get the writer lock and call setenv while another thread executes an external lib function (including libc calls made by crystal stdlib) that internally calls getenv 💣

For this reason, the documentation has been updated to insist that ENV should be considered an immutable global collection.

Reduces the thread safety issues of ENV usages in crystal code.

It doesn't fix the thread safety issue of the libc getenv, setenv and
unsetenv implementations. It is entirely possible for external
libraries, including libc calls made by the stdlib itself, to call
getenv directly while another thread in Crystal calls setenv while
holding the write lock.
@ysbaddaden
Copy link
Collaborator Author

ysbaddaden commented Nov 28, 2025

CI is broken 😞

In fact Crystal::System::Env is completely broken... despite passing the specs.

Environ is parsed, @@env is populated, and we can iterate the key/value pairs, but trying to find by key fails. It's as if the key/value pair never existed in the first place.

Why? Because we require env before we require hash that will in turn require crystal/hasher and initialize the hasher seed... after we already initialized a Hash (oops).

Maybe I'll use an Array({String, String}) instead of a Hash(String, String).

@ysbaddaden ysbaddaden force-pushed the feature/add-thread-safety-to-env branch from 14cdf76 to ae5e178 Compare November 28, 2025 21:08
@ysbaddaden
Copy link
Collaborator Author

I modified Crystal::System::Env to use an array, but I decided to remove the initial parsing of environ. It is not required to fix thread safety issues, especially since we keep mutating the libc environment and libc and libraries will keep calling getenv anyway.

We'll only need the internal collection if we decide to have -Denv=safe. See #16449.

# Skip overrides in `env`
next if env.has_key?(key)
# Dup LibC.environ, skipping overrides in env.
each do |key, value|
Copy link
Collaborator Author

@ysbaddaden ysbaddaden Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is expensive and since we can't return environ anymore, this happens all the time.

I have a simple patch that yields slices to the kv pointer instead of strings, which avoids the intermediary string allocations and directly builds the "key=value" string. I'll push it in a follow up.

@ysbaddaden
Copy link
Collaborator Author

Superseded by #16591 and #16592.

@ysbaddaden ysbaddaden closed this Jan 20, 2026
@github-project-automation github-project-automation bot moved this from In Progress to Done in Multi-threading Jan 20, 2026
@ysbaddaden ysbaddaden deleted the feature/add-thread-safety-to-env branch January 20, 2026 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant