Skip to content

Reimplement futex manager to use interruptible waits#501

Merged
jstarks merged 8 commits intomicrosoft:mainfrom
jstarks:futex4
Nov 20, 2025
Merged

Reimplement futex manager to use interruptible waits#501
jstarks merged 8 commits intomicrosoft:mainfrom
jstarks:futex4

Conversation

@jstarks
Copy link
Member

@jstarks jstarks commented Nov 19, 2025

Update futex manager to use interruptible waits. As part of this, implement an intrusive linked list to avoid heap
allocations in the futex path.

The futex and linked list tests have been run against miri to check for uses of undefined behavior.

Update futex manager to use interruptible waits. As part of this,
implement an intrusive linked list to avoid heap
allocations in the futex path.

The futex and linked list tests have been run against miri to check
for uses of undefined behavior.
@jstarks jstarks marked this pull request as ready for review November 19, 2025 05:25
@jstarks jstarks marked this pull request as draft November 19, 2025 07:07
@jstarks jstarks marked this pull request as ready for review November 19, 2025 07:59
Copy link
Member

@wdcui wdcui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the code change, John. I left some minor comments. Please wait for @CvvT 's review.

@wdcui wdcui requested a review from CvvT November 20, 2025 04:31
Copy link
Contributor

@CvvT CvvT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! With this new design of WaitContext, things got simplifed a lot and futex now can support bitset mask. I only have a high level question: why we switched from hashmap to hashtable with linked list? You mentioned it avoids heap allocation. I wonder it is purely for optimization or has any other reasons.

@jstarks
Copy link
Member Author

jstarks commented Nov 20, 2025

LGTM! With this new design of WaitContext, things got simplifed a lot and futex now can support bitset mask. I only have a high level question: why we switched from hashmap to hashtable with linked list? You mentioned it avoids heap allocation. I wonder it is purely for optimization or has any other reasons.

As LiteBox becomes more general-purpose, I think we'll have to confront the reality that "panic on OOM" is not a viable strategy for some scenarios. So, if we can design some of our core data structures to avoid needing allocations in the first place, then the eventual conversion will be that much easier.

It's probably OK to terminate the current process on OOM in cases where the contract doesn't allow us to return ENOMEM, so this is not strictly necessary. But still, better to just avoid allocations when it's straightforward to do so.

In this case, I had to change the data structure for the futex manager anyway--the old hash table was based on a single entry per futex, but with the new wait model (and to correctly support bitsets) we need an entry per waiter. Or at least a list of waiters per entry. So, I figured that we might as well switch to a data structure that's much more efficient for the use case, allocations or no--an external chaining hash table gives us good concurrency (lock per bucket instead of a global lock) while not requiring resizing under high load.

It might be interesting to use this LoanList for event wait queues as well. I have a prototype to do this.

@github-actions
Copy link

🤖 SemverChecks 🤖 ⚠️ Potential breaking API changes detected ⚠️

Click for details
--- failure enum_discriminants_undefined_non_unit_variant: enum's variants no longer have defined discriminants due to non-unit variant ---

Description:
An enum's variants no longer have well-defined discriminant values due to a tuple or struct variant in the enum. This breaks downstream code that accesses discriminants via a numeric cast like `as isize`.
        ref: https://doc.rust-lang.org/reference/items/enumerations.html#assigning-discriminant-values
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.45.0/src/lints/enum_discriminants_undefined_non_unit_variant.ron

Failed in:
  enum FutexError in /home/runner/work/litebox/litebox/litebox/src/sync/futex.rs:187

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.45.0/src/lints/enum_variant_added.ron

Failed in:
  variant FutexError:WaitError in /home/runner/work/litebox/litebox/litebox/src/sync/futex.rs:193

--- failure enum_variant_missing: pub enum variant removed or renamed ---

Description:
A publicly-visible enum has at least one variant that is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.45.0/src/lints/enum_variant_missing.ron

Failed in:
  variant FutexError::TimedOut, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-main/2d724bf73ec7896701693dd0803e8a2cd86f7525/litebox/src/sync/futex.rs:360

@jstarks jstarks enabled auto-merge November 20, 2025 18:38
@jstarks jstarks added this pull request to the merge queue Nov 20, 2025
Merged via the queue into microsoft:main with commit 441d5f8 Nov 20, 2025
8 checks passed
@jstarks jstarks deleted the futex4 branch November 20, 2025 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants