Skip to content

Moves threading/synchronization to iree/base/threading/ for public use. #23337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
benvanik merged 5 commits into from
Jan 31, 2026
Merged

Moves threading/synchronization to iree/base/threading/ for public use. #23337

benvanik merged 5 commits into from
Jan 31, 2026
+5,429 −1,608

Conversation

@benvanik
Copy link
Collaborator

@benvanik benvanik commented Jan 30, 2026

Reorganizes threading primitives into a proper iree/base/threading/ module and adds comprehensive testing. The threading code was previously scattered through base/internal/ alongside unrelated utilities, making it hard to discover what synchronization primitives IREE provides and how they relate to each other.

The new structure groups related concepts:

  • Synchronization: mutex (fat and slim), notification (event counters)
  • Threading: thread lifecycle, call_once
  • Memory: NUMA-aware allocation with huge page support
  • Low-level: futex abstraction for building custom primitives

Key improvements:

  • Slim mutex is now well-documented with platform-specific implementation notes (os_unfair_lock on Apple, SRW on Windows, futex-based elsewhere)
  • NUMA API provides cross-platform huge page allocation with graceful fallbacks
  • Notification primitive replaces condvars in lock-free code paths
  • Atomic freelist enables lock-free pool management without pointer-based linked lists
  • Testing/benchmarking was improved as part of the move and exporting of the functions
  • IREE_SYNCHRONIZATION_DISABLE_UNSAFE is now a cmake var

Breaking changes:

  • shouldn't be any, as it was an internal/ API before ;)

ci-extra: all

@benvanik benvanik added runtime/api IREE's public C runtime API and iree-run-module cleanup 🧹 post-merge-review Ben's special place. People can pick these up and review them for forward fixes if interested. labels Jan 30, 2026
@benvanik @claude
Moves threading/synchronization to iree/base/threading/.
9b5db82 
Elevates threading primitives from scattered internal utilities to a proper
first-class module. The threading code was previously buried in
`iree/base/internal/` alongside unrelated utilities and was not exported for
public use. This reorganization makes the threading API surface explicit and
sets up a foundation for future NUMA and concurrency work that will expand the
file count.

New/moved module components:
- `api.h`: Umbrella header for convenient inclusion
- `mutex.h`: Fat mutex and slim mutex with detailed platform documentation
- `notification.h`: Wait-free event counters for lock-free coordination
- `thread.h`: Thread creation, affinity, and lifecycle management
- `call_once.h`: One-time initialization primitive (moved from internal)

Trampoline headers were left in iree/base/internal/ for future cleanup.

Co-Authored-By: Claude <[email protected]>
@benvanik benvanik force-pushed the users/benvanik/threading branch 4 times, most recently from 55a1c6e to f9b8d9d Compare January 30, 2026 23:12
benvanik and others added 4 commits January 31, 2026 07:51
@benvanik @claude
Adding iree_atomic_freelist_t, a lock-free index pool.
2f23170 
Uses 64-bit CAS with generation counters to avoid ABA problems.
This is useful for managing fixed-size pools without locks.


Co-Authored-By: Claude <[email protected]>
@benvanik @claude
Adding iree_futex_t, a cross-platform futex abstraction.
07ff418 
Works on Linux, Windows, Emscripten, but is not available on MacOS.
This is intended for use by implementations that know they want it.


Co-Authored-By: Claude <[email protected]>
@benvanik @claude
Adds NUMA-aware memory allocation utilities.
6e986b6 
Provides a cross-platform API for NUMA-placed memory allocation with
optional huge page support. Platforms without NUMA support (macOS,
Emscripten) get transparent fallbacks that use standard allocation paths.

This centralizes the platform-specific mmap/mbind/VirtualAllocExNuma code
that would otherwise be duplicated in buffer pools, slab allocators, and
other infrastructure requiring NUMA-aware placement.

Moved iree_bitmap_t out to the public API, as it's a useful primitive.

Co-Authored-By: Claude <[email protected]>
@benvanik @claude
Completes migration of iree/base/threading and adds comprehensive tests.
5293145 
Finishes the threading module migration by removing backward-compatibility
shim headers and updating all consumers to use direct includes. This makes
dependencies explicit and removes the indirection that was masking the
true module structure.

Adds a full test and benchmark suite:
- Mutex: correctness under contention, try_lock semantics, performance
  characteristics of slim vs fat mutex across thread counts
- Notification: producer/consumer patterns, timeout behavior, spurious
  wakeup handling, throughput under varying contention
- Thread: creation/join lifecycle, thread-local storage, affinity,
  priority inheritance behavior
- Call-once: concurrent initialization races, exception safety

Co-Authored-By: Claude <[email protected]>
@benvanik benvanik force-pushed the users/benvanik/threading branch from f9b8d9d to 5293145 Compare January 31, 2026 15:51
@benvanik benvanik marked this pull request as ready for review January 31, 2026 21:15
@benvanik benvanik merged commit e4ab789 into main Jan 31, 2026
97 of 139 checks passed
@benvanik benvanik deleted the users/benvanik/threading branch January 31, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@stellaraccident stellaraccident Awaiting requested review from stellaraccident stellaraccident is a code owner

@antiagainst antiagainst Awaiting requested review from antiagainst antiagainst is a code owner

@ScottTodd ScottTodd Awaiting requested review from ScottTodd ScottTodd is a code owner

@AWoloszyn AWoloszyn Awaiting requested review from AWoloszyn AWoloszyn is a code owner

Assignees

No one assigned

Labels

cleanup 🧹 post-merge-review Ben's special place. People can pick these up and review them for forward fixes if interested. runtime/api IREE's public C runtime API and iree-run-module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@benvanik