|
| 1 | +<div align="center"> |
| 2 | + <img src="./logo.png" alt="Uniflight Logo" width="96"> |
| 3 | + |
| 4 | +# Uniflight |
| 5 | + |
| 6 | +[](https://crates.io/crates/uniflight) |
| 7 | +[](https://docs.rs/uniflight) |
| 8 | +[](https://crates.io/crates/uniflight) |
| 9 | +[](https://github.com/microsoft/oxidizer/actions/workflows/main.yml) |
| 10 | +[](https://codecov.io/gh/microsoft/oxidizer) |
| 11 | +[](../../LICENSE) |
| 12 | +<a href="../.."><img src="../../logo.svg" alt="This crate was developed as part of the Oxidizer project" width="20"></a> |
| 13 | + |
| 14 | +</div> |
| 15 | + |
| 16 | +Coalesces duplicate async tasks into a single execution. |
| 17 | + |
| 18 | +This crate provides [`Merger`][__link0], a mechanism for deduplicating concurrent async operations. |
| 19 | +When multiple tasks request the same work (identified by a key), only the first task (the |
| 20 | +“leader”) performs the actual work while subsequent tasks (the “followers”) wait and receive |
| 21 | +a clone of the result. |
| 22 | + |
| 23 | +## When to Use |
| 24 | + |
| 25 | +Use `Merger` when you have expensive or rate-limited operations that may be requested |
| 26 | +concurrently with the same parameters: |
| 27 | + |
| 28 | +* **Cache population**: Prevent thundering herd when a cache entry expires |
| 29 | +* **API calls**: Deduplicate concurrent requests to the same endpoint |
| 30 | +* **Database queries**: Coalesce identical queries issued simultaneously |
| 31 | +* **File I/O**: Avoid reading the same file multiple times concurrently |
| 32 | + |
| 33 | +## Example |
| 34 | + |
| 35 | +```rust |
| 36 | +use uniflight::Merger; |
| 37 | + |
| 38 | +let group: Merger<String, String> = Merger::new(); |
| 39 | + |
| 40 | +// Multiple concurrent calls with the same key will share a single execution. |
| 41 | +// Note: you can pass &str directly when the key type is String. |
| 42 | +let result = group.execute("user:123", || async { |
| 43 | + // This expensive operation runs only once, even if called concurrently |
| 44 | + "expensive_result".to_string() |
| 45 | +}).await.expect("leader should not panic"); |
| 46 | +``` |
| 47 | + |
| 48 | +## Flexible Key Types |
| 49 | + |
| 50 | +The [`Merger::execute`][__link1] method accepts keys using [`Borrow`][__link2] semantics, allowing you to pass |
| 51 | +borrowed forms of the key type. For example, with `Merger<String, T>`, you can pass `&str` |
| 52 | +directly without allocating: |
| 53 | + |
| 54 | +```rust |
| 55 | +let merger: Merger<String, i32> = Merger::new(); |
| 56 | + |
| 57 | +// Pass &str directly - no need to call .to_string() |
| 58 | +let result = merger.execute("my-key", || async { 42 }).await; |
| 59 | +assert_eq!(result, Ok(42)); |
| 60 | +``` |
| 61 | + |
| 62 | +## Thread-Aware Scoping |
| 63 | + |
| 64 | +`Merger` supports thread-aware scoping via a [`Strategy`][__link3] |
| 65 | +type parameter. This controls how the internal state is partitioned across threads/NUMA nodes: |
| 66 | + |
| 67 | +* [`PerProcess`][__link4] (default): Single global state, maximum deduplication |
| 68 | +* [`PerNuma`][__link5]: Separate state per NUMA node, NUMA-local memory access |
| 69 | +* [`PerCore`][__link6]: Separate state per core, no deduplication (useful for already-partitioned work) |
| 70 | + |
| 71 | +```rust |
| 72 | +use uniflight::Merger; |
| 73 | +use thread_aware::PerNuma; |
| 74 | + |
| 75 | +// NUMA-aware merger - each NUMA node gets its own deduplication scope |
| 76 | +let merger: Merger<String, String, PerNuma> = Merger::new_per_numa(); |
| 77 | +``` |
| 78 | + |
| 79 | +## Cancellation and Panic Handling |
| 80 | + |
| 81 | +`Merger` handles task cancellation and panics explicitly: |
| 82 | + |
| 83 | +* If the leader task is cancelled or dropped, a follower becomes the new leader |
| 84 | +* If the leader task panics, followers receive [`LeaderPanicked`][__link7] error with the panic message |
| 85 | +* Followers that join before the leader completes receive the value the leader returns |
| 86 | + |
| 87 | +When a panic occurs, followers are notified via the error type rather than silently |
| 88 | +retrying. The panic message is captured and available via [`LeaderPanicked::message`][__link8]: |
| 89 | + |
| 90 | +```rust |
| 91 | +let merger: Merger<String, String> = Merger::new(); |
| 92 | +match merger.execute("key", || async { "result".to_string() }).await { |
| 93 | + Ok(value) => println!("got {value}"), |
| 94 | + Err(err) => { |
| 95 | + println!("leader panicked: {}", err.message()); |
| 96 | + // Decide whether to retry |
| 97 | + } |
| 98 | +} |
| 99 | +``` |
| 100 | + |
| 101 | +## Memory Management |
| 102 | + |
| 103 | +Completed entries are automatically removed from the internal map when the last caller |
| 104 | +finishes. This ensures no stale entries accumulate over time. |
| 105 | + |
| 106 | +## Type Requirements |
| 107 | + |
| 108 | +The value type `T` must implement [`Clone`][__link9] because followers receive a clone of the |
| 109 | +leader’s result. The key type `K` must implement [`Hash`][__link10] and [`Eq`][__link11]. |
| 110 | + |
| 111 | +## Thread Safety |
| 112 | + |
| 113 | +[`Merger`][__link12] is `Send` and `Sync`, and can be shared across threads. The returned futures |
| 114 | +are `Send` when the closure, future, key, and value types are `Send`. |
| 115 | + |
| 116 | +## Performance |
| 117 | + |
| 118 | +Run benchmarks with `cargo bench -p uniflight`. The suite covers: |
| 119 | + |
| 120 | +* `single_call`: Baseline latency with no contention |
| 121 | +* `high_contention_100`: 100 concurrent tasks on the same key |
| 122 | +* `distributed_10x10`: 10 keys with 10 tasks each |
| 123 | + |
| 124 | +Use `--save-baseline` and `--baseline` flags to track regressions over time. |
| 125 | + |
| 126 | + |
| 127 | +<hr/> |
| 128 | +<sub> |
| 129 | +This crate was developed as part of <a href="../..">The Oxidizer Project</a>. Browse this crate's <a href="https://github.com/microsoft/oxidizer/tree/main/crates/uniflight">source code</a>. |
| 130 | +</sub> |
| 131 | + |
| 132 | + [__cargo_doc2readme_dependencies_info]: ggGkYW0CYXSEGy4k8ldDFPOhG2VNeXtD5nnKG6EPY6OfW5wBG8g18NOFNdxpYXKEGxgwNFq9VUtfG5xaBNm6U4VGG97W2YkyKkPjG4KVgSbTgdOrYWSCgmx0aHJlYWRfYXdhcmVlMC42LjGCaXVuaWZsaWdodGUwLjEuMA |
| 133 | + [__link0]: https://docs.rs/uniflight/0.1.0/uniflight/struct.Merger.html |
| 134 | + [__link1]: https://docs.rs/uniflight/0.1.0/uniflight/?search=Merger::execute |
| 135 | + [__link10]: https://doc.rust-lang.org/stable/std/?search=hash::Hash |
| 136 | + [__link11]: https://doc.rust-lang.org/stable/std/cmp/trait.Eq.html |
| 137 | + [__link12]: https://docs.rs/uniflight/0.1.0/uniflight/struct.Merger.html |
| 138 | + [__link2]: https://doc.rust-lang.org/stable/std/?search=borrow::Borrow |
| 139 | + [__link3]: https://docs.rs/thread_aware/0.6.1/thread_aware/?search=storage::Strategy |
| 140 | + [__link4]: https://docs.rs/thread_aware/0.6.1/thread_aware/?search=PerProcess |
| 141 | + [__link5]: https://docs.rs/thread_aware/0.6.1/thread_aware/?search=PerNuma |
| 142 | + [__link6]: https://docs.rs/thread_aware/0.6.1/thread_aware/?search=PerCore |
| 143 | + [__link7]: https://docs.rs/uniflight/0.1.0/uniflight/struct.LeaderPanicked.html |
| 144 | + [__link8]: https://docs.rs/uniflight/0.1.0/uniflight/?search=LeaderPanicked::message |
| 145 | + [__link9]: https://doc.rust-lang.org/stable/std/clone/trait.Clone.html |
0 commit comments