Skip to content

Commit 8719ad8

Browse files
feat(span_concentrator): replace std map with hashbrown for better performances (#1234)
feat(span_concentrator): replace std map with hashbrown for better performances # Motivations Creating and checking for the existence of the aggregation key is the most expensive operation performed by data-pipeline if there is a high drop rate. Currently this operation is not optimal as equality has to go through virtual dispatch because of limitations on the std hashmap. Changing to hashbrown gives us: * A more efficient hashing function * We can implement the Equivalent trait instead of the borrow trait to compare owned and borrowed key, which performs better * We can drop the Cow and just use either String or &str * We can use the entry API on the hashmap so we don't have to potentially access the hashmap twice These changes bring a 25% perf improvement on the span concentrators benchmarks # Changes * Use hashbrown::HashMap instead of std::HashMap in the stats collector * Replace AggregationKey<'static> with OwnedAggregationKey * implement hashbrown::Equivalent instead of Borrow Merge branch 'main' into paullgdc/data-pipeline/stats_collector_hmap
1 parent f61c42a commit 8719ad8

File tree

5 files changed

+134
-159
lines changed

5 files changed

+134
-159
lines changed

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

data-pipeline/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ tokio = { version = "1.23", features = [
3030
], default-features = false }
3131
uuid = { version = "1.10.0", features = ["v4"] }
3232
tokio-util = "0.7.11"
33+
hashbrown = { version = "0.15" }
3334

3435
ddcommon = { path = "../ddcommon", default-features = false }
3536
ddtelemetry = { path = "../ddtelemetry", default-features = false }

0 commit comments

Comments
 (0)