Skip to content

Add uniform block tracking stats#14513

Open
joshkang97 wants to merge 1 commit intofacebook:mainfrom
joshkang97:add_uniform_block_tracking
Open

Add uniform block tracking stats#14513
joshkang97 wants to merge 1 commit intofacebook:mainfrom
joshkang97:add_uniform_block_tracking

Conversation

@joshkang97
Copy link
Contributor

@joshkang97 joshkang97 commented Mar 26, 2026

Summary

Add tracking of uniform index blocks as a table property (num_uniform_blocks). When uniform_cv_threshold is set, the block builder detects uniformly distributed keys via coefficient of variation of key gaps. This information is now surfaced end-to-end: from block building through index builders to SST table properties.

Key changes

  • BlockBuilder: Persist the uniformity result from ScanForUniformity() as a member (is_uniform_) exposed via IsUniform(), rather than a local variable discarded after Finish().
  • Index builders: All three index builder types (ShortenedIndexBuilder, HashIndexBuilder, PartitionedIndexBuilder) implement NumUniformIndexBlocks(). For partitioned indexes, the count accumulates across partition Finish() calls.
  • Table property serialization: Added TablePropertiesNames::kNumUniformBlocks ("rocksdb.num.uniform.blocks") with full serialization/deserialization support. Without this, the property was computed in memory but never persisted to SST files.
  • Table property aggregation: num_uniform_blocks included in Add() and GetAggregatablePropertiesAsMap() for correct cross-SST aggregation.

Test Plan

  • Unit tests: block_test (7504 passed), table_test (6917 passed), table_properties_collector_test (8 passed).
  • End-to-end verification via db_bench + sst_dump:
    • Positive: ./db_bench --benchmarks=fillseq,compact --num=10000 --uniform_cv_threshold=0.2 --index_shortening_mode=0 --compression_type=none produces SST with # uniform blocks: 1.
    • Negative: Same with --uniform_cv_threshold=-1 (disabled) produces # uniform blocks: 0.

@meta-cla meta-cla bot added the CLA Signed label Mar 26, 2026
@meta-codesync
Copy link

meta-codesync bot commented Mar 26, 2026

@joshkang97 has imported this pull request. If you are a Meta employee, you can view this in D98343170.

@github-actions
Copy link

github-actions bot commented Mar 26, 2026

⚠️ clang-tidy: 1 warning(s) on changed lines

Completed in 126.3s.

Summary by check

Check Count
cert-err58-cpp 1
Total 1

Details

table/table_properties.cc (1 warning(s))
table/table_properties.cc:287:41: warning: initialization of 'kNumUniformBlocks' with static storage duration may throw an exception that cannot be caught [cert-err58-cpp]

@joshkang97 joshkang97 force-pushed the add_uniform_block_tracking branch from 64f4df6 to cb10d07 Compare March 26, 2026 21:16
@joshkang97 joshkang97 marked this pull request as ready for review March 26, 2026 21:17
@joshkang97 joshkang97 force-pushed the add_uniform_block_tracking branch from cb10d07 to f369465 Compare March 26, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant