feat: staking end-block queue optimization#26023
Open
randy-cro wants to merge 7 commits intocosmos:mainfrom
Open
feat: staking end-block queue optimization#26023randy-cro wants to merge 7 commits intocosmos:mainfrom
randy-cro wants to merge 7 commits intocosmos:mainfrom
Conversation
Contributor
Greptile SummaryThis PR optimizes staking end-block processing by replacing expensive iterator-based queue scanning with a pending slot index. Instead of iterating from time 0 to current time for each queue (validators, unbonding delegations, redelegations), the code now maintains an index of slots that have entries and only checks those specific slots during end-block. Key Changes:
Issues Found:
Confidence Score: 4/5
Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
Start[End-Block Starts] --> GetSlots[Get Pending Slots from Index]
GetSlots --> CheckEmpty{Slots Empty?}
CheckEmpty -->|Yes| Done[Return - No Work]
CheckEmpty -->|No| IterSlots[Iterate Pending Slots]
IterSlots --> CheckMature{Slot Mature?<br/>time <= currTime<br/>height <= currHeight}
CheckMature -->|No| IterSlots
CheckMature -->|Yes| GetQueue[Get Queue Entry for Slot]
GetQueue --> CheckNil{Queue Entry<br/>Exists?}
CheckNil -->|No - Already Deleted| RemoveSlot[Remove from Pending Index]
RemoveSlot --> IterSlots
CheckNil -->|Yes| ProcessEntries[Process Queue Entries<br/>Unbond Validators/Delegations]
ProcessEntries --> DeleteQueue[Delete Queue Entry]
DeleteQueue --> UpdatePending[Update Pending Slots<br/>Remove Processed Slot]
UpdatePending --> IterSlots
IterSlots --> AllDone{More Slots?}
AllDone -->|No| SaveIndex[Save Updated Pending Index]
SaveIndex --> Complete[Complete]
style Start fill:#e1f5e1
style Complete fill:#e1f5e1
style GetSlots fill:#e3f2fd
style UpdatePending fill:#fff9c4
style SaveIndex fill:#fff9c4
Last reviewed commit: 612cf7c |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #26023 +/- ##
==========================================
- Coverage 59.88% 59.85% -0.04%
==========================================
Files 981 967 -14
Lines 65140 64439 -701
==========================================
- Hits 39011 38570 -441
+ Misses 26129 25869 -260
... and 20 files with indirect coverage changes 🚀 New features to boost your workflow:
|
Contributor
|
@randy-cro looks like the linter is failing! |
Contributor
Author
fixed! |
randy-cro
force-pushed
the
fix/staking-iterators
branch
from
758e1e9 to
9d9a4fd
Compare
randy-cro
force-pushed
the
fix/staking-iterators
branch
from
9d9a4fd to
773c8da
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pending Queue Slots Optimization
Overview
The pending queue slots optimization is a performance improvement for the staking module's end-block processing. Instead of using expensive iterators to scan through all queue entries (which can be very slow on archival nodes with large amounts of historical data), the system maintains compact indexes of pending queue slots that need to be processed.
This optimization eliminates the need for full-range iteration in end-block, replacing it with direct key-value lookups using
Get/Setoperations, which are significantly faster.Architecture
The optimization maintains three separate pending slot indexes:
(time, height)slots for validator unbondingtimeslots for unbonding delegationstimeslots for redelegationsEach index is stored as a single key-value pair in the KV store, with a compact binary encoding that minimizes storage overhead.
Store Keys
The pending slot indexes are stored under the following keys (defined in
x/staking/types/keys.go):ValidatorQueuePendingSlotsKey = []byte{0x44}- Validator queue pending slotsUBDQueuePendingSlotsKey = []byte{0x45}- UBD queue pending slotsRedelegationQueuePendingSlotsKey = []byte{0x46}- Redelegation queue pending slotsBinary Encoding Format
Constants
The encoding uses the following constants (defined in
x/staking/keeper/pending_queue_slots.go):Validator Queue Pending Slots
The validator queue tracks both time and height, requiring a more complex encoding.
Byte Layout
Where:
uint32in big-endian format representing the number of slotsuint64in big-endian format representing Unix nanoseconds since epochuint64in big-endian format representing the block heightExample
For 2 slots:
2024-01-01 00:00:00 UTC(1704067200000000000 nanoseconds), Height = 1002024-01-02 00:00:00 UTC(1704153600000000000 nanoseconds), Height = 200Detailed Breakdown
Header (bytes 0-3): Count of slots
00 00 00 02= 2 slotsSlot 1 (bytes 4-19):
17 04 06 72 00 00 00 00= 1704067200000000000 nanoseconds00 00 00 00 00 00 00 64= 100Slot 2 (bytes 20-35):
17 04 15 36 00 00 00 00= 1704153600000000000 nanoseconds00 00 00 00 00 00 00 C8= 200Total Size Calculation
For
Nvalidator queue slots:UBD Queue Pending Slots
The UBD (Unbonding Delegation) queue tracks only time, using a simpler encoding.
Byte Layout
Where:
uint32in big-endian format representing the number of slotsuint64in big-endian format representing Unix nanoseconds since epochExample
For 3 slots:
2024-01-01 00:00:00 UTC(1704067200000000000 nanoseconds)2024-01-02 00:00:00 UTC(1704153600000000000 nanoseconds)2024-01-03 00:00:00 UTC(1704240000000000000 nanoseconds)Detailed Breakdown
Header (bytes 0-3): Count of slots
00 00 00 03= 3 slotsSlot 1 (bytes 4-11):
17 04 06 72 00 00 00 00= 1704067200000000000 nanosecondsSlot 2 (bytes 12-19):
17 04 15 36 00 00 00 00= 1704153600000000000 nanosecondsSlot 3 (bytes 20-27):
17 04 24 00 00 00 00 00= 1704240000000000000 nanosecondsTotal Size Calculation
For
NUBD queue slots:Redelegation Queue Pending Slots
The redelegation queue uses the same encoding format as the UBD queue (time-only).
Byte Layout
Identical to UBD queue:
Example
Same format as UBD queue example above.
Total Size Calculation
For
Nredelegation queue slots:Data Structure Properties
Sorting
All pending slot lists are maintained in sorted order:
This ensures efficient processing during end-block, as slots can be processed in order and early termination is possible when encountering future slots.
Uniqueness
Duplicate slots are automatically deduplicated when setting the pending slots:
(time, height)Empty Lists
When a pending slot list becomes empty, the key is deleted from the store rather than storing an empty byte array. This minimizes storage overhead.
Usage in End-Block
Validator Queue Processing
The end-block handler (
UnbondAllMatureValidators) uses the pending slots as follows:GetValidatorQueuePendingSlots()retrieves all pending(time, height)slotsslot.Height <= blockHeightANDslot.Time <= blockTimeGetValidatorQueueKey(slot.Time, slot.Height)and perform a directGetoperationRemoveValidatorQueuePendingSlot()This replaces the previous approach of iterating through all keys with the
ValidatorQueueKeyprefix.UBD Queue Processing
Similar pattern for unbonding delegations:
GetUBDQueuePendingSlots()retrieves all pending time slotsslot.Time <= blockTimeGetUnbondingDelegationTimeKey(slot.Time)and perform a directGetoperationRedelegation Queue Processing
Same pattern as UBD queue:
GetRedelegationQueuePendingSlots()retrieves all pending time slotsslot.Time <= blockTimeGetRedelegationTimeKey(slot.Time)and perform a directGetoperationSlot Management
Adding Slots
When a new queue entry is created, the corresponding slot is added to the pending list:
AddValidatorQueuePendingSlot(ctx, endTime, endHeight)AddUBDQueuePendingSlot(ctx, completionTime)AddRedelegationQueuePendingSlot(ctx, completionTime)These functions:
Removing Slots
When a queue entry is processed or deleted, the slot is removed:
RemoveValidatorQueuePendingSlot(ctx, endTime, endHeight)Migration
The migration from v5 to v6 (
x/staking/migrations/v6/store.go) populates the pending slot indexes by:This ensures that the optimization is immediately effective after the upgrade, without requiring a full iteration in the first end-block after upgrade.
Performance Benefits
Before Optimization
After Optimization
Getper pending slot, plus oneGetfor the pending slots listTypical Improvement
On an archival node with millions of historical queue entries but only dozens of pending slots, the improvement can be orders of magnitude faster, reducing end-block processing time from seconds to milliseconds.
Summary
The pending queue slots optimization provides a significant performance improvement for staking module end-block processing by:
The binary encoding is designed for efficiency, using fixed-size fields and big-endian byte order for consistent parsing across different architectures.