UCT/IB/RC: add adaptive TX CQ moderation#11305
Open
ndg8743 wants to merge 1 commit intoopenucx:masterfrom
Open
UCT/IB/RC: add adaptive TX CQ moderation#11305ndg8743 wants to merge 1 commit intoopenucx:masterfrom
ndg8743 wants to merge 1 commit intoopenucx:masterfrom
Conversation
ndg8743
force-pushed
the
fix/adaptive-tx-cq-moderation
branch
from
6ee59d0 to
4052e51
Compare
Author
|
CI failure on ASAN new worker 2:
Retriggering CI. |
ndg8743
force-pushed
the
fix/adaptive-tx-cq-moderation
branch
from
0a8c53b to
4052e51
Compare
When many RC endpoints each send fewer messages than the tx_moderation threshold, no signaled sends are generated and TX CQ credits are never reclaimed, eventually exhausting the TX buffer pool. Fix by also checking iface-level cq_available in the moderation decision. When CQ credits drop to the tx_moderation threshold, force the next send on any endpoint to be signaled, ensuring timely completion processing regardless of per-endpoint send counts. Closes openucx#1307
ndg8743
force-pushed
the
fix/adaptive-tx-cq-moderation
branch
from
4052e51 to
e1e4199
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tx_moderation, no signaled sends are generated and TX CQ credits are never reclaimed, exhausting the TX buffer pool.uct_rc_iface_tx_moderation()to also check iface-levelcq_available: when CQ credits drop to thetx_moderationthreshold, force the next send on any endpoint to be signaled.Closes #1307