Skip to content

Commit 6ee59d0

Browse files
author
Nathan
committed
UCT/IB/RC: add adaptive TX CQ moderation
When many RC endpoints each send fewer messages than the tx_moderation threshold, no signaled sends are generated and TX CQ credits are never reclaimed, eventually exhausting the TX buffer pool. Fix by also checking iface-level cq_available in the moderation decision. When CQ credits drop to the tx_moderation threshold, force the next send on any endpoint to be signaled, ensuring timely completion processing regardless of per-endpoint send counts. Closes #1307
1 parent 6dda7bd commit 6ee59d0

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

src/uct/ib/rc/base/rc_ep.h

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -469,7 +469,9 @@ uct_rc_txqp_completion_inl_resp(uct_rc_txqp_t *txqp, const void *resp, uint16_t
469469
static UCS_F_ALWAYS_INLINE uint8_t
470470
uct_rc_iface_tx_moderation(uct_rc_iface_t *iface, uct_rc_txqp_t *txqp, uint8_t flag)
471471
{
472-
return (txqp->unsignaled >= iface->config.tx_moderation) ? flag : 0;
472+
return (txqp->unsignaled >= iface->config.tx_moderation) ||
473+
(iface->tx.cq_available <= (signed)iface->config.tx_moderation)
474+
? flag : 0;
473475
}
474476

475477
static UCS_F_ALWAYS_INLINE void

0 commit comments

Comments
 (0)