Skip to content

Conversation

@ziggie1984
Copy link
Collaborator

@ziggie1984 ziggie1984 commented Dec 1, 2025

I analysed a pg related OOM issue and analysing the logs I saw a LOT of this error messages:

2025-11-25 07:41:34.774 [WRN] CRTR: failed to get bandwidth for channel XXX:XX:X: cannot add outgoing htlc to channel XXX:XX:X with amount 0 mSAT: commitment transaction exceed max htlc number
2025-11-25 07:41:34.876 [WRN] CRTR: failed to get bandwidth for channel  XXX:XX:X:: cannot add outgoing htlc to channel  XXX:XX:X with amount 38942412 mSAT: commitment transaction exceed max htlc 
2025-11-25 07:41:34.876 [WRN] CRTR: failed to get bandwidth for channel  XXX:XX:X: cannot add outgoing htlc to channel  XXX:XX:X with amount 0 mSAT: commitment transaction exceed max htlc number
2025-11-25 07:41:34.969 [WRN] CRTR: failed to get bandwidth for channel  XXX:XX:X:: cannot add outgoing htlc to channel XXX:XX:X with amount 19457218 mSAT: commitment transaction exceed max htlc 
2025-11-25 07:41:34.976 [WRN] CRTR: failed to get bandwidth for channel  XXX:XX:X: cannot add outgoing htlc to channel  XXX:XX:X with amount 0 mSAT: commitment transaction exceed max htlc number

what we see above is the reason because we would not distinguish between a channel which is just not in our map or which is currently unusable. That let to a lot of unnecessary retries and pathfinding calls because we would assume full bandwidth when we get this error.

This problem was already documented in 2 TODOs which are now done and removed.

The reason why it only got fixed now is because we changed some logging in this area which now made this problem obvious.

Log-Entries showed like 22000 similar log lines in 5 days (node does a lot of heavy rebalancing)

Replace the (bandwidth, bool) return signature with (bandwidth, error)
to provide more context about why bandwidth is unavailable. This allows
callers to distinguish between:
- Channel not found in local channels map (ErrLocalChannelNotFound)
- Channel found but unusable (offline, HTLC limits, etc.)

The new error-based approach improves error handling throughout the
routing package:
- pathfind: Use capacity fallback only for channels which are not found
  in the local graph map, which can happen when channels were opened and
  activated during the payment process started for example.
- unified_edges: Skip unusable local channels instead of using max bandwidth
- Tests updated to use checkErrIs/checkErrContains for precise validation

This fixes TODO comments about unclear bandwidth hint failures and
improves channel selection by avoiding attempts to route through
channels that cannot carry payments.
@ziggie1984 ziggie1984 force-pushed the fix-sendpaymentv2-performance branch from ce40b53 to 745bcf5 Compare December 1, 2025 18:54
@ziggie1984 ziggie1984 marked this pull request as ready for review December 1, 2025 18:55
@ziggie1984 ziggie1984 added this to the v0.20.1 milestone Dec 1, 2025
@ziggie1984 ziggie1984 self-assigned this Dec 1, 2025
@ziggie1984 ziggie1984 added payments Related to invoices/payments path finding labels Dec 1, 2025
@saubyk saubyk added this to v0.21 Dec 2, 2025
@saubyk saubyk moved this to In progress in v0.21 Dec 2, 2025
@ziggie1984 ziggie1984 modified the milestones: v0.20.1, v0.21.0 Dec 3, 2025
Copy link
Member

@Roasbeef Roasbeef left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🪘

bandwidth = lnwire.NewMSatFromSatoshis(
channel.Capacity,
)
} else {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok, so before we'd assume the capacity and potentially try to route over a channel that may not actually be online. This would be caught later in the switch, but it's better to catch this issue earlier.

"this channel", shortID, bandwidth)
if err != nil {
// If the channel is not in our local channels map, use
// the channel capacity as a fallback. This can happen
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep this is accurate, as this is a per session snapshot. We could try to make this update dynanmically with newly mined channels, but I don't think that's really worth the extra effort.

@saubyk saubyk moved this from In progress to In review in v0.21 Dec 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

path finding payments Related to invoices/payments

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

2 participants