onionmessage: graph-based pathfinding for onion messages

[Abdulkbk]

Change Description

Part of #10220
Depends on #10089

This PR builds on top of #10089 (onion message forwarding) and is part of the onion message tracking issue #10220. The forwarding layer can relay messages hop-by-hop, but a sender still needs a way to find a path through the graph to a destination. This PR adds that piece.

A FindPath function in onionmessage/pathfind.go that discovers a route from a local node to a destination through nodes that advertise onion message support (feature bit 38/39, OnionMessagesOptional/OnionMessagesRequired).

We use Breadth-First Search (BFS) over hop count. Since onion messages are not priced by fees or liquidity, the meaningful routing metric is path length. Shorter paths reduce the relay burden on intermediate nodes and lower latency. BFS naturally finds the minimum-hop path, which is appropriate for the channel graph. Intermediate nodes are included only if they advertise support for onion messages (bits 39/38).

A set of unit tests is included to test for various scenarios.

[gemini-code-assist]

Summary of Changes

Hello @Abdulkbk, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly advances onion message capabilities by introducing a graph-based pathfinding mechanism. It refactors the internal actor system for more robust message handling and establishes a dedicated endpoint for processing and forwarding onion messages. These changes lay the groundwork for more sophisticated onion message interactions within the network, allowing nodes to efficiently discover and utilize paths while adhering to new feature signaling standards.

Highlights

• Onion Message Pathfinding: Introduced a FindPath function in onionmessage/pathfind.go that uses Breadth-First Search (BFS) to discover minimum-hop routes for onion messages. This function filters the channel graph to include only nodes that advertise onion message support (feature bits 38/39).
• Actor System Refactoring: Refactored the actor package to introduce a Mailbox interface and a ChannelMailbox implementation, improving message queue management and actor lifecycle handling. This change streamlines message processing and draining during actor shutdown.
• Onion Message Endpoint and Peer Actors: Added a new onionmessage/endpoint.go to handle incoming onion messages, including logic for forwarding messages to the next hop or delivering them locally. This is complemented by onionmessage/actor.go, which introduces OnionPeerActor to manage sending onion messages to specific peers, integrating with the new actor system.
• New Feature Bits and Protocol Options: Defined OnionMessagesRequired (38) and OnionMessagesOptional (39) feature bits in lnwire/features.go to signal support for onion messaging. A new protocol.no-onion-messages configuration option was added to allow disabling this feature.
• RPC and Wire Protocol Updates: Updated the lnrpc and lnwire packages to support the new onion message payload structure, including fields for reply paths, encrypted recipient data, and custom TLV records. The SubscribeOnionMessages RPC now returns a more detailed OnionMessageUpdate.
• Integration Tests: Added comprehensive integration tests (itest/lnd_onion_message_forward_test.go) to validate onion message forwarding across various scenarios, including forwarding by node ID, by Short Channel ID (SCID), and with concatenated blinded paths.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• .golangci.yml
  • Added github.com/lightningnetwork/lightning-onion to replace-local configuration.
• actor/actor.go
  • Replaced direct channel usage with a new Mailbox interface for actor message queues.
  • Updated NewActor to initialize with NewChannelMailbox.
  • Refactored process, Tell, and Ask methods to leverage the Mailbox interface for message handling and draining.
• actor/mailbox.go
  • Added new file defining the Mailbox interface and its ChannelMailbox implementation, providing methods for sending, trying to send, receiving, closing, and draining messages.
• actor/mailbox_test.go
  • Added new file with comprehensive unit tests for the ChannelMailbox implementation, covering send, try send, receive, close, drain, concurrent operations, zero capacity, and actor context cancellation.
• docs/release-notes/release-notes-0.21.0.md
  • Updated the description for onion messaging forwarding to clarify the new message type and serialization logic.
• feature/default_sets.go
  • Added lnwire.OnionMessagesOptional to the default feature sets for initialization and node announcements.
• feature/manager.go
  • Added a NoOnionMessages boolean field to the Config struct to control onion message feature advertisement.
  • Implemented logic in newManager to unset OnionMessagesOptional and OnionMessagesRequired feature bits if NoOnionMessages is enabled.
• go.mod
  • Updated github.com/lightningnetwork/lightning-onion to v1.3.0.
  • Added github.com/lightningnetwork/lnd/actor v0.0.3 as a new dependency.
  • Updated github.com/cpuguy83/go-md2man/v2 to v2.0.4.
  • Updated github.com/russross/blackfriday/v2 to v2.1.0.
  • Added a replace directive for github.com/lightningnetwork/lnd/actor to point to the local ./actor path.
• go.sum
  • Updated checksums to reflect changes in go.mod dependencies.
• htlcswitch/hop/fuzz_test.go
  • Updated sphinx.MaxPayloadSize to sphinx.MaxRoutingPayloadSize in fuzz tests.
  • Changed sphinx.HopPayload to *sphinx.HopPayload in fuzz tests to align with updated lightning-onion library.
• itest/list_on_test.go
  • Added testOnionMessageForwarding to the list of all integration test cases.
• itest/lnd_onion_message_forward_test.go
  • Added new file containing integration tests for various onion message forwarding scenarios, including forwarding by node ID, SCID, and concatenated paths.
• itest/lnd_onion_message_test.go
  • Updated testOnionMessage to use lnrpc.OnionMessageUpdate for subscribing to messages.
  • Modified the test to build a valid blinded onion message using onionmessage.BuildOnionMessage and record.BlindedRouteData.
• lncfg/protocol.go
  • Added NoOnionMessagesOption boolean field to ProtocolOptions to disable onion message forwarding.
  • Added NoOnionMessages() method to ProtocolOptions to check if onion messaging is disabled.
• lncfg/protocol_integration.go
  • Added NoOnionMessagesOption boolean field to ProtocolOptions for integration tests.
  • Added NoOnionMessages() method to ProtocolOptions for integration tests.
• lnrpc/lightning.proto
  • Changed the return type of SubscribeOnionMessages RPC from OnionMessage to OnionMessageUpdate.
  • Renamed OnionMessage message to OnionMessageUpdate.
  • Added reply_path, encrypted_recipient_data, and custom_records fields to OnionMessageUpdate.
• lnrpc/lightning.swagger.json
  • Updated Swagger definitions to reflect the renaming of lnrpcOnionMessage to lnrpcOnionMessageUpdate.
  • Added reply_path, encrypted_recipient_data, and custom_records properties to the lnrpcOnionMessageUpdate definition.
• lnrpc/lightning_grpc.pb.go
  • Updated gRPC client and server interfaces (Lightning_SubscribeOnionMessagesClient, Lightning_SubscribeOnionMessagesServer) to use OnionMessageUpdate.
• lnwire/features.go
  • Added OnionMessagesRequired (38) and OnionMessagesOptional (39) feature bits to lnwire.
  • Mapped new onion message feature bits to their string representations in the Features map.
• lnwire/onion_msg_payload.go
  • Added new file defining OnionMessagePayload structure for encoding/decoding onion message TLVs.
  • Introduced FinalHopTLV to represent TLVs reserved for the final hop.
  • Implemented encoding and decoding logic for OnionMessagePayload, including handling ReplyPath, EncryptedData, and FinalHopTLVs.
• msgmux/msg_router.go
  • Updated comment for Router interface, changing MsgRouter to Router.
• onionmessage/actor.go
  • Added new file defining Request and Response messages for onion peer actors.
  • Introduced OnionPeerActorRef and OnionPeerActor for managing onion message sending to individual peers.
  • Implemented SpawnOnionPeerActor and StopPeerActor functions to manage actor lifecycle for peers.
• onionmessage/actor_test.go
  • Added new file with unit tests for OnionPeerActor functionality, including message sending, actor lookup, and lifecycle management.
• onionmessage/endpoint.go
  • Added new file defining OnionEndpoint as a msgmux.Endpoint to handle incoming lnwire.OnionMessages.
  • Implemented SendMessage logic to process onion messages, decrypt blinded hop data, determine routing action (forward or deliver), and send updates to subscribers.
  • Introduced forwardMessage to send processed onion messages to the next hop via OnionPeerActor.
• onionmessage/endpoint_test.go
  • Added new file with unit tests for OnionEndpoint, covering various routing scenarios like forwarding by node ID, SCID, concatenated paths, and handling invalid messages.
• onionmessage/errors.go
  • Added new file defining specific error types for onion message operations, such as ErrActorShuttingDown, ErrNextNodeIdEmpty, ErrSCIDEmpty, ErrNilReceptionist, ErrNilRouter, ErrNilResolver, ErrNoPathFound, and ErrDestinationNoOnionSupport.
• onionmessage/hop.go
  • Added new file defining forwardAction and deliverAction structs for routing decisions.
  • Introduced NodeIDResolver interface for resolving node public keys from SCIDs.
  • Implemented processOnionMessage to decrypt and interpret onion message packets, deriving next hop information and path keys.
• onionmessage/hop_test.go
  • Added new file with unit tests for processOnionMessage, isForwarding, and deriveNextPathKey functions, covering different routing scenarios and path key derivation logic.
• onionmessage/onion_endpoint.go
  • Removed file, its functionality was replaced and expanded by onionmessage/endpoint.go.
• onionmessage/pathfind.go
  • Added new file implementing FindPath function for onion messages, utilizing BFS to find shortest paths based on hop count and filtering by onion message feature support.
• onionmessage/pathfind_test.go
  • Added new file with unit tests for the FindPath function, covering direct neighbors, multi-hop paths, feature filtering, disconnected graphs, max hop limits, cycles, and shortest path selection.
• onionmessage/resolver.go
  • Added new file defining GraphNodeResolver to resolve node public keys from SCIDs using the channel graph and an LRU cache for efficiency.
• onionmessage/resolver_test.go
  • Added new file with unit tests for mockNodeIDResolver to verify SCID to pubkey resolution.
• onionmessage/test_utils.go
  • Added new file containing helper functions and mock implementations for onion message testing, including mockNodeIDResolver, EncodeBlindedRouteData, BuildBlindedPath, ConcatBlindedPaths, BuildOnionMessage, and PeelOnionLayers.
• peer/brontide.go
  • Updated Config struct to include SphinxPayment (renamed from Sphinx), OnionEndpoint, and ActorSystem.
  • Modified Start method to spawn an OnionPeerActor for remote peers supporting onion messages and register the OnionEndpoint with the message router.
  • Updated Disconnect method to stop the OnionPeerActor associated with the disconnected peer.
  • Updated addLink to use SphinxPayment for HTLC processing.
  • Added lnwire.OnionMessage handling to messageSummary for better logging.
• peer/test_utils.go
  • Updated createTestPeer to initialize sphinx.Router, subscribe.Server, actor.ActorSystem, and onionmessage.OnionEndpoint for test peer configurations.
  • Added noopNodeIDResolver for tests that do not require SCID resolution.
• record/blinded_data.go
  • Added NewNonFinalBlindedRouteDataOnionMessage function to create blinded route data specifically for onion messages, supporting next node ID, short channel ID, blinding override, and features.
• record/hop.go
  • Added new TLV types for onion message packets: ReplyPathType, EncryptedDataTLVType, InvoiceRequestNamespaceType, InvoiceNamespaceType, and InvoiceErrorNamespaceType.
  • Introduced corresponding New...Record functions for these new onion message TLV types.
• routing/pathfind.go
  • Updated sphinx.MaxPayloadSize to sphinx.MaxRoutingPayloadSize when checking routing info size limits during pathfinding.
• routing/route/blindedroute.go
  • Added new file with OnionMessageBlindedPathToSphinxPath function to convert a blinded path for onion messages into a Sphinx PaymentPath, including handling final hop TLVs and reply paths.
• rpcserver.go
  • Modified SubscribeOnionMessages RPC handler to send lnrpc.OnionMessageUpdate messages, populating new fields like ReplyPath, EncryptedRecipientData, and CustomRecords.
• sample-lnd.conf
  • Added a new configuration option protocol.no-onion-messages to disable onion message support.
• server.go
  • Updated the server struct to include sphinxPayment (renamed from sphinx), sphinxOnionMsg, onionEndpoint, and actorSystem.
  • Initialized a dedicated sphinx.Router (sphinxOnionMsg) for onion messages without replay protection.
  • Integrated NoOnionMessages protocol option into feature manager configuration.
  • Created and initialized onionmessage.OnionEndpoint with GraphNodeResolver if onion messaging is not disabled.
  • Updated peerConnected to pass the new SphinxPayment, OnionEndpoint, and ActorSystem to the peer configuration.
  • Ensured proper shutdown of sphinxOnionMsg and actorSystem during server stop.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[Abdulkbk]

The newly added commit is 744769f.

[gemini-code-assist]

Code Review

This pull request introduces graph-based pathfinding for onion messages and refactors the actor model's mailbox implementation. My review focuses on improving code clarity, documentation, and robustness in the new onionmessage and related packages. I've suggested simplifying a boolean flag in the message endpoint, correcting a function comment to align with the style guide, adding a missing function comment, and adjusting a length check for more precise error handling in payload decoding. Overall, the changes are well-structured and the new functionality is thoroughly tested.

[lightninglabs-deploy]

@Abdulkbk, remember to re-request review from reviewers when ready

[Abdulkbk]

Rebased on master since #10089 is merged