Askrene parallel solving support #8723
Open
+1,226
−814
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…g an entore response. Signed-off-by: Rusty Russell <[email protected]>
We reimplemented this redundantly: hash_scid was called short_channel_id_hash, so I obviously missed it. Rename, and implement hash_scidd helper too. Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: Rusty Russell <[email protected]>
This is fairly simple. We do all the prep work, fire off the child, and it continues all the way to producing JSON output (or an error). The parent then forwards it. Limitations (fixed in successive patches): 1. Child logging currently gets lost. 2. We wait for the child, so this code is not a speedup. Signed-off-by: Rusty Russell <[email protected]>
We just shim rq_log for now, but we'll be weaning the child process off that soon. Signed-off-by: Rusty Russell <[email protected]>
Lagrang3
approved these changes
Dec 4, 2025
Collaborator
Lagrang3
left a comment
Lagrang3
left a comment
There was a problem hiding this comment.
Very clever parallelization!
We want to make it clear when future generations edit the code, which routines are called in the child (i.e. all the routing), and which in the parent. Signed-off-by: Rusty Russell <[email protected]>
Weird that it was in askrene.c Signed-off-by: Rusty Russell <[email protected]>
Now there's only one file clearly shared by both parent and child. Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: Rusty Russell <[email protected]>
Notably no access to the struct command and struct plugin. Note: we actually *do* mess with askrene->reserves, but the previous code used cmd to get to it. Now we need to include a non-const pointer in struct route_query. Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: Rusty Russell <[email protected]>
Changelog-Changed: Plugins: `askrene` now runs routing in parallel. Signed-off-by: Rusty Russell <[email protected]>
Queue them before we query local channels, so they don't use stale information. Changelog-Added: Config: `askrene-max-threads` to control how many CPUs we use for routing (default 4). Signed-off-by: Rusty Russell <[email protected]>
The fork logic itself is pretty simple, so do that directly in askrene.c, and then call into "run_child()" almost as soon as we do the fork. Signed-off-by: Rusty Russell <[email protected]>
This makes sure it cannot interfere with the parent askrene's connection to lightningd, for example. Signed-off-by: Rusty Russell <[email protected]>
This speeds them up, and exercises the askrene parallel code. Before: test_real_data: 260s test_real_biases: 173s After: test_real_data: 133s test_real_biases: 106s And this is because much of the time is spent uncompressing the gossmap and startup. Signed-off-by: Rusty Russell <[email protected]>
rustyrussell
force-pushed
the
guilt/askrene-parallel
branch
from
c3fb159 to
8ffb1fc
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is actually fairly simple: we fork() at the point we're going to call the solver, and the child runs until it either produces an error message or a JSON result.
The main changes are in refactoring to make it clear which parts of the code run in the child, and which run in the parent.