Architecture Questions (Node/Client) #527
Description
I'm writing this issue after speaking with @rustaceanrob on a dev call so I have some answers to these questions, but I still wanted to take the time to put them in writing, if only to allow others to find it and/or refer to it and opine on it.
I'm basically looking to better understand what is the upside of using the node/client paradigm in Kyoto. Keep in mind that my experience with it is mostly through the bdk language bindings so there might be things I have not run into.
The current API, as I understand it, offers a builder pattern that returns to the user a node as well as a client. The node is meant to handle peer to peer connections and filter/block downloads, and the client allows users to speak to and command the node. The API on the node construct however (at the ffi layer not the Rust layer), consists of a single method, node.run(). Everything else is handled by the client. What's more, this split between the node and the client initially gave me the impression that the node was independent from the client/clients it might serve, but this is not the case; in general, a node will start downloading filters on startup (node.run()) and needs an associated client to check those filters against its list of interested spks so that the node can purge/drop the filters. If there are no clients there ready to do this, the node will simply attempt to download all filters (up to 12GB for a genesis-sync on mainnet) and run out of memory, potentially crashing the process on an OOM error.
Given the importance of the two constructs working together to accomplish the task of "syncing a wallet" (I know that's a bit handwavy sorry), and the fact that running a node is a given if you have a client attempting to sync a wallet, I'm wondering why the two are not simply combined into one object that handles it's own node running in the background (could also be triggered by this new single object) without requiring the users of the library to start the node and then start the sync on the client.
Related to this are questions like "are there methods on the node we would not want to simply pass to it through the client?" Maybe in a situation where a node handled multiple clients? Otherwise, why not simply use the client to handle all node-related configs and methods?
In my conversation with Rob earlier I think he mentioned allowing users to decide where to run the node process. I wonder if it might be possible to pass this as an argument to the NodeClient builder or on the ::run() method.
Example API
In this example I'm naming this combined node + client structure Kyoto.
// bdk_kyoto like API
let kyoto: Kyoto = Kyoto::new(connections, data_dir, scan_type, etc.)
kyoto.run()
let update = kyoto.update()
// All other methods currently on the node are simply available on the Kyoto object