-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Motivation
Currently this library relies on gateways which are provided by the user at initialization. However, this should not be the case as the whole point of using IPFS in the first place is that you can defer to the decentralized IPFS network of nodes which are replicating data to ensure redundancy (when data is replicated) and speed ups from multiple providers hosting the same data (similar to Bittorrent). The "issue" is that currently if the user doesn't have a gateway they must spin up their own local IPFS node in order to make any requests which is clunky. The user shouldn't have to be expected to get access to a gateway or run their own external IPFS node and should be able to make IPFS requests directly from the library itself similar to https://github.com/ipfs/helia-verified-fetch.
User Story
As a user I should be able to add a flag to conditionally start up a native libp2p/ipfs node to direct make requests on the network from the DHT/Bitswap in addition to the gateway (similar to Helia).
Acceptance Criteria
- Integrate most reasonable libp2p node implementation for ipfs native requests while keeping gateway functionality (racing/run in parallel)
Things to Investigate
- https://github.com/libp2p/rust-libp2p See the status so that it can be called from python via a FFI
- There is the alternative of go-libp2p which can compile to WASM and also be called from python but this is usually a clunkier experience and networking doesn't work as well Attempt to compile WASM number 2 libp2p/go-libp2p#3360
- Ideally we could use a python native libp2p node https://github.com/libp2p/py-libp2p which has had a lot of recent work. That said, rust is usually more performant, it's been around for quite a bit longer (and therefore the rough edges w.r.t memory leaks/connectivity would likely be more fleshed out than the rust version)