Add multipath #28
Add multipath #28
Conversation
divagant-martian
force-pushed
the
multipath-quinn-0.11.x
branch
from
6f37612 to
f5485c2
Compare
divagant-martian
force-pushed
the
multipath-quinn-0.11.x
branch
from
6fa6b81 to
f3f1a44
Compare
|
Hi there! |
|
@MikeRomaniuk This is still very early right now, maybe like the first 5% if that. We are still in the process of restructuring things for the basic building blocks for multipath to be there. If you want to help you can take a look at the code so far, but it is currently still rather difficult to point at anything. It will probably be easier to jump in once we have the most basic 2-path connection working. But if you're keen to really help, right now it is mostly a case of trying to understand what would be a useful next step based on the specs and code. And then try and implement some of it. With just two of us this seems to move us forwards so far. |
|
@MikeRomaniuk Can I inquire as to what your usecase is? |
|
@flub, thanks for answering. |
|
@MikeRomaniuk The usecase does kind of matter, because we have a very specific usecase in mind (documented in my fosdem talk, though the recording is terrible - i should re-record that sometime). We want to do the absolute minimum we can do for our usecase, and this keeps the number of public APIs that will be needed small as well. The multipath spec does give you enough to make a largely interoperable protocol, but also still leaves a lot of things as essentially further research topics. The largest one is probably packet scheduling, also things like how to interoperate with the ack-frequency spec and probably a bunch more that escape me right now. So what you want to get out of this does matter in that regard. |
|
@flub, I might have understood your speech in the video wrong, so feel free to correct me. |
|
@MikeRomaniuk Yes, more or less. Our "typical" case is to establish the connection via a relayed path, we'll use an IPv6 private address space IP to identify paths via the relay and handle that in our impl of Quinn's AsyncUdpSockt trait. This path will be set as PATH_BACKUP. Then combine draft-seeman-quic-nat-traversal with multipath to open another path using real IP addresses and mark that path as PATH_AVAIALBE. This will give us very simple packet scheduling semantics, I think the multipath part of this it's about the simplest version of multipath you can build. Once we have this working stable and reliable we will probably start looking at more advanced packet scheduling, there are a lot of possible things, as you say maximise bandwith by using multiple paths, or interactions with lost packets etc. But that is much further down the line for us, we really want to have the simplest multipath first. |
|
@flub, thanks for clarification. |
|
@MikeRomaniuk We would like to get to a stage where we can schedule packets on multiple paths at the same time to increase bandwidth. But as I explained for us that's not a high priority and not yet on our roadmap. But that doesn't mean we wouldn't love it if someone else figured out what good behaviour and a nice API is there. So we'd totally accept contributions. (Though at this stage packet scheduling is still some way away.) I'm not sure what your goal is here, because tquic already has multipath and claims to care about performance (I have never tried it). Quinn is not the highest performant QUIC stack around so might be worth to benchmark even just single-path connections to see if they meet your goals. We also had other considerations like being able to use the Lastly I'd like to repeat what this repo says in the readme: we intent to upstream anything that's not a gross hack and would discourage anyone from depending on us directly because we may make changes not very nice to users. Just be aware of that. |
|
@flub I am very grateful for your explanations.
Honestly, I would rather use quinn over tquic, since it is widely used in the community and I believe will be a better choice in the longrun. I will need to talk to the team to take into account their opinion and I will let you if we would decide to proceed our way with you. |
|
@MikeRomaniuk Hello, I'm also working on an MPQUIC-based project (specifically for my bachelor's final year project), focusing on developing custom MPQUIC schedulers. I've found that the event-based You might be interested in @qdeconinck's fork of I'm curious about which direction you're planning to take! Would love to collaborate on this problem if you're interested. |
|
@shirok1 thanks for the recommendation! I am very glad that you are interested in QUIC and the problem I am about to tackle. However, I cannot invite you to participate, since this is not my personal project. |
Did you progress on your evaluation? Would be good to know either way what the outcome is. |
|
Hi @flub! |
@MikeRomaniuk No worries, all the best! |
Bumps [socket2](https://github.com/rust-lang/socket2) from 0.5.9 to 0.5.10. - [Release notes](https://github.com/rust-lang/socket2/releases) - [Changelog](https://github.com/rust-lang/socket2/blob/master/CHANGELOG.md) - [Commits](https://github.com/rust-lang/socket2/commits) --- updated-dependencies: - dependency-name: socket2 dependency-version: 0.5.10 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>
Bumps [async-io](https://github.com/smol-rs/async-io) from 2.4.0 to 2.4.1. - [Release notes](https://github.com/smol-rs/async-io/releases) - [Changelog](https://github.com/smol-rs/async-io/blob/master/CHANGELOG.md) - [Commits](smol-rs/async-io@v2.4.0...v2.4.1) --- updated-dependencies: - dependency-name: async-io dependency-version: 2.4.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>
Bumps [criterion](https://github.com/bheisler/criterion.rs) from 0.5.1 to 0.6.0. - [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md) - [Commits](bheisler/criterion.rs@0.5.1...0.6.0) --- updated-dependencies: - dependency-name: criterion dependency-version: 0.6.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>
I didn't understand this fully at first and was worried that the places where a loss timer was set did not always correspond with the path they should be set for. Having reviewed this I think it's doing the correct thing for the paths. Another benefit is that multiple loss-detection timers can now expire and be handled in the same poll_timers() call, which is a little more efficient. And it also simplifies the code a bunch, so that's good. Maybe it is even worth making this timers per-space as well?
…pSocket` in connections
…recv` take `&mut self`
This reverts commit 44a3dc6.
This also includes removing async-std support. These are commits on top of which Phillip's ones are made
* Add OpenPath::path_id and some docs * Try and word this a little better
The previous logic does not work: the client has the path validated immediately, even before any packet is sent. Now we need to have received an authenticated handshake packet before we stop allowing migration. This exposes us to a little more off-path attacking, but in terms of processing all the earlier packets: only the correct server can generate an authenticated handshake packet IIUC.
* Fix WeakConnection::upgrade It created a ConnectionRef that did not increment the manually-tracked reference count inside the Arc<ConnectionInner>. This triggered the ConnectionRef's Drop impl to actually drop the connection. This works around this by using the fact that the ConnectionRef's Clone impl does know how to manipulate the reference count in a way that works together with the ConnectionRef's Drop. I think this is preferred over manipulating the ref_count directly in the WeakConnectionHandle since that code is closer by. * Nicer implementation
This in the same style as the other frames are logged. It's very confusing to not have a trace log of this.
If only tracing would be type-safe. But yeah, strings are fiiiine
When building packets in GSO batches the batch is terminated if too many padding bytes would be needed to continue this batch. This was chosen fairly arbitrarily to be 16 bytes. However when you have multipath enabled and allow a half-decent number of simulatneous paths to be opened, you have to issue 5 * initial_max_path_id PATH_NEW_CONNECTION_ID frames. This quickly spills over into multiple datagrams, especially since this happens early in the connection when the PMTU has not yet been discovered. CIDs however, are 32 bytes by default. So if such a frame does no longer fit in a datagram you would easily need more than 16 bytes of padding. And in that case the GSO batch would break. Even though all you're doing is sending a series of datagrams full of CIDs. So tweaking this number to 32 allows us to build these packes without breaking the GSO batch.
* Fix remote addresses in paths - Remove quinn_proto::Connection::remote_address. You should use path_remote_address now. - Change quinn::Connection::remote_address to return *any* remote address from any of the open paths. - Fix opening paths on IPv4 addresses, when the endpoint supports IPv6. When the endpoint supports IPv6 it uses IPv4-mapped IPv6 addresses for IPv4 remotes. This is enforced when establishing a connection, and quinn-udp automatically does this for incoming datagrams. However when opening a path we should do the same mapping. Otherwise we will end up with an IPv4 remote address in the PathData and incoming datagrams will come from an IPv6 address and be dropped. * fix tests
* remove obsolete dead code annotation * add missing docs for coding and encoding path related frames * make `RetireConnectionId` max size into consts * make `NewConnectionId` max size into consts * remove unwraps comming from n0 members in frame.rs
I messed this up earlier, this is already logged.
* use PathId::ZERO whenever possible * avoid accesing PathId's inner in general * fmt * prefer PathId::MAX over max construction
* calculate this connection's mtu as the minimum across all paths * add TODO to check buffer reserving * improve docs * apply clippy fixes * more docs tweaks * second perspective on TODO * spelling * address TODO: reserving is already done in poll_transmit with the appropiate sizes
20 participants
No description provided.