Interest in async runtime integrations?

[bobozaur]

Is there any interest in abstracting the event handling so that instead of a background thread the events could be monitored by the async runtime itself?

The background thread could still be the default and the event handler could probably be a static that holds a trait object. I experimented with something similar for rusb here.

Runtime specific event handlers could live in separate crates, so as not to bloat nusb.

[kevinmehall]

Now that I've added tokio and smol features to enable the corresponding threadpool for blocking operations, it's something to re-consider.

I think it would only be possible on Linux. Tokio and async-io don't do IOCP for Windows or CFRunLoop on macOS. With rusb, libusb is running background threads anyway on those platforms. There very well could be a common runtime crate that can run a shared event loop on those platforms, but it doesn't seem to exist.

And on Linux, nusb calls USBDEVFS_REAP_URB on the event thread and then directly wakes the target waker. If you put Tokio in the middle, Tokio would wait for events on the usbfs FD and wake nusb's task, which would then call USBDEVFS_REAP_URB, and dispatch to the waker for that endpoint. Not sure what kind of latency that additional task switch would add.

On Windows, I'm looking at using the Windows threadpool APIs, which can dispatch IOCP events and timers to callbacks on shared threads managed by Windows APIs. In practice, that wouldn't likely be used by other libraries in a Rust process, so would still be a dedicated thread.

So I don't think it's worth the complexity, and I don't think the overhead of running a background thread matters for anyone. I've actually considered going the opposite direction and running a thread per device to avoid global state, and because a single device at a time is by far the common case.

[bobozaur]

I think it would only be possible on Linux. Tokio and async-io don't do IOCP for Windows or CFRunLoop on macOS.

While I don't know about CFRunLoop, couldn't file descriptors be used there as well? EDIT: By there I meant on macOS.
EDIT2: Looked it up a bit and it seems that would not be possible and IOKit must be used instead, hence the CFRunLoop.

And on Linux, nusb calls USBDEVFS_REAP_URB on the event thread and then directly wakes the target waker...

Maybe I'm missing something as I'm not familiar with nusb or its codebase, but ideally the Tokio runtime would wait for events, call USBDEVFS_REAP_URB and populate the transfer buffer before waking the nusb task.

[kevinmehall]

While I don't know about CFRunLoop, couldn't file descriptors be used there as well? EDIT: By there I meant on macOS.

The iokit APIs provide events by calling a callback on a CFRunLoop. It's likely that it's a Mach port underneath, but in either case, it's not a file descriptor.

Tokio runtime would wait for events, call USBDEVFS_REAP_URB and populate the transfer buffer before waking the nusb task.

Tokio knows how to register a file descriptor with epoll and wake a waker when it becomes ready. It does not know anything about usbdevfs ioctls, so you'd have to do that on an async task that is waiting for the file descriptor event and gets woken by Tokio. Basically the issue is that the device is a single file descriptor, but there may be multiple endpoints or transfers that want to register separate wakers.

[bobozaur]

Ah, I see what you mean now. The last bit about the device being a single file descriptor was my oversight.

I was also a bit stuck thinking in terms of glib, particularly something like this. Tokio is too high level for this, but I believe mio could help with the lower level details of registering the file descriptor for polling and calling USBDEVFS_REAP_URB to avoid the need of another async task.

The single file descriptor multiple waker situation doesn't seem that different from what's currently being done in the event thread using ids. Would that not work in this case too?

[kevinmehall]

Yeah, so you could say I've re-implemented the relevant parts of mio in 150 lines in linux/events.rs, without unsafe thanks to rustix. The IDs and slab there are used to dispatch to the different file descriptors / devices that it's waiting for. Using different wakers for different endpoints is handled implicitly because each URB points to an object that holds a waker that gets woken in notify_completion.

That glib API would indeed allow you to do the same direct wake. You could do the same in a tokio task but it involves an extra wake and task switch (Tokio IO thread -> tokio task from nusb -> another tokio task where the user code is waiting for a transfer). I don't want to say that's a problem without benchmarking it though.

[kevinmehall]

On the topic of premature optimization, I mainly don't see the advantage in adding API complexity to allow it to be pluggable. Calling the event loop thread an implementation detail of each backend keeps things simple and works with any async runtime. On Linux specifically, spawning a thread that sits blocked in epoll_wait is very cheap. Maybe a web server handling hundreds of thousands of requests can't run a thread per request, but if a few libraries spawn a thread to handle events I don't think that's going to be a problem for anyone. What makes the event loop integration important to you?

Most nusb users don't even use an async runtime, and in v0.2, it's not even going to require touching a Future unless you want to. The big thing missing in rusb is that libusb's async API is required to submit multiple transfers on an endpoint, but when designing from scratch that's actually independent of how you wait for those transfers to complete. Nusb v0.2's API lets you submit any number of transfers with a non-blocking API, and wait for them with either a blocking wait or with a Future. The main reasons to choose the nusb async APIs are either that you're forced to (WebUSB), or for more flexible control flow capabilities like select. Unlike a network app, you're never handling thousands of USB endpoints at once, so you don't choose async just because thread-per-task is too expensive.

[bobozaur]

Thank you for taking the time to answer this so diligently.

I understand your reasoning and I agree with it with respect to what I gather that nusb wants to achieve. I explicitly asked if there's any interest in abstracting the event loop not because it would be some huge performance improvement but rather a nice-to-have capability for performance junkies.

To give some context, I built a custom USB device and wrote a user-space driver for it using rusb. The device is meant to be always connected and the driver exposes a system tray icon which controls it, which is implemented using gtk-rs. As one might expect, the tray icon mostly sits idle. So the glib event loop spends most of its time processing no events. As a personal ambition, I wanted to see if I could integrate the libusb async API into the glib UI event loop to avoid a background thread when the main one is busy doing nothing. And I did just that, hence the PR to rusb.

Now that I looked a bit more through nusb and its abstraction over sync/async it seems that providing event loop integrations wouldn't come with considerable amount of changes. And with you providing your point of view over this, I'll take that the answer to the issue is simply "No, there isn't much interest in event loop integrations", which is really fine. Doesn't hurt to ask :)!

I'll close the issue since my question has been answered. I really appreciate you taking the time!

[korran]

I am interested in serially processing line-rate superspeed data via nusb, and I'm a bit concerned that the kernel context switch between the event-handling thread and the application code might be noticeable in latency benchmarks. Ideally I'd like the poll/USBDEVFS_REAP_URB syscalls and application code that processes the data to be in the same thread, with non-send futures.

[kevinmehall]

@korran One idea if that becomes a problem would be to add a method Device::handle_events<T>(impl Future<Output = T>) -> T that works like async_io::block_on. It would remove the device from the global epoll loop and run its own epoll loop for it that also polls the provided Future on the same thread. I think that could be implemented cross-platform.

[korran]

@kevinmehall Could this be composed with the epoll loop from something like current-thread-flavor tokio, to allow network events, USB events, and application code to be processed in the same thread?

[kevinmehall]

That function signature would probably not be the way to do it in that case. It would be possible to integrate with the Tokio runtime, but it would have the drawbacks described above: Only possible on Linux, and would require bouncing through an extra Tokio task to call USBDEVFS_REAP_URBNDELAY before dispatching to the task waiting for a specific endpoint. But for a single threaded Tokio runtime, that would be a Tokio-level task switch, not an OS context switch, so might be worthwhile.

[korran]

I really wish application developers were given more control over the event loop in the Rust async ecosystem; dynamic dispatch shouldn't be required just to reuse an HTTP or USB library.

Ideally libraries like nusb and hyper would just be simple state machines at the lowest level that I can pump myself without needing to commit to a specific event loop implementation.