add documentation for using an external event manager #1944
Conversation
| 3. Register read events with the event manager so that the loop is notified when datagrams arrive. | ||
|
|
||
| Your callback should read all available packets. Picoquic accepts full datagrams (including coalesced QUIC packets). For each datagram, call `picoquic_incoming_packet_ex` with the raw bytes, source/destination addresses, interface index (if available), ECN bits, and the current timestamp (`picoquic/packet.c:2369`). | ||
|
|
There was a problem hiding this comment.
"full datagrams, defined as the entire payload of an UDP packet (it might include several coalesced QUIC packets)."
| ## 4. Producing Packets for Transmission | ||
|
|
||
| Whenever the UDP socket becomes writable (or immediately after processing inbound packets), call `picoquic_prepare_next_packet_ex`. The function returns either a stateless packet (retry/close) or the next connection’s datagram (`picoquic/sender.c:4227`). You pass in buffers and Picoquic fills in the destination address and interface index. | ||
|
|
There was a problem hiding this comment.
In some case, the function will return nothing (send_length == 0) -- the application just needs to update its timers, as explained in the "scheduling timers" section.
| ### Handling Multiple Paths or Alternate Ports | ||
|
|
||
| If your application binds multiple sockets (for multipath or alternate ports), call `picoquic_prepare_next_packet_ex` once per writable socket and respect the returned `addr_from` / interface hints. Packet production is coordinated with each path’s congestion controller during `picoquic_prepare_packet_ex`, which chooses the path tuple providing the datagram (`picoquic/sender.c:4055`). | ||
|
|
There was a problem hiding this comment.
Not quite.
"respect the returned addr_from / interface hints" means sending the packet on the appropriate socket. This is a bit of a gray area. The current code calls picoquic_prepare_next_packet_ex once, not once per socket. The code will determine the outgoing interface and source address based on the internal path selection logic. It does not monitor the "available for sending" API per socket; instead, it relies on congestion control to ensure that each interface only gets a reasonable amount of traffic.
We could change the API and let the application tell picoquic that it is now ready to send a packet on interface X. But we do not have that API yet.
|
|
||
| return ctx; | ||
| } | ||
|
|
There was a problem hiding this comment.
+1 for evutil_make_socket_nonblocking. See previous remark on not having a "per socket" version of the "prepare packet". Make the socket non blocking is the way to do it!
| ### Packet Coalescing and Maximum MTU | ||
|
|
||
| For best throughput, keep the send buffer at least `PICOQUIC_MAX_PACKET_SIZE` bytes. Picoquic will coalesce multiple QUIC packets into a single UDP datagram when GSO is available (`picoquic/sockloop.c:1006`), but falling back to standard `sendto` still works—the function simply returns the exact length to transmit. | ||
|
|
There was a problem hiding this comment.
Not quite. There are two separate concepts, coalescing and UDP GSO.
Coalescing is used internally, to format the content of a UDP payload. It does require that the buffer be at least 1200 bytes (the minimum length of a handshake packet), but it will work fine with larger buffers.
UDP GSO allows picoquic to prepare several UDP packets on a single call. All these packets will have the "addr_from", "addr_to" and "interface" parameters. All but the last one will have the same packet size. To use UDP GSO, the application should provide a buffer capable of holding the payload of multiple packets -- a size of at least 16K is nice, the current code uses 64K. UDP GSO is controlled by the parameter send_msg_size in:
int picoquic_prepare_packet_ex(picoquic_cnx_t* cnx,
uint64_t current_time, uint8_t* send_buffer, size_t send_buffer_max, size_t* send_length,
struct sockaddr_storage* p_addr_to, struct sockaddr_storage* p_addr_from, int* if_index,
size_t* send_msg_size);
If that parameter is a NULL pointer, UDP GSO is not used. Otherwise, the code will attempt to format multiple packets in the send_buffer. The send_length will return the combined length of all these packets, and the send_msg_size will return the length of the first packet. The packets after the first will have the same length, except for the last one which may be shorter, and would carry the remaining bytes in the send_buffer.
| ### Error Handling and Connection Cleanup | ||
|
|
||
| If `picoquic_prepare_next_packet_ex` returns an error, inspect the connection (`last_cnx`) and call `picoquic_close` or `picoquic_free` as needed. Connection states progress through `picoquic_state_client_ready`, `picoquic_state_disconnected`, etc., just as they do in the built-in loop (`sample/sample_client.c:380`). | ||
|
|
There was a problem hiding this comment.
I don't think that's necessary. The event management does not have to concern itself with connection handling. If a connection fails, the application will receive a callback such as:
picoquic_callback_stateless_reset, /* Stateless reset received from peer. Stream=0, bytes=NULL, len=0 */
picoquic_callback_close, /* Connection close. Stream=0, bytes=NULL, len=0 */
picoquic_callback_application_close, /* Application closed by peer. Stream=0, bytes=NULL, len=0 */
The exception is if we have an internal error, indicating that picoquic has lost its internal state and must terminate.
In that case, calling picoquic_free to free the quic handle is appropriate.
And yes, the builtin loop has a bit of complicated logic here. That's because it is also used for testing and debugging, and applications do not normally have to do that.
| 3. Poll `picoquic_prepare_next_packet_ex` after every receive or wake-up and send any returned datagrams immediately. | ||
| 4. Reschedule a timer using `picoquic_get_next_wake_delay` so PTOs trigger correctly. | ||
| 5. Use Picoquic’s logging (`picoquic_log_packet`) or your own instrumentation to monitor traffic. | ||
|
|
There was a problem hiding this comment.
The application can use one of the logging providers (text, binary or qlog) to get a log. It should not have to log packet from within the event manager. But consider "5. Free the QUIC context once when closing the event loop." That will trigger the appropriate "connection close" callbacks, allowing the application to properly close its connection contexts.
Fixes #1608:
add a comprehensive doc/howto_use_external_event_manager.md guide so downstream applications can run Picoquic from their own event loop. The new doc covers context creation with picoquic_create_and_configure, wiring UDP I/O via picoquic_incoming_packet_ex / picoquic_prepare_next_packet_ex, timer scheduling with picoquic_get_next_wake_delay, and provides a libevent-based C example with correct API usage.