feat: introduce HTTP/1.1 protocol state machine#12146
feat: introduce HTTP/1.1 protocol state machine#12146algebraic-dev wants to merge 76 commits intosofia/async-http-bodyfrom
Conversation
|
Mathlib CI status (docs):
|
|
Reference manual CI status:
|
0d3c00a to
077fb47
Compare
2437c9f to
f8ad249
Compare
63fe820 to
6ffd5ad
Compare
|
|
||
| /-- | ||
| Count of messages that this connection already parsed | ||
| -/ |
There was a problem hiding this comment.
Not a blocker, but worth noting for future consideration:
When feed appends data (reader.input.array ++ data), the already-parsed bytes before input.pos are retained. Since resetForNextMessage preserves the input buffer across keep-alive requests, parsed bytes accumulate over the connection lifetime.
Rough impact estimate:
- Typical HTTP headers: 500 bytes - 2KB per request
- Default
maxMessages = 100per connection - Worst case: ~100-200KB accumulated per connection
- At scale (10K connections): 1-2GB of wasted memory
How Rust handles this:
Rust's BytesMut does automatic compaction:
"Before allocating new buffer space, the reserve function will attempt to reclaim space in the existing buffer. If the current handle references a view into a larger original buffer, and all other handles have been dropped, then the current view can be copied/shifted to the front of the buffer."
A similar compaction strategy (shift unread bytes to front when parsed portion exceeds a threshold) could be added as a future optimization.
|
Mathlib CI status (docs):
|
This PR introduces the H1 module, a pure HTTP/1.1 state machine that incrementally parses incoming byte streams and emits response bytes without side effects.
This contains the same code as #10478, divided into separate pieces to facilitate easier review.
The pieces of this feature are:
Headersdata type for HTTP #12127URIdata type for HTTP #12128Bodytype class and some Body types for HTTP #12144