How do I handle Expect: 100-continue when using BodyDataStream?

[Scripter17]

Summary

Code with the problem

The code contains a POST endpoint that reads the request body and sends a response body in parallel, allowing clients without access to a decent WebSocket client to process arbitrarily large jobs without having to manually break them up into smaller jobs.

The issue is that when using curl with a large payload, curl sends an Expect: 100-continue header and waits 1 second for a 100 response that isn't coming. All issues and discussions I could find seem to say that body extractors with finite limits correctly handle this by sending a 100 response then letting the handler return the real status, but there doesn't seem to be any way to mimic this while accepting arbitrarily large request bodies.

The timeline when using cargo run -p url-cleaner-site and yes https://example.com | head -n 1000000 | curl http://127.0.0.1:9149/clean --data-binary @- is

1. Curl sends an HTTP 1.1 POST request with the Expect header sent to 100-continue.
2. URL Cleaner Site responds with a 200 status and waits for curl to send the body.
3. Curl waits 1 second for a 100 response, completely ignoring the 200 response.
4. Curl gives up, accepting the 200 response.
5. Curl sends the body.
6. URL Cleaner Site returns the response.

What I need to do is somehow inject a "URL Cleaner Site notices the Expect header and sends a 100 status" between 1 and 2, thus skipping 3 and 4.

While I can pass -H Expect: to curl to disable the header and waiting, asking every user of a client that does this to find their client's equivalent seems unreasonable for a problem Rocket handles out of the box. Additionally, having the handler notice the header and return a 100 response makes what should be the real response instead be a 502 error.

axum version

0.8.8

[jplatte]

Can you try curl -v? Does it use HTTP 1.1 or HTTP 2.0? I saw in the h2 (crate used for HTTP 2.0 inside hyper) release notes from only 5d ago that they added support for 1xx responses: https://github.com/hyperium/h2/releases/tag/v0.4.13. Maybe you're using HTTP 2.0 but are still on an older version of h2? You can search your Cargo.lock.

[Scripter17]

(Using port 9148 because my main instance is on 9149 but it's the same thing)

*   Trying 127.0.0.1:9148...
* Connected to 127.0.0.1 (127.0.0.1) port 9148
* using HTTP/1.x
> POST /clean HTTP/1.1
> Host: 127.0.0.1:9148
> User-Agent: curl/8.14.1
> Accept: */*
> Content-Length: 19000000
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
> 
< HTTP/1.1 200 OK
< transfer-encoding: chunked
< date: Sun, 11 Jan 2026 00:46:26 GMT
< 
* Done waiting for 100-continue
} [65536 bytes data]
* upload completely sent off: 19000000 bytes
{ [65550 bytes data]
* Connection #0 to host 127.0.0.1 left intact

h2 is at 0.4.13, not that it's relevant here. Neither --http2-prior-knowledge nor --http2 work as described in curl's --help all. (Edit: They do, though Axum ignores http2 upgrades?)

Edit: I probably should clarify that I'm using a slightly updated version of the linked code that fixes a few bugs, but this particular issue behaves identically between the two versions. Just saying that now in case a proposed fix works on the linked code but not on my local copy.

[jplatte]

Okay. Well, I guess you should report this to hyper. You can copy paste or serve implementation and slim it down, it really doesn't do a while lot of interesting things. All the actually interesting logic is in hyper / hyper-util.

[sunnypatell]

@Scripter17 this is a timing issue in hyper's 100-continue mechanism. it's not a bug, just a consequence of your handler's architecture.

why it happens: hyper automatically sends 100 Continue when the request body is first polled via poll_frame, but only if the response headers haven't been written yet (Writing::Init state). in your handler, you move the body into tokio::spawn and return a streaming response immediately. hyper sees the response is ready, writes the 200 headers (Writing transitions to Body), and by the time your spawned task polls the body, the Writing::Init check in poll_read_body fails and 100-continue is skipped.

extractors like Bytes or Json work because they consume the body before the handler returns, so the poll happens while Writing is still Init.

fix: poll the body once before returning the response

use futures_util::StreamExt;

pub async fn clean_http(
    State(state): State<&'static crate::State>,
    job: Job<'static>,
    body: Body,
) -> Result<Response, (StatusCode, &'static str)> {
    let mut stream = body.into_data_stream();
    let first_chunk = stream.next().await; // triggers 100-continue

    let full_stream = futures_util::stream::iter(first_chunk)
        .chain(stream);

    tokio::spawn(async move {
        let mut reader = full_stream
            .map_err(std::io::Error::other)
            .into_async_read();
        // ... rest of your line-by-line reading ...
    });

    // return streaming response after body has been polled
    Ok(Body::from_stream(/* ... */).into_response())
}

the stream.next().await call triggers Incoming::poll_frame() which sends WANT_READY on the internal watch channel, unblocking hyper's poll_read_body while Writing is still Init. hyper writes HTTP/1.1 100 Continue\r\n\r\n to the connection, curl sees it immediately, and sends the body without the 1-second wait.

then futures_util::stream::iter(first_chunk).chain(stream) reconstructs the full stream with the first chunk prepended so no data is lost.

ref: hyper PR #2119 (100-continue-on-first-poll design) | axum #863 (confirms poll triggers 100-continue) | hyper #3827 (related issue)

[Scripter17]

The design is very ergonomic.

Edit: At the risk of making an actually useful comment:

1. Thank you.

2. This should be doable by just writing body.handle_expect_100_correctly();.

[mladedav]

Instead of stream.next().await; you may also use stream.next().now_or_never(); from futures::FutureExt which returns the data if it's ready or None if it's not, so that you're not waiting in the handler while waiting for the first body chunk if that's an issue.

[Scripter17]

Instead of stream.next().await; you may also use stream.next().now_or_never(); from futures::FutureExt which returns the data if it's ready or None if it's not, so that you're not waiting in the handler while waiting for the first body chunk if that's an issue.

It seems I need to use a full .await. Which is fine since I can't do anything until I get a chunk anyway.

[sunnypatell]

@Scripter17 lol honestly body.handle_expect_100_correctly() would be a great addition to axum or hyper. the whole "poll once to kick the state machine" thing works but nobody reading that code 6 months later would know why it is there without a comment

@mladedav now_or_never() is a neat trick for when you just need the side effect without actually waiting on data, good to know