Skip to content

Support HTTP/1.1 request pipelining on the downstream session#876

Open
CodyPubNub wants to merge 2 commits intocloudflare:mainfrom
CodyPubNub:h1-pipelining-support
Open

Support HTTP/1.1 request pipelining on the downstream session#876
CodyPubNub wants to merge 2 commits intocloudflare:mainfrom
CodyPubNub:h1-pipelining-support

Conversation

@CodyPubNub
Copy link
Copy Markdown

Resolves #377, #673

What

Adds an opt-in HTTP/1.1 pipelining mode on the downstream session that serves pipelined requests sequentially in request order per RFC 9112 §9.3.2. Default behavior is unchanged.

  • Adopter opts in per-session with HttpSession::set_pipelining_enabled(true) (mirrors the field+accessor shape of existing setters like keepalive_timeout, abort_on_close, proxy_tasks_enabled). No new ProxyHttp trait method.
  • ServerSession::finish() and H1 HttpSession::reuse() preserve their pre-pipelining Result<Option<Stream>> signatures and discard any captured pipelined prefix — non-adopters are unaffected.
  • Adopters call finish_reuse() / reuse_pipelined() to receive a ReusedHttpConnection (opaque struct; fields private; into_parts() escape hatch). ReusedHttpStream::from_reused_connection() is the canonical converter that carries the pipelined prefix through HttpPersistentSettings into the next session.

Wire shapes covered

Both observed pipelining wire shapes are handled:

  1. Same-segment overread — request 1 and request 2 arrive in one read. reuse_pipelined() hands the body-reader overread back as the pipelined prefix for the next session.
  2. Two-segment overread — request 2 arrives while request 1's response is still being written. read_body_or_idle()'s idle branch stashes non-zero idle reads onto the body reader's overread surface instead of raising ConnectError("Sent data after end of body"). abort_on_close / half_closed are untouched so FIN handling is unchanged — the new branch is exclusive to the pipelining-bytes case.

HttpPersistentSettings carries pipelining_enabled + the prefix across keep-alive reuses; read_request() consumes the prefix first before pulling more bytes from the stream.

Positioning

Restores nginx parity for the traffic shape in #377 without inheriting nginx's upstream re-pipelining behavior. Each pipelined request gets its own independent upstream selection and its response is fully written before the next request begins — matching Envoy, AWS CloudFront, Google Classic LB, Akamai, and IIS.

Tests

cargo test -p pingora-core test_pipelining — 12 unit tests covering:

  • default-off behavior
  • toggle round-trip
  • reuse rejects overread when pipelining is disabled (both explicit-off and never-set)
  • reuse extracts overread when pipelining is enabled (with and without prior body poll, including Content-Length: 0)
  • read_request() consumes a complete prefix without touching the stream
  • read_request() falls through to the stream for partial prefixes
  • idle-branch stashes bytes when pipelining is enabled (two-segment shape)
  • idle-branch still raises ConnectError when pipelining is disabled
  • end-to-end session A → session B chain via take_body_overread + set_pipelined_prefix

Plus an examples/pipelining.rs that opts in from early_request_filter. Minimal reproducer:

RUST_LOG=INFO cargo run --example pipelining
printf 'GET /get HTTP/1.1\r\nHost: httpbin.org\r\n\r\nGET /get HTTP/1.1\r\nHost: httpbin.org\r\n\r\n' \
  | ncat --no-shutdown localhost 6191 \
  | grep -oE 'HTTP/1.1 [0-9]{3}'
HTTP/1.1 200
HTTP/1.1 200

Before this PR the same command returns a single HTTP/1.1 200 and the connection is marked un-reusable.

Runtime cost for non-adopters

  • Two extra bool and one Option<BytesMut> on HttpSession (comparable scale to existing fields like abort_on_close, proxy_tasks_enabled).
  • The idle-branch check is self.pipelining_enabled && self.pipelined_idle_bytes_stashed (two bool reads) inside an already-async idle poll.
  • Two extra fields on HttpPersistentSettings, constructed once per keep-alive boundary.
  • No changes to the ProxyHttp trait.

If a cargo feature gate is preferred (e.g. matching the connection_filter pattern), happy to add one in a follow-up commit on this PR.

CodyPubNub added 2 commits May 6, 2026 12:36
Add opt-in HTTP/1.1 pipelining via HttpSession::set_pipelining_enabled().
When enabled, pipelined requests on a keep-alive connection are served
sequentially in request order per RFC 9112 §9.3.2, matching nginx
behavior on the same traffic shape. Default (off) is unchanged: the
second pipelined request is dropped or surfaced as a 400 by the
body-pump idle branch.

Non-adopters are untouched: ServerSession::finish() and the H1
HttpSession::reuse() keep their pre-pipelining Result<Option<Stream>>
signatures and discard any captured pipelined prefix. Adopters call
finish_reuse() / reuse_pipelined() to receive ReusedHttpConnection.

Covers both wire shapes: same-segment overread (both requests arrive
in one read) via reuse_pipelined(), and two-segment overread (request
N+1 arrives while request N's response is still being written) via
read_body_or_idle()'s idle branch stashing non-zero reads into the
body reader's overread surface. abort_on_close / half_closed are
untouched so FIN handling is unchanged.

HttpPersistentSettings carries pipelining_enabled + pipelined_prefix
across keep-alive reuses; read_request() consumes the prefix first.

Resolves cloudflare#377, cloudflare#673
Minimal ProxyHttp that opts in via set_pipelining_enabled(true) in
early_request_filter. Matches the reproducer shape from cloudflare#377: pipelined
GETs on one connection now both return responses.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pingora-proxy examples don't support pipelining

1 participant