Skip to content

Conversation

@skidder
Copy link

@skidder skidder commented Jan 20, 2026

Summary

  • Fixes issue where server-side handlers continue waiting for cache lock timeout even after client disconnects
  • Introduces cache_lock_wait_or_disconnect() which uses tokio::select! to race cache lock wait against client disconnect detection
  • When client disconnects, server exits immediately instead of waiting for full lock timeout (e.g., 60 seconds)

Problem

When multiple clients request the same uncached resource:

  1. One client becomes the "writer" (gets WritePermit) and fetches from origin
  2. Other clients become "readers" (get ReadLock) and wait for the writer
  3. If a reader's client disconnects, the server-side handler previously continued waiting for the full lock timeout before releasing resources

This wasted server resources and caused delays, particularly noticeable with long lock timeouts.

Solution

Race the cache lock wait against client disconnect detection:

tokio::select! {
    biased;
    lock_status = session.cache.cache_lock_wait() => Some(lock_status),
    _ = session.downstream_session.read_body_or_idle(true) => None, // Client disconnected
}

Protocol Support

Protocol Disconnect Detection
HTTP/1.1 TCP close (read returns 0)
HTTP/2 Stream reset via poll_reset()
HTTP/3 Via QUIC termination layer signal propagation

Test plan

  • Added test_cache_lock_reader_client_disconnect - basic disconnect test
  • Added test_cache_lock_reader_disconnect_timing - timing regression test that fails without the fix
    • With fix: Server closes connections in ~0.5ms
    • Without fix: Server waits ~2 seconds for lock timeout
  • All existing cache lock tests pass (8 tests total)

🤖 Generated with Claude Code

When a client disconnects while waiting on a cache lock, the server-side
handler previously continued waiting for the full lock timeout (e.g., 60
seconds) before releasing resources. This wasted server resources and
caused delays when clients cancelled requests.

The fix introduces `cache_lock_wait_or_disconnect()` which uses
`tokio::select!` to race the cache lock wait against client disconnect
detection via `read_body_or_idle(true)`. When a client disconnects, the
server-side handler now exits immediately instead of waiting for the
lock timeout.

This applies to HTTP/1.1 (TCP close detection) and HTTP/2 (stream reset
detection). For HTTP/3, this helps when the QUIC termination layer
properly propagates cancellation signals.

Includes regression tests that verify:
- Server closes connections within 1 second of client disconnect (with fix)
- Without fix, server waits ~2 seconds for lock age timeout

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@skidder skidder closed this Jan 20, 2026
@skidder skidder deleted the skidder/cache-lock-client-disconnect branch January 20, 2026 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant