Skip to content

Proxy: Fix memory load balancer server expiration handling#4649

Open
liukun4515 wants to merge 3 commits intoossrs:developfrom
liukun4515:pr-1
Open

Proxy: Fix memory load balancer server expiration handling#4649
liukun4515 wants to merge 3 commits intoossrs:developfrom
liukun4515:pr-1

Conversation

@liukun4515
Copy link

Summary

Fixes a critical issue in memory-based load balancer where streams could not recover when origin servers become unhealthy.

Changes

  • Fixed MemoryLoadBalancer.Pick(): Now automatically switches to healthy servers when current server's UpdatedAt exceeds ServerAliveDuration (300s)
  • Added double-checked locking: Ensures thread safety during concurrent server selection
  • Added comprehensive unit tests: Verifies server expiration handling and concurrent access safety

Problem

Previously, once a stream was assigned to a server, it would remain bound to that server forever, even if the server became unhealthy (no heartbeat updates for >300s). This prevented streams from recovering when origin servers failed.

Solution

The Pick() method now:

  1. Checks if previously picked server is still healthy (fast path, no lock)
  2. If unhealthy, acquires lock and re-checks (double-checked locking pattern)
  3. Selects a new healthy server if needed

Testing

  • ✅ All unit tests pass
  • ✅ Go race detector shows no race conditions
  • ✅ Concurrent access tests verify thread safety (100 goroutines)
  • ✅ Server expiration tests verify automatic failover

Files Changed

  • proxy/internal/lb/mem.go: Added health checking and mutex-based locking
  • proxy/internal/lb/mem_test.go: Added 5 comprehensive test cases (286 lines)

🤖 Generated with Claude Code

When origin server's UpdatedAt exceeds ServerAliveDuration (300s),
Pick() now automatically selects a healthy server instead of
returning the expired one. Added double-checked locking to ensure
thread safety during concurrent server selection.

Added unit tests to verify server expiration handling and
concurrent access safety.
@winlinvip winlinvip added the EnglishNative This issue is conveyed exclusively in English. label Mar 18, 2026
@liukun4515
Copy link
Author

@winlinvip PTAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

EnglishNative This issue is conveyed exclusively in English.

Development

Successfully merging this pull request may close these issues.

2 participants