Skip to content

Comments

Cap default max_workers by CPU count#233

Open
fuziontech wants to merge 1 commit intomainfrom
fix/cap-max-workers-by-cpu
Open

Cap default max_workers by CPU count#233
fuziontech wants to merge 1 commit intomainfrom
fix/cap-max-workers-by-cpu

Conversation

@fuziontech
Copy link
Member

Summary

  • DefaultMaxWorkers() previously only considered memory (budget / 256MB), which derived 24 workers on an m7g.large (2 vCPUs, 8GB RAM)
  • 22+ DuckDB worker processes on 2 vCPUs caused CPU starvation, cascading health check DeadlineExceeded timeouts, worker crashes, and force-kills
  • Now takes min(budget/256MB, numCPU*4) with a floor of 1 — caps at 8 workers on m7g.large instead of 24

Test plan

  • TestDefaultMaxWorkers updated with table-driven tests covering memory-limited, CPU-limited, equal, floor, and large instance cases
  • All controlplane tests pass
  • Deploy to m7g.large and verify health check timeouts stop

🤖 Generated with Claude Code

On small instances (e.g., m7g.large with 2 vCPUs and 8GB RAM), the
memory-only formula (budget / 256MB) derived 24 max_workers, causing
CPU starvation and cascading health check timeouts across all workers.

DefaultMaxWorkers now takes min(budget/256MB, numCPU*4), with a floor
of 1. On the m7g.large this caps at 8 workers instead of 24.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant