Benchmark minor improvements #418
Conversation
There was a problem hiding this comment.
Pull request overview
This PR enhances the benchmark suite with improved timeout handling, stall detection, and workload adjustments. Key changes include dynamic timeout configuration based on GitHub Actions settings, enhanced monitoring for stalled rollouts with detailed worker diagnostics, and optimized benchmark parameters to reduce execution time while maintaining test coverage.
Key Changes:
- Added dynamic timeout calculation from environment variables and stall detection mechanism with worker status reporting
- Increased file descriptor limits in Docker configuration to prevent resource exhaustion
- Adjusted benchmark workload parameters (reduced task counts, modified timeouts and concurrency settings)
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/benchmark/benchmark_store.py | Implements dynamic timeout from environment, adds time-based stall detection with worker diagnostics |
| docker/compose.store.yml | Increases ulimit for file descriptors to support higher concurrency |
| .github/workflows/benchmark.yml | Adds scheduled runs, adjusts timeouts and task counts, passes timeout to benchmark script |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| workflow_dispatch: | ||
| schedule: | ||
| # Every Monday and Thursday at 3 AM UTC+8 | ||
| - cron: '0 19 * * 0,3' |
There was a problem hiding this comment.
The cron schedule '0 19 * * 0,3' runs on Sundays (0) and Wednesdays (3), not Mondays and Thursdays as stated in the comment. To run on Mondays and Thursdays at 3 AM UTC+8 (7 PM UTC previous day), use '0 19 * * 1,4'.
| - cron: '0 19 * * 0,3' | |
| - cron: '0 19 * * 1,4' |
No description provided.