Skip to content

Commit e1a4767

Browse files
CodingAnarchyclaude
andcommitted
Release v0.9.0: Worker Autoscaling and Enhanced Worker Pool Management
This release introduces comprehensive worker autoscaling capabilities that automatically adjust worker pool sizes based on queue demand, significantly improving resource efficiency and operational management. ## Worker Autoscaling Features ### Core Autoscaling Components - AutoscaleConfig with comprehensive configuration options and sane defaults - Three preset configurations: conservative(), aggressive(), and disabled() - Dynamic scaling based on queue depth per worker metrics - Configurable min/max worker limits, scaling thresholds, and cooldown periods - AutoscaleMetrics for real-time monitoring of scaling decisions and utilization ### Smart Scaling Logic - Background autoscaling task with configurable evaluation windows and scaling steps - Queue depth averaging over time windows for stable decision making - Graceful scaling with proper cooldown periods to prevent thrashing - Respect for min/max worker boundaries and configurable scaling steps ### Enhanced WorkerPool - Added with_autoscaling() and without_autoscaling() configuration methods - Worker template system for creating new workers during scale-up operations - Improved worker lifecycle management with proper shutdown handling - Enhanced metrics collection and monitoring integration ## Implementation Details ### Configuration Defaults - Enabled by default with conservative 1-10 worker range - 60-second cooldown periods to prevent rapid oscillations - 30-second evaluation windows for responsive but stable scaling - Scale up at 5 jobs/worker, scale down at 2 jobs/worker ### Testing and Examples - Comprehensive test suite covering all scaling scenarios and edge cases - Complete autoscaling_example.rs demonstrating various configurations - Type safety and proper error handling throughout - Integration with existing statistics and monitoring systems ### Code Quality - Full clippy compliance and code formatting - Type aliases for complex types to improve readability - Proper resource cleanup and graceful shutdown handling - Backward compatibility maintained for existing WorkerPool usage This feature significantly enhances hammerwork's production readiness by automatically optimizing resource usage based on actual workload demand while maintaining system stability and preventing resource waste. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
1 parent 7108318 commit e1a4767

File tree

17 files changed

+1350
-298
lines changed

17 files changed

+1350
-298
lines changed

CHANGELOG.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,28 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [0.9.0] - 2025-06-27
9+
10+
### Added
11+
- **📈 Worker Autoscaling**
12+
- `AutoscaleConfig` with comprehensive configuration options and sane defaults
13+
- Three preset configurations: `conservative()`, `aggressive()`, and `disabled()`
14+
- Dynamic worker pool scaling based on queue depth per worker metrics
15+
- Configurable min/max worker limits, scaling thresholds, and cooldown periods
16+
- `AutoscaleMetrics` for real-time monitoring of scaling decisions and worker utilization
17+
- Background autoscaling task with configurable evaluation windows and scaling steps
18+
- Graceful scaling with proper cooldown periods to prevent thrashing
19+
- Integration with existing WorkerPool infrastructure and statistics collection
20+
- Comprehensive test suite covering all scaling scenarios and edge cases
21+
- Complete example demonstrating various autoscaling configurations
22+
23+
### Enhanced
24+
- **🔧 WorkerPool Improvements**
25+
- Added `with_autoscaling()` and `without_autoscaling()` configuration methods
26+
- Worker template system for creating new workers during scale-up operations
27+
- Improved worker lifecycle management with proper shutdown handling
28+
- Enhanced metrics collection and monitoring integration
29+
830
## [0.8.0] - 2025-06-27
931

1032
### Added

Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ clap = { version = "4.0", features = ["derive"] }
2929

3030
[package]
3131
name = "hammerwork"
32-
version = "0.8.0"
32+
version = "0.9.0"
3333
edition = "2024"
3434
description = "A high-performance, database-driven job queue for Rust with PostgreSQL and MySQL support, featuring job prioritization, cron scheduling, timeouts, rate limiting, Prometheus metrics, alerting, and comprehensive statistics collection"
3535
license = "MIT"

ROADMAP.md

Lines changed: 1 addition & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -62,24 +62,6 @@ worker.on_job_complete(|job, duration| {
6262
});
6363
```
6464

65-
### 📈 Worker Auto-scaling
66-
**Impact: High** | **Complexity: Medium-High** | **Priority: Medium-High**
67-
68-
Significant operational improvement for handling variable workloads.
69-
70-
```rust
71-
// Dynamic worker scaling based on queue depth
72-
let auto_scaler = AutoScaler::new()
73-
.min_workers(2)
74-
.max_workers(20)
75-
.scale_up_threshold(100) // Scale up when queue > 100 jobs
76-
.scale_down_threshold(10) // Scale down when queue < 10 jobs
77-
.scale_up_cooldown(Duration::from_minutes(2))
78-
.scale_down_cooldown(Duration::from_minutes(5));
79-
80-
let worker_pool = WorkerPool::new()
81-
.with_auto_scaler(auto_scaler);
82-
```
8365

8466
### 🌐 Admin Dashboard & CLI Tools
8567
**Impact: High** | **Complexity: Medium-High** | **Priority: Medium**
@@ -270,8 +252,7 @@ Features are ordered within each phase by priority and should generally be imple
270252
**Phase 2 (Advanced Features)**
271253
1. Job Dependencies & Workflows
272254
2. Job Tracing & Correlation
273-
3. Worker Auto-scaling
274-
4. Admin Dashboard & CLI Tools
255+
3. Admin Dashboard & CLI Tools
275256

276257
**Phase 3 (Operational Features)**
277258
1. Job Archiving & Retention

0 commit comments

Comments
 (0)