Skip to content

Commit 94661d9

Browse files
CodingAnarchyclaude
andcommitted
Release v0.4.0: Comprehensive job prioritization system
## Major Features Added - 🎯 **Complete Job Prioritization System** - Five priority levels: Background, Low, Normal, High, Critical - Enhanced Job struct with priority field and builder methods - String parsing and integer conversion for database storage - ⚖️ **Advanced Priority-Aware Scheduling** - Weighted priority scheduling with configurable weights - Strict priority scheduling for guaranteed ordering - Fairness mechanisms to prevent job starvation - Hash-based selection for Send compatibility - 🗄️ **Database Schema Enhancements** - Added priority column to PostgreSQL and MySQL schemas - Optimized indexes for priority-based querying - Priority-aware dequeue methods with weighted algorithms - Full backward compatibility - 👷 **Worker Priority Configuration** - Configurable priority weights and selection strategies - Support for strict and weighted priority modes - Worker pools with heterogeneous priority configurations - Integration with existing timeout and statistics systems - 📊 **Priority Statistics and Monitoring** - PriorityStats tracking per-priority metrics - Distribution analysis and starvation detection - Integration with InMemoryStatsCollector - Comprehensive priority performance monitoring - 🧪 **Comprehensive Testing Suite** - 12 new priority-specific unit tests - Priority example demonstrating all features - Statistics integration testing - Edge case and configuration testing ## Technical Improvements - Enhanced package description and documentation - Applied clippy suggestions for cleaner code - Consistent formatting across all files - Fixed all stats module tests with priority field 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
1 parent bd9e7c4 commit 94661d9

File tree

17 files changed

+2483
-489
lines changed

17 files changed

+2483
-489
lines changed

CHANGELOG.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,61 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [0.4.0] - 2025-06-26
9+
10+
### Added
11+
- **🎯 Comprehensive Job Prioritization System**
12+
- Five priority levels: `Background`, `Low`, `Normal` (default), `High`, `Critical`
13+
- Enhanced `Job` struct with priority field and builder methods: `as_critical()`, `as_high_priority()`, `as_low_priority()`, `as_background()`, `with_priority()`
14+
- Utility methods: `is_critical()`, `is_high_priority()`, `is_normal_priority()`, `is_low_priority()`, `is_background()`, `priority_value()`
15+
- String parsing support with multiple aliases (e.g., "crit", "c" for Critical)
16+
- Integer conversion methods for database storage: `as_i32()`, `from_i32()`
17+
18+
- **⚖️ Advanced Priority-Aware Job Scheduling**
19+
- Weighted priority scheduling with configurable weights per priority level
20+
- Strict priority scheduling (highest priority jobs always first)
21+
- Fairness factor to prevent low-priority job starvation
22+
- Hash-based job selection for Send compatibility in async contexts
23+
- `PriorityWeights` configuration with builder pattern
24+
25+
- **🗄️ Database Schema and Query Enhancements**
26+
- Added `priority` column to both PostgreSQL and MySQL schemas with default value `2` (Normal)
27+
- Optimized database indexes: `idx_hammerwork_jobs_queue_status_priority_scheduled` for efficient priority-based querying
28+
- Priority-aware `dequeue()` and `dequeue_with_priority_weights()` methods
29+
- Backward compatibility with existing job records (default to Normal priority)
30+
31+
- **👷 Worker Priority Configuration**
32+
- `with_priority_weights()` - Configure custom priority weights
33+
- `with_strict_priority()` - Enable strict priority mode
34+
- `with_weighted_priority()` - Enable weighted priority scheduling (default)
35+
- Worker pools support mixed priority configurations across workers
36+
- Integration with existing timeout and statistics systems
37+
38+
- **📊 Priority Statistics and Monitoring**
39+
- `PriorityStats` tracking job counts, processing times, and throughput per priority
40+
- Priority distribution percentage calculations
41+
- Starvation detection with configurable thresholds
42+
- Integration with `InMemoryStatsCollector` and `JobEvent` system
43+
- Most active priority identification and trend analysis
44+
45+
- **🧪 Comprehensive Testing Suite**
46+
- 12 new priority-specific unit tests covering all functionality
47+
- Priority ordering, serialization, and string parsing tests
48+
- Worker configuration and edge case testing
49+
- Statistics integration and starvation detection tests
50+
- Example demonstrating weighted scheduling, strict priority, and statistics
51+
52+
### Enhanced
53+
- Updated package description to highlight job prioritization capabilities
54+
- Added `priority_example.rs` demonstrating all priority features
55+
- Enhanced statistics collection to track priority-specific metrics
56+
- Worker pools now support heterogeneous priority configurations
57+
58+
### Fixed
59+
- All stats module tests now include required priority field
60+
- Applied clippy suggestions for cleaner derive macro usage
61+
- Consistent code formatting across all files
62+
863
## [0.3.0] - 2025-06-26
964

1065
### Added

CLAUDE.md

Lines changed: 26 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
44

55
## Project Overview
66

7-
Hammerwork is a high-performance, database-driven job queue library for Rust with support for both PostgreSQL and MySQL. It provides async/await job processing with configurable retry logic, delayed jobs, and worker pools.
7+
Hammerwork is a high-performance, database-driven job queue library for Rust with support for both PostgreSQL and MySQL. It provides async/await job processing with configurable retry logic, delayed jobs, job prioritization with weighted scheduling, and worker pools.
88

99
## Common Development Commands
1010

@@ -23,6 +23,8 @@ cargo test --features mysql
2323
# Run examples
2424
cargo run --example postgres_example --features postgres
2525
cargo run --example mysql_example --features mysql
26+
cargo run --example cron_example --features postgres
27+
cargo run --example priority_example --features postgres
2628

2729
# Check code formatting
2830
cargo fmt --check
@@ -44,33 +46,43 @@ The codebase follows a modular architecture with clear separation of concerns:
4446
### Core Components
4547

4648
1. **Job (`src/job.rs`)**
47-
- Defines the `Job` struct with UUID, payload, status, retry logic
48-
- Job statuses: Pending, Running, Completed, Failed, Retrying
49-
- Supports delayed execution and configurable retry attempts
49+
- Defines the `Job` struct with UUID, payload, status, priority, retry logic
50+
- Job statuses: Pending, Running, Completed, Failed, Retrying, Dead, TimedOut
51+
- Job priorities: Background, Low, Normal, High, Critical
52+
- Supports delayed execution, configurable retry attempts, and priority levels
5053

5154
2. **Queue (`src/queue.rs`)**
5255
- `DatabaseQueue` trait defines the interface for database operations
5356
- `JobQueue<DB>` generic struct wraps database connection pools
5457
- Database-specific implementations in `postgres` and `mysql` modules
5558
- Uses database transactions for atomic job operations
59+
- Priority-aware job selection with weighted algorithms
5660

5761
3. **Worker (`src/worker.rs`)**
5862
- `Worker<DB>` processes jobs from specific queues
59-
- Configurable polling intervals, retry delays, and max retries
63+
- Configurable polling intervals, retry delays, max retries, and priority weights
64+
- Support for strict priority or weighted priority scheduling
6065
- `WorkerPool<DB>` manages multiple workers with graceful shutdown
6166
- Uses async channels for shutdown coordination
6267

63-
4. **Error Handling (`src/error.rs`)**
68+
4. **Priority System (`src/priority.rs`)**
69+
- `JobPriority` enum with 5 levels: Background, Low, Normal, High, Critical
70+
- `PriorityWeights` for configurable weighted job selection
71+
- `PriorityStats` for monitoring priority distribution and performance
72+
- Prevents job starvation through fairness mechanisms
73+
74+
5. **Error Handling (`src/error.rs`)**
6475
- `HammerworkError` enum using thiserror for structured error handling
6576
- Wraps SQLx, serialization, and custom errors
6677

6778
### Database Schema
6879

6980
Both PostgreSQL and MySQL implementations use a single table `hammerwork_jobs` with:
70-
- Job metadata (id, queue_name, status, attempts)
71-
- Timing information (created_at, scheduled_at, started_at, completed_at)
81+
- Job metadata (id, queue_name, status, priority, attempts)
82+
- Timing information (created_at, scheduled_at, started_at, completed_at, failed_at, timed_out_at)
7283
- Payload data (JSON/JSONB) and error messages
73-
- Optimized indexes for queue polling
84+
- Cron scheduling fields (cron_schedule, next_run_at, recurring, timezone)
85+
- Optimized indexes for priority-aware queue polling
7486

7587
### Key Design Patterns
7688

@@ -79,11 +91,15 @@ Both PostgreSQL and MySQL implementations use a single table `hammerwork_jobs` w
7991
- **Async-first**: Built on Tokio with async/await throughout
8092
- **Transaction safety**: Uses database transactions for job state changes
8193
- **Type-safe job handling**: Job handlers return `Result<()>` for error handling
94+
- **Priority-aware scheduling**: Weighted and strict priority algorithms prevent starvation
95+
- **Comprehensive monitoring**: Statistics track priority distribution and performance
8296

8397
## Development Notes
8498

8599
- SQLx compile-time query checking requires database connections
86100
- PostgreSQL uses `FOR UPDATE SKIP LOCKED` for efficient job polling
87101
- MySQL implementation uses transaction-based locking (less optimal)
102+
- Priority-aware queries use `ORDER BY priority DESC, scheduled_at ASC`
103+
- Weighted selection uses hash-based algorithms for Send compatibility
88104
- Tracing is integrated for observability
89-
- Examples demonstrate both database configurations
105+
- Examples demonstrate database configurations, cron scheduling, and prioritization

Cargo.toml

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,13 @@ tracing = "0.1"
2121
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
2222
async-trait = "0.1"
2323
tokio-test = "0.4"
24+
rand = "0.8"
2425

2526
[package]
2627
name = "hammerwork"
27-
version = "0.3.0"
28+
version = "0.4.0"
2829
edition = "2021"
29-
description = "A high-performance, database-driven job queue for Rust with PostgreSQL and MySQL support, featuring cron scheduling, job timeouts, dead job management, and comprehensive statistics collection"
30+
description = "A high-performance, database-driven job queue for Rust with PostgreSQL and MySQL support, featuring job prioritization, cron scheduling, timeouts, dead job management, and comprehensive statistics collection"
3031
license = "MIT"
3132
repository = "https://github.com/CodingAnarchy/hammerwork"
3233
authors = ["CodingAnarchy <[email protected]>"]
@@ -48,6 +49,7 @@ serde_json = { workspace = true }
4849
thiserror = { workspace = true }
4950
tracing = { workspace = true }
5051
async-trait = { workspace = true }
52+
rand = { workspace = true }
5153

5254
[features]
5355
default = []
@@ -69,3 +71,7 @@ required-features = ["mysql"]
6971
[[example]]
7072
name = "cron_example"
7173
required-features = ["postgres"]
74+
75+
[[example]]
76+
name = "priority_example"
77+
required-features = ["postgres"]

0 commit comments

Comments
 (0)