Skip to content

Commit 9704e9f

Browse files
SmileyChrisSmileyChris
committed
management: add rich easy_images status output and tests; deprecate old command\n\n- New command: easy_images with subcommands (status/build/requeue)\n- Status: add totals (sources, generated, queued) and avg per source\n- Status: support --format pretty|plain|json, --stale-after; suggestions\n- Status: health gates --fail-on-stale/--fail-on-errors\n- Process queue: stale + max-errors handling retained\n- Deprecated build_img_queue: emit DeprecationWarning and map options\n- Tests: add management tests; guard VIPS tests with marker/import skip\n- Docs: updated API docs for new command
1 parent b24bd07 commit 9704e9f

File tree

12 files changed

+849
-18
lines changed

12 files changed

+849
-18
lines changed

‎docs/api.md‎

Lines changed: 118 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -84,12 +84,127 @@ Database models for storing image information
8484

8585
## Management Commands
8686

87-
### `build_img_queue`
88-
Processes pending images in the queue:
87+
### `easy_images`
88+
89+
Manages the EasyImage queue with subcommands for building, requeuing, and checking status. This is the primary command for managing image processing.
90+
91+
#### Basic Usage
92+
8993
```bash
90-
python manage.py build_img_queue
94+
# Show queue status (default action)
95+
python manage.py easy_images
96+
python manage.py easy_images status
97+
98+
# Build queued images
99+
python manage.py easy_images build
100+
101+
# Requeue failed images
102+
python manage.py easy_images requeue
91103
```
92104

105+
#### Subcommands
106+
107+
##### `status` (default)
108+
Shows queue statistics and current state:
109+
110+
```bash
111+
python manage.py easy_images status
112+
python manage.py easy_images status --verbose # Show error distribution
113+
```
114+
115+
Output includes:
116+
- Total images in queue
117+
- Breakdown by status (queued, building, errors)
118+
- Detection of stale builds
119+
- Error count distribution (with --verbose)
120+
121+
##### `build`
122+
Processes queued images with intelligent stale detection:
123+
124+
```bash
125+
python manage.py easy_images build
126+
python manage.py easy_images build --stale-after 300 # Stale if BUILDING > 5 minutes
127+
python manage.py easy_images build --max-errors 3 # Skip images with > 3 errors
128+
python manage.py easy_images build --verbose # Show detailed progress
129+
```
130+
131+
Options:
132+
- `--stale-after <seconds>` - Images stuck in BUILDING status longer than this are considered stale and reprocessed (default: 600 seconds)
133+
- `--max-errors <count>` - Only retry images with at most this many previous errors
134+
- `--verbose` - Show detailed progress and error information
135+
136+
##### `requeue`
137+
Resets failed images back to QUEUED status for reprocessing:
138+
139+
```bash
140+
python manage.py easy_images requeue
141+
python manage.py easy_images requeue --max-errors 5 # Only if ≤ 5 errors
142+
python manage.py easy_images requeue --include-stale # Also requeue stale builds
143+
```
144+
145+
Options:
146+
- `--max-errors <count>` - Only requeue images with at most this many errors
147+
- `--include-stale` - Also requeue images stuck in BUILDING status
148+
- `--stale-after <seconds>` - With --include-stale, defines stale threshold (default: 600)
149+
150+
#### Image Processing States
151+
152+
- **Queued** - New images waiting to be processed
153+
- **Building** - Images currently being processed (auto-detected as stale if too old)
154+
- **Built** - Successfully processed images
155+
- **Source Error** - Source file couldn't be accessed
156+
- **Build Error** - Failed during processing
157+
158+
#### Smart Stale Detection
159+
160+
The command automatically handles crashed or stuck builds by checking the `status_changed_date`:
161+
- Images in BUILDING status with recent timestamps are skipped (actually building)
162+
- Images in BUILDING status with old timestamps are treated as stale and reprocessed
163+
- No need for manual `--force` flag in most cases
164+
165+
#### Integration Examples
166+
167+
1. **Cron Job** - Regular processing with automatic stale handling:
168+
```bash
169+
# Process queue every 5 minutes
170+
*/5 * * * * /path/to/python /path/to/manage.py easy_images build
171+
```
172+
173+
2. **Error Recovery Workflow**:
174+
```bash
175+
# Check current status
176+
python manage.py easy_images
177+
178+
# Requeue failed images with < 3 errors
179+
python manage.py easy_images requeue --max-errors 3
180+
181+
# Process the requeued images
182+
python manage.py easy_images build
183+
```
184+
185+
3. **Monitoring Script**:
186+
```bash
187+
# Get detailed status for monitoring
188+
python manage.py easy_images status --verbose
189+
```
190+
191+
### `build_img_queue` (Deprecated)
192+
193+
**⚠️ Deprecated:** This command is maintained for backwards compatibility. Please use `easy_images build` instead.
194+
195+
```bash
196+
# Old command (deprecated)
197+
python manage.py build_img_queue --retry 3
198+
199+
# New equivalent
200+
python manage.py easy_images build --max-errors 3
201+
```
202+
203+
The old command will continue to work but displays a deprecation warning. It maps to the new command with these defaults:
204+
- `--retry` → `--max-errors`
205+
- Stale detection enabled with 600 second threshold
206+
- All other behavior remains the same
207+
93208
## Template Tags
94209

95210
### `easy_images`

‎easy_images/management/commands/build_img_queue.py‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import warnings
2+
13
from django.core.management.base import BaseCommand
24
from django.db.models import Count, Q
35

@@ -6,7 +8,7 @@
68

79

810
class Command(BaseCommand):
9-
help = "Process EasyImages that need to be built"
11+
help = "Process EasyImages that need to be built (DEPRECATED: use 'easy_images build' instead)"
1012

1113
def add_arguments(self, parser):
1214
parser.add_argument(
@@ -26,6 +28,14 @@ def add_arguments(self, parser):
2628
)
2729

2830
def handle(self, *, verbosity, retry=None, force=None, count_only=False, **options):
31+
# Show deprecation warning
32+
warnings.warn(
33+
"The 'build_img_queue' command is deprecated. "
34+
"Please use 'python manage.py easy_images build' instead.",
35+
DeprecationWarning,
36+
stacklevel=2
37+
)
38+
2939
if count_only:
3040
count = EasyImage.objects.filter(image="").count()
3141
self.stdout.write(f"{count} <img> thumbnails need building")
@@ -101,7 +111,13 @@ def handle(self, *, verbosity, retry=None, force=None, count_only=False, **optio
101111
)
102112
if verbosity:
103113
self.stdout.flush()
104-
built = process_queue(force=bool(force), retry=retry, verbose=verbosity > 1)
114+
# Map old parameters to new ones
115+
built = process_queue(
116+
force=bool(force),
117+
retry=retry, # Will be mapped to max_errors in process_queue
118+
verbose=verbosity > 1,
119+
stale_after_seconds=600 # Default 10 minutes for backwards compatibility
120+
)
105121
if built is None:
106122
if verbosity:
107123
self.stdout.write("No <img> thumbnails required building")

0 commit comments

Comments
 (0)