-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
enhancementNew feature or requestNew feature or request
Description
If processing crashes at image 800/1246, the next run restarts from zero. For large datasets this wastes significant compute time and is a production reliability gap.
Scope:
- Create
internal/checkpoint/checkpoint.gomanaging acheckpoint.jsonin the output directory - Track completed file paths with their output hash
- On engine startup, load existing checkpoint and skip already-processed files
- Atomic checkpoint writes (write to temp file, rename) to prevent corruption on crash
- Add
-no-resumeflag to force full reprocessing
checkpoint.json shape:
{
"version": 1,
"started_at": "2026-02-18T10:00:00Z",
"completed": ["images/photo1.jpg", "images/photo2.png"],
"total_processed": 800
}Acceptance Criteria:
- Interrupted run resumes from last checkpoint on restart
checkpoint.jsonis never left in a corrupted state-no-resumeflag bypasses checkpoint entirely- Checkpoint file excluded from output metrics
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request