perf: configurable scan speed for picture mode (up to 16x faster)#1368
Open
codingfrog wants to merge 2 commits intoarsenetar:masterfrom
Open
perf: configurable scan speed for picture mode (up to 16x faster)#1368codingfrog wants to merge 2 commits intoarsenetar:masterfrom
codingfrog wants to merge 2 commits intoarsenetar:masterfrom
Conversation
…ation - Pre-scale images to 450x450 before block extraction (qt/pe/photo.py) instead of processing full-resolution images (e.g., 4000x3000) - Parallelize image preparation using ThreadPoolExecutor across all CPU cores instead of sequential single-threaded processing - Enable SQLite WAL mode for better concurrent cache access Benchmarked: ~2 img/s → 33 img/s on real photos (16x speedup). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…urbo) Add a user-facing "Scan speed" dropdown in the Picture preferences dialog that controls how aggressively images are pre-scaled before block extraction: - Accurate (slow): no pre-scaling, processes full-resolution images - Balanced (recommended): pre-scale to 450px — good accuracy/speed trade-off - Turbo: pre-scale to 150px — fastest, suitable for large collections Each mode uses a separate cache file so switching modes doesn't require re-scanning. The preference is persisted across sessions. Also improves Filter Hardness UI by disabling all related labels (not just the slider) when the control is inactive. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Picture scanning in dupeGuru processes full-resolution images (e.g. 4000x3000) one at a time, which makes scanning large photo libraries very slow (~2 images/second).
This PR adds:
ThreadPoolExecutoracross all CPU cores instead of sequential single-threaded processingEach mode uses a separate cache file, so switching modes doesn't require re-scanning the entire collection.
Benchmark
Tested on real photos (mix of 12-24MP JPEGs):
Changes
core/pe/matchblock.py— parallel preparation + prescale multiplier plumbingcore/pe/cache_sqlite.py— enable WAL modeqt/pe/photo.py— configurable pre-scaling in block extractioncore/pe/scanner.py— passpicture_prescaleoption throughcore/app.py— separate cache files per prescale modeqt/pe/preferences_dialog.py— add "Scan speed" combo boxqt/preferences.py— persistpicture_prescalepreferenceqt/preferences_dialog.py— improved Filter Hardness enable/disable logicqt/app.py— wire preference to model optionsqt/me/preferences_dialog.py,qt/se/preferences_dialog.py— use shared_setFilterHardnessEnabled()Test plan
pytest)🤖 Generated with Claude Code