Add post-analysis cleanup hook to free per-scan memory#297
Merged
Conversation
Introduce cleanup() to SingleDeviceScanAnalyzer to free memory after analysis by clearing large per-scan attributes (raw_data, results, _data_file_map, stateful_results, _pending_aux_updates). Delegates to renderer.cleanup() when available and emits a debug log on completion. This helps release numpy arrays and result objects held between analyses.
Import logging and create a module logger, and add a cleanup() method to BaseRenderer. The cleanup method resets self.display_contents to an empty list to avoid returning paths from previous scans, and logs completion at debug level. Subclasses can override cleanup to release additional cached state (e.g. large arrays).
Introduce a required cleanup() method on ScanAnalyzer (ScanAnalysis/scan_analysis/base.py) to force subclasses to release per-scan resources and avoid unbounded memory growth. The runner (ScanAnalysis/scan_analysis/task_queue.py) now calls analyzer.cleanup() in the finally block after the heartbeat thread is joined, ensuring cleanup always runs after analysis. NotImplementedError is raised by default to make missing implementations explicit.
- ScatterPlotterAnalysis.cleanup() added with pass (no heavy state) - Remove hasattr guard in SingleDeviceScanAnalyzer.cleanup() — BaseRenderer now always has cleanup(), so the guard was misleading Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human description
During LiveTaskRunner, persistent ScanAnalzyer objects iteratively run analysis as new data comes in. However, nothing purges the internal states of these analyzers until the next analysis call. Thus, memory is not really ever released. Now, after successful or failed analysis, the internal states are emptied, freeing up the memory.
AI description
Introduces a
cleanup()lifecycle method to the analyzer stack to prevent unbounded memory growth when the task runner processes many scans in a single session.Changes
ScanAnalyzer.cleanup()(base) — raisesNotImplementedErrorintentionally; any subclass that omits an implementation will halt the task runner loudly rather than silently leak memory.BaseRenderer.cleanup()— resetsdisplay_contents = []; subclasses may override to release additional cached state.SingleDeviceScanAnalyzer.cleanup()— clearsraw_data,results,_data_file_map,stateful_results, and_pending_aux_updates, then callsself.renderer.cleanup()directly (nohasattrguard needed sinceBaseRendereralways provides the method).ScatterPlotterAnalysis.cleanup()— implemented withpass; no large data is held between scans.task_queue.run_worklist()— callsanalyzer.cleanup()in thefinallyblock so it runs unconditionally after every analysis (success or failure).NotImplementedErrorpropagates unguarded to halt the runner.Design decisions
cleanup()implementation raisesNotImplementedErrorand stops the runner rather than silently accumulating memory.finallyso cleanup always runs, even after a failed analysis where partial data may still be in memory.🤖 Generated with Claude Code