Make scan paths attribute of scan data #192
Merged
Conversation
Introduces minimal primitives for asset filename construction, device folder listing, and extension inference in the ScanPaths class. These additions support canonical asset naming, device subfolder discovery, and automatic selection of file extensions based on existing files.
Replaced 'jpg', 'jpeg', and 'bmp' with 'himg' in the set of acceptable file extensions. This change reflects updated requirements for supported file types.
Major refactor of ScanData: introduces BinningConfig and DeviceSpec dataclasses, flexible binning and aggregation for scalars, lazy asset indexing with normalized tables, device registration, and improved column resolution. Adds methods for grouping and averaging images by bin, ECS dump loading, and s-file management. Enhances type annotations and documentation throughout.
Eliminated asset indexing methods and the DeviceSpec dataclass from ScanData, including all related image grouping and averaging by bin. Added new methods for expected image path grouping and expected asset column handling, streamlining asset management to rely on expected paths rather than file system discovery.
Replaces direct usage of ScanData for path and tag operations with ScanPaths and ScanTag across multiple modules, scripts, and tests. Updates method calls, imports, and documentation to reflect the new separation of scan metadata and path management, improving code clarity and maintainability.
Corrected the usage example to import ScanTag instead of ScanData and updated the instantiation accordingly. Also removed an unnecessary trailing space in the installation section.
Improved docstrings and comments throughout frog_analysis.py for clarity and maintainability. Refactored FrogAnalyzer class to enhance parameter documentation, error handling, and code structure. Added detailed explanations for methods, clarified return types, and improved logging. No functional changes to analysis logic; changes are focused on readability and developer usability.
Added new plotting utilities in geecs_data_utils/plotting_utils.py for visualizing binned data. Updated the basic_usage.ipynb notebook to demonstrate ScanData creation, column searching, and binned data plotting, with improved example data and explanations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
this PR refactors ScanData to not be a child of ScanPaths, but rather use ScanPaths as an attribute. This is a more logical approach for ScanData.
Additionally, some new methods have been added:
-mapping expected file paths for saved devices into the loaded data_frame (which loads using the sfile). This looks in the ScanXXX directory and does light interrogation of the individual contents to determine file extension
-binning of scalar data now possible and configurable
-simple plotting utility to visualize binned data