graph LR
data_dataset["data.dataset"]
data_hr_dataset["data.hr_dataset"]
data_lr_dataset["data.lr_dataset"]
data_download_archive["data.download_archive"]
data__populate_cache["data._populate_cache"]
data__images_dataset["data._images_dataset"]
data_dataset -- "calls" --> data_hr_dataset
data_dataset -- "calls" --> data_lr_dataset
data_hr_dataset -- "calls" --> data_download_archive
data_hr_dataset -- "calls" --> data__populate_cache
data_hr_dataset -- "calls" --> data__images_dataset
data_lr_dataset -- "calls" --> data_download_archive
data_lr_dataset -- "calls" --> data__populate_cache
data_lr_dataset -- "calls" --> data__images_dataset
The data subsystem is designed to efficiently manage the acquisition, caching, and preparation of image datasets for super-resolution models. At its core, the data.dataset component orchestrates the entire data pipeline, delegating responsibilities to specialized data.hr_dataset and data.lr_dataset components for handling high-resolution and low-resolution images, respectively. These dataset-specific components rely on shared utilities: data.download_archive for fetching raw image archives, data._populate_cache for optimizing local storage and retrieval, and data._images_dataset for converting image files into a TensorFlow-compatible format. This modular design ensures a clear separation of concerns, allowing for independent management of HR and LR data while leveraging common functionalities for data handling.
The primary orchestrator for constructing the complete dataset, coordinating the acquisition and preparation of both high-resolution (HR) and low-resolution (LR) image data. It serves as the main interface for consumers to obtain the prepared dataset.
Related Classes/Methods:
Manages the entire lifecycle of high-resolution (HR) image data, including downloading, caching, and preparing it for use in the dataset. It ensures HR images are correctly sourced and made available. This component encapsulates the logic for managing HR cache indices and resolving HR image file paths.
Related Classes/Methods:
data.download_archive:188-192data._populate_cache:142-146data._images_dataset:135-140data._hr_cache_index:97-98data._hr_image_files:103-105
Manages the entire lifecycle of low-resolution (LR) image data, including downloading, caching, and preparing it for use in the dataset. It ensures LR images are correctly sourced and made available. This component encapsulates the logic for managing LR cache indices and resolving LR image file paths.
Related Classes/Methods:
data.download_archive:188-192data._populate_cache:142-146data._images_dataset:135-140data._lr_cache_index:100-101data._lr_image_files:107-109
A shared utility function responsible for securely downloading compressed image archives from specified external URLs using tf.keras.utils.get_file. It abstracts the complexities of file downloading.
Related Classes/Methods:
A shared utility method responsible for managing the local caching mechanism by iterating through a TensorFlow dataset to trigger the caching of decoded image data. It prevents redundant downloads and processing.
Related Classes/Methods:
A shared utility method that abstracts the process of creating a TensorFlow dataset from a collection of image file paths, handling the underlying dataset construction logic including reading and decoding PNG images.
Related Classes/Methods: