Skip to content

Add support for downloader middleware to read WACZ files from Google Cloud Storage #24

@leewesleyv

Description

@leewesleyv

Currently, the downloader middleware supports reading WACZ files from local storage and Amazon S3. To enhance the flexibility of the middleware, it would be beneficial to add support for reading WACZ files stored in Google Cloud Storage (GCS).

Proposed Changes

Implement a new GCSZipStorageHandler class extending the ZipStorageHandler base class.
Use the Google Cloud Storage API to fetch files and metadata using range requests for optimal performance. Update the ZipStorageHandlerFactory to recognize gs:// URIs and return the GCSZipStorageHandler.

Tasks

  • Create GCSZipStorageHandler with methods for fetching files and metadata.
  • Update the ZipStorageHandlerFactory to handle gs:// URIs.
  • Write unit tests for GCSZipStorageHandler.
  • Update documentation to include GCS support.
  • Create GCSFileResolver with methods for resolving files matching a regex pattern.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions