Skip to content

Warchafter/b2-s3-uploader

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

B2 Image Uploader & WebP Converter

This PHP script automates the process of converting images to WebP format and uploading them to Backblaze B2 cloud storage. It is designed to be run from the command line.

Purpose

The primary goal of this project is to provide an efficient way to:

  1. Scan a local directory for images.
  2. Convert supported image formats (JPEG, PNG) to the highly optimized WebP format.
  3. Upload these WebP images to a specified Backblaze B2 bucket.
  4. Maintain the original directory structure in the B2 bucket.
  5. Verify uploads by downloading the files and checking their SHA1 checksums.

Features

  • Automatic WebP Conversion: Converts JPG and PNG images to WebP.
  • Backblaze B2 Integration: Securely uploads files to B2 using API v3.
  • Directory Structure Preservation: Replicates the local folder structure in the B2 bucket.
  • Environment-Based Configuration: Uses a .env file for secure credential management.
  • Progress Logging: Provides clear CLI output for each step of the process.
  • Upload Verification: Downloads uploaded files from B2 to a local directory and verifies their integrity using SHA1 checksums.
  • Recursive File Discovery: Scans through subdirectories to find all images.

Prerequisites

  • PHP 7.4 or higher
  • Composer (for dependency management)
  • gd PHP extension (for image processing)
  • A Backblaze B2 account with an Application Key and Key ID.
  • A B2 Bucket.

Installation

  1. Clone the repository:

    git clone <your-repository-url>
    cd <repository-directory>
  2. Install PHP dependencies using Composer:

    composer install

    This will install phpdotenv and other necessary libraries defined in composer.json.

  3. Set up environment variables:

    • Create a .env file in the project root by copying the example file:
      cp .env.example .env
    • Edit the .env file and fill in your Backblaze B2 credentials and Bucket ID:
      B2_KEY_ID=your_b2_key_id
      B2_APP_KEY=your_b2_application_key
      B2_BUCKET_ID=your_b2_bucket_id
  4. Prepare image directories:

    • The script expects images to be in a directory named original-test-images (or as configured in local-uploader.php).
    • Converted WebP images will be temporarily stored in test-images before upload.
    • Verified downloads will be saved to downloaded-test-images.

    You can change these directory paths by modifying the $localFolder and $webpFolder variables at the top of local-uploader.php.

Usage

To run the script, navigate to the project directory in your terminal and execute:

php local-uploader.php

The script will then:

  1. Authenticate with Backblaze B2.
  2. Scan the $localFolder for images.
  3. Convert each image to WebP format and save it to $webpFolder, maintaining the subdirectory structure.
  4. Upload the WebP images to your B2 bucket, preserving the relative paths.
  5. Download the uploaded files from B2 to the downloaded-test-images directory.
  6. Verify the integrity of the downloaded files against the uploaded ones using SHA1 checksums.

Progress, errors, and success messages will be printed to the console.

Configuration

Most configuration is handled via environment variables in the .env file:

  • B2_KEY_ID: Your Backblaze B2 Key ID.
  • B2_APP_KEY: Your Backblaze B2 Application Key.
  • B2_BUCKET_ID: The ID of the B2 bucket you want to upload files to.

Other configurations within local-uploader.php:

  • $localFolder: Path to the directory containing original images. Default: __DIR__ . '/original-test-images/'.
  • $webpFolder: Path to the directory where WebP images will be temporarily stored. Default: __DIR__ . '/test-images/'.
  • The WebP conversion quality (1-100) is hardcoded to 75 in the anyImageToWebp function call within local-uploader.php. You can modify this if needed.

Directory Structure

.
├── .env                # Local environment variables (DO NOT COMMIT)
├── .env.example        # Example environment variables
├── .gitignore          # Specifies intentionally untracked files
├── any-to-webp.php     # Script for image to WebP conversion
├── composer.json       # Defines project dependencies for Composer
├── composer.lock       # Records exact versions of dependencies
├── downloaded-test-images/ # Stores downloaded files for verification (created automatically)
├── local-uploader.php  # Main script for processing and uploading
├── original-test-images/ # Place your original images here
├── README.md           # This documentation file
├── test-images/        # Temporary storage for converted WebP images (created automatically)
└── vendor/             # Composer dependencies (managed by Composer)

Error Handling

The script includes basic error handling and will output messages to the console:

  • Authentication failures.
  • Failures in obtaining an upload URL.
  • Invalid or empty files.
  • Image conversion failures.
  • Checksum mismatches during download verification.

If a fatal error occurs, the script will typically exit with a non-zero status code.

Contributing

Contributions are welcome! Please feel free to submit a pull request or open an issue if you find any bugs or have suggestions for improvements.

When contributing, please ensure:

  1. Your code adheres to the existing style.
  2. You have tested your changes thoroughly.
  3. You update documentation if necessary.

License

This project is open-source. You can specify a license here (e.g., MIT License). If no license is specified, default copyright laws apply.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages