⚗️ Benchmark GDAL Libertiff driver vs async-tiff (#3)

* ⚗️ Benchmark GDAL Libertiff driver vs async-tiff

Obtain Sentinel-2 TCI image and convert it to a tiled GeoTIFF, apply LZW compression with Horizontal differencing predictor. Then run benchmarks comparing GDAL's Libertiff driver against async-tiff, using 16 threads.

* 💚 Download and process sample GeoTIFF

Need to remember to put the sample GeoTIFF on CI. Processing it with a newer version of GDAL from official docker image.

* 👷 Run on ghcr.io/osgeo/gdal:ubuntu-small-3.12.0

Try running directly on official docker image for GDAL 3.12.0. Should have Libertiff driver included.

* 🔥 One-liner to download and process TCI.tif file

The ghcr.io/osgeo/gdal:ubuntu-small-3.12.0 docker image doesn't have wget, so might as well use a one-liner to get and process the file.

* 👷 Compile on ghcr.io/osgeo/gdal:ubuntu-small-3.11.5

Fixing various linker issues, also pin georust/gdal to a version that supports GDAL 3.11, xref georust/gdal#664.

* 🛂 Run docker container with --privileged

Try to fix `setarch: failed to set personality to x86_64: Operation not permitted` on CodSpeedHQ/action. Adapted from suggestion at https://stackoverflow.com/questions/57830375/github-actions-workflow-error-permission-denied/70207187#70207187

* ⬆️ Bump CodSpeedHQ/action from 4.3.3 to 4.3.4

Bumps [CodSpeedHQ/action](https://github.com/codspeedhq/action) from 4.3.3 to 4.3.4.
- [Release notes](https://github.com/codspeedhq/action/releases)
- [Changelog](https://github.com/CodSpeedHQ/action/blob/main/CHANGELOG.md)
- [Commits](CodSpeedHQ/action@v4.3.3...v4.3.4)

Also install git following instructions at https://codspeed.io/docs/integrations/ci/github-actions#using-container-images

* 📌 Install Criterion.rs compatibility layer for CodSpeed

Xref https://codspeed.io/docs/benchmarks/rust/criterion#installation

* 💚 Only do actual codspeed run on Rust stable

Compile on Rust MSRV (1.85), but run benchmarks on stable (currently 1.91.0).

* 🧵 Parametrize functions to run on n_threads=X

Allow setting number of threads to use for both the read_geotiff_gdal and read_geotiff_async_tiff functions. Set to 4 cores to align with GitHub Actions linux runners.

For some reason, LiberTIFF doesn't seem to respect NUM_THREADS properly? Either it uses 1 core or all cores...

* ⬆️ Bump MSRV from 1.85.0 to 1.89.0

To get temporary lifetime extension, xref rust-lang/rust#140593

* 👷 Consolidate apt install steps

Author: weiji14

.github/workflows/bench.yml

@@ -19,21 +19,30 @@ permissions:
 jobs:
   rust:
     runs-on: ubuntu-24.04
+    container:
+      image: ghcr.io/osgeo/gdal:ubuntu-small-3.11.5
+      options: --privileged
     strategy:
       matrix:
         toolchain:
-          - 1.85.0 # msrv
+          - 1.89.0 # msrv
           - stable
     steps:
+      - name: Install dev dependencies and setup git
+        run: |
+          apt update
+          apt install -y build-essential cmake git libclang-dev pkg-config
+          git config --global --add safe.directory $GITHUB_WORKSPACE
+
       - name: Checkout repository
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
         with:
           persist-credentials: false
 
-      - name: Install apt packages
+      - name: Download and process sample GeoTIFF
         run: |
-          sudo apt update
-          sudo apt install -y --no-install-recommends libgdal-dev
+          gdal raster convert --co COMPRESS=LZW --co TILED=YES --co PREDICTOR=2 https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/37/M/BV/2024/10/S2A_37MBV_20241029_0_L2A/TCI.tif benches/TCI_lzw.tif
+          ls -lh benches/
 
       - name: Setup rust toolchain, cache and cargo-codspeed binary
         uses: moonrepo/setup-rust@ede6de059f8046a5e236c94046823e2af11ca670 # v1.2.2
@@ -47,7 +56,8 @@ jobs:
         run: cargo codspeed build
 
       - name: Run the benchmarks
-        uses: CodSpeedHQ/action@bb005fe1c1eea036d3894f02c049cb6b154a1c27 # v4.3.3
+        if: matrix.toolchain == 'stable'
+        uses: CodSpeedHQ/action@6a8e2b874c338bf81cc5e8be715ada75908d3871 # v4.3.4
         with:
           mode: instrumentation
           run: cargo codspeed run

Cargo.lock

@@ -6,7 +6,7 @@ description = "Accelerating GeoTIFF readers with Rust."
 readme = "README.md"
 repository = "https://github.com/weiji14/foss4g2025"
 license = "MPL-2.0"
-rust-version = "1.85.0"
+rust-version = "1.89.0"
 authors = ["Wei Ji <[email protected]>"]
 
 [dependencies]
@@ -18,9 +18,12 @@ cudarc = { version = "0.17.7", features = [
 [dev-dependencies]
 async-tiff = { version = "0.1.0", git = "https://github.com/developmentseed/async-tiff", tag = "py-v0.2.0" }
 cog3pio = "0.0.1"
-criterion = "0.7.0"
+criterion = { package = "codspeed-criterion-compat", version = "4.1.0" }
 dlpark = { version = "0.6.0", features = ["ndarray"] }
-gdal = { version = "0.18.0", features = ["array", "bindgen"] }
+gdal = { git = "https://github.com/georust/gdal.git", version = "0.18.0", features = [
+    "array",
+    "bindgen",
+], rev = "3df2c28f8c83f80d209b34d46f30d6c35e803e07" }
 ndarray = "0.16.1"
 object_store = "0.12.4"
 rayon = "1.11.0"

Cargo.toml

@@ -53,6 +53,28 @@ be generated, run the following commands with sudo:
     sed --in-place "s/stream=0/stream/g" /usr/include/nvtiff.h
     sed --in-place "s/nvtiffTagDataType type/enum nvtiffTagDataType type/g" /usr/include/nvtiff.h
 
+## Running the benchmarks
+
+To try out the code, start by cloning this [repo-url](/../../)
+
+    git clone <repo-url>
+    cd foss4g2025
+
+Next, download a sample Sentinel-2 True Colour Image (TCI) GeoTIFF file (318.0MB), and
+convert it to a tiled GeoTIFF, changing from Deflate to LZW compression while setting
+Horizontal differencing predictor (272.2MB).
+
+    gdal raster convert --co COMPRESS=LZW --co TILED=YES --co PREDICTOR=2 https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/37/M/BV/2024/10/S2A_37MBV_20241029_0_L2A/TCI.tif benches/TCI_lzw.tif
+
+Then, use [cargo](https://doc.rust-lang.org/cargo/index.html) to install the
+dependencies, compile and run the benchmarks like so:
+
+    cargo bench
+
+Output results will be printed to the terminal, and a HTML report can be viewed by
+opening the `target/criterion/report/index.html` file.
+
+
 ## License
 
 All code in this repository is licensed under