Support for bbox filtering with Geoparquet > 1.1

[scottyhq]

The recent release of the geoparquet 1.1 format specification implemented new options for bbox filtering and encoding that could help analysis of returned sliderule results (for a motivating example see https://developmentseed.org/lonboard/latest/examples/overture-maps/). I'm curious if it seems straightforward to support this new version? Happy to try and help out!

I used the https://github.com/planetlabs/gpq to check the current versioning gpq describe output.geoparquet from sliderule 4.5.3 ( GeoParquet Version │ 1.0.0-beta.1)

In the meantime, I'm planning to explore converting to the new format with GDAL>=3.9 / geopandas v1 https://geopandas.org/en/stable/docs/reference/api/geopandas.GeoDataFrame.to_parquet.html (which still uses 1.0 as the default).

[dshean]

Sounds good to me! I also appreciate you kicking the tires with geopandas v1 for SlideRule - I feel like this has come up in the past, and maybe we're OK, but my guess is we will need to update some things.

[scottyhq]

Just quick update that current sliderule (v4.19.0) supports GeoParquet v1.0 output (so WKB instead of geoarrow geometry encoding & no bboxes on points). Would be good to evaluate at what point adding the bbox column helps for large-scale analyses...

Modified quickstart example:

from sliderule import sliderule

grand_mesa = [ {"lon": -108.3435200747503, "lat": 38.89102961045247},
               {"lon": -107.7677425431139, "lat": 38.90611184543033},
               {"lon": -107.7818591266989, "lat": 39.26613714985466},
               {"lon": -108.3605610678553, "lat": 39.25086131372244},
               {"lon": -108.3435200747503, "lat": 38.89102961045247} ]

parms = {
    "poly": grand_mesa,
    "track": 1,
    "cnf": 0,
    "fit": {},
    "output": {
    "path": "quickstart-sliderule.parquet",
    "format": "parquet",
    "as_geo": True, 
    "open_on_complete": True}
}

gdf = sliderule.run("atl03x", parms)

gpq describe quickstart-sliderule.parquet

╭────────────────────────┬────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬────────────┬─────────────┬──────────┬─────────────── ≈
│ COLUMN                 │ TYPE   │ ANNOTATION                                                                                                           │ REPETITION │ COMPRESSION │ ENCODING │ GEOMETRY TYPES ≈
├────────────────────────┼────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────┼──────────┼─────────────── ≈
│ time_ns                │ int64  │ timestamp(isadjustedtoutc=false, timeunit=nanoseconds, is_from_converted_type=false, force_set_converted_type=false) │ 0..1       │ zstd        │          │                ≈
│ pflags                 │ int32  │ int(bitwidth=16, issigned=false)                                                                                     │ 0..1       │ zstd        │          │                ≈
│ region                 │ int32  │ int(bitwidth=8, issigned=false)                                                                                      │ 0..1       │ zstd        │          │                ≈
│ gt                     │ int32  │ int(bitwidth=8, issigned=false)                                                                                      │ 0..1       │ zstd        │          │                ≈
│ n_fit_photons          │ int32  │                                                                                                                      │ 0..1       │ zstd        │          │                ≈
│ w_surface_window_final │ float  │                                                                                                                      │ 0..1       │ zstd        │          │                ≈
│ rgt                    │ int32  │ int(bitwidth=16, issigned=false)                                                                                     │ 0..1       │ zstd        │          │                ≈
│ rms_misfit             │ float  │                                                                                                                      │ 0..1       │ zstd        │          │                ≈
│ h_mean                 │ float  │                                                                                                                      │ 0..1       │ zstd        │          │                ≈
│ dh_fit_dx              │ float  │                                                                                                                      │ 0..1       │ zstd        │          │                ≈
│ x_atc                  │ double │                                                                                                                      │ 0..1       │ zstd        │          │                ≈
│ h_sigma                │ float  │                                                                                                                      │ 0..1       │ zstd        │          │                ≈
│ spot                   │ int32  │ int(bitwidth=8, issigned=false)                                                                                      │ 0..1       │ zstd        │          │                ≈
│ y_atc                  │ float  │                                                                                                                      │ 0..1       │ zstd        │          │                ≈
│ srcid                  │ int32  │                                                                                                                      │ 0..1       │ zstd        │          │                ≈
│ cycle                  │ int32  │ int(bitwidth=8, issigned=false)                                                                                      │ 0..1       │ zstd        │          │                ≈
│ photon_start           │ int32  │ int(bitwidth=32, issigned=false)                                                                                     │ 0..1       │ zstd        │          │                ≈
│ geometry               │ binary │                                                                                                                      │ 0..1       │ zstd        │ WKB      │ Point          ≈
├────────────────────────┼────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────────┴─────────────┴──────────┴─────────────── ≈
│ Rows                   │ 260159                                                                                                                                                                               ≈
│ Row Groups             │ 1                                                                                                                                                                                    ≈
│ GeoParquet Version     │ 1.0.0                                                                                                                                                                                ≈
╰────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────