Could Ducklake be integrated?

[mkmitchell]

It seems like Ducklake can handle many of the backend operations that rustac is doing. Has there been any discussion about somehow utilizing that instead of handling all that complex stuff here? Yes, you would still need to run Ducklake and it wouldn't be as simple as writing a new parquet file directly to blob storage and reading from them all but maybe that extra lift is worth it?

[gadomski]

It seems like Ducklake can handle many of the backend operations that rustac is doing.

rustac isn't doing much "complex stuff" — it's simply translating STAC JSON to stac-geoparquet and converting STAC API searches to DuckDB SQL. I don't think Ducklake (or DuckDB) does that out-of-the-box.

You can absolutely use DuckDB with parquet files written by rustac, e.g. https://stac-utils.github.io/rustac-py/latest/notebooks/its-live/.