z_order optimisation on s3 with locking provider

[Libbum]

Hi! Using the python wrapper for this task, but happy to work with rust too if the solution is available there.

I'm looking at the z order docs and can see it's possible to pass in WriterProperties to the method, but not storage_options.

When using AWS and storing data on s3, I've got a dynamodb table configured as a lock client, so when I'm calling write_delta(..., storage_options={"AWS_S3_LOCKING_PROVIDER": "dynamodb"}, ...) is passed. This option is outside of the WriterProperties struct, so if I call dt.optimize.z_order(["some_id"]) it will fail with the

DeltaProtocolError: Atomic rename requires a LockClient for S3 backends. Either configure the LockClient, or set AWS_S3_ALLOW_UNSAFE_RENAME=true to opt out of support for concurrent writers.

complaint. Any idea how I can tell z_order my storage_options?

[ion-elgreco]

You pass those storage options in the DeltaTable object, every class method uses than that instance to do it's operations.

[Libbum]

Ah. Perfect! That makes sense. Thank you!