Wrapper scripts to simplify export of a large table to azure blob storage.
Assumes external table has already been provisioned.
Requires that the table to be exported has some sort of timestamp column, either datetime or numeric unix time.
Because this module is designed to be long-running and bound to a laptop session, leveraging PowerToys Awake is recommended.
Anecdotally, .export async only gives about 5 mb/sec throughput per thread. Additionally, there is a 60-minute hard limit timeout after which any .export command will be terminated by the cluster.
You need to batch & parallelize your .export commands to maximize throughput and minimize errors. This module provides functions and helper scripts for that.
Just needed to quickly pick something short & distinct. Xb == "Export Batch".