Duckbridge - a fully Pythonic interface to httpserver

[MHromiak]

Hello!

, , and I have been working on a workflow to lower the bar of entry to using CDM data for medical research. So far the flow uses DuckDB and dbt-synthea to synthesize millions of dummy patients' data and reduce storage by a factor of 10. (200GB -> 18GB!) However, storing this much data may be out of the question for aspiring researchers in infrastructure-poor regions of the world.

Thankfully, httpserver provides an answer. A user can host a database in one part of the world while making it available in other parts. httpserver's biggest remaining hurdle is that of language accessibility. Currently, users must use SQL and shell scripting with direct access to the database in order to manipulate the database's permissions and install extensions. Duckbridge "bridges" that gap by providing a fully-Pythonic interface. Now popular python workflows (i.e. Jupyter Notebooks, python scripts) will be able to harness httpserver's full potential while sticking to a single point of entry.

Future efforts will include Julia (courtesy of TheCedarPrince) and (potentially since I've never worked with it) R packages.

Feel free to check it out! A more polished version is published on PyPI and the library is actively being worked on. Additionally, a repository of example usages is being put together to help newcomers spin up with Duckbridge more rapidly.

[lmangani]

@MHromiak @katy-sadowski @TheCedarPrince this is ⭐ AMAZING ⭐ thank you for creating and sharing this!
If you'd like we'd gladly accept a README PR to showcase this possibility with the extensions users

[MHromiak]

@lmangani absolutely! What would you need?