Introducing Narwhals as a dependency to interoperate between dataframe libraries

[gregorywaynepower]

Hey Folks,

I've seen some requests for integrations between DuckDB and polars. I know y'all already have an integration with pandas dataframes. I wanted to propose Narwhals as a solution to keep y'all from having to build custom integrations between various dataframe libraries within the Python ecosystem.

[tomweber-sas]

Hey Gregory, it's me again. Sorry I didn't reply when you first posted this. They are keeping me too busy with regular SAS work. I'm looking at that doc, esp the API reference and I'm not seeing where the integration would be. This seems to be a package that wraps other existing dataframes and provides methods that 'normalize' these other data frames discrepancies or variations for those methods. I don't see any I/O methods for reading/writing to Narwhal data frames; I don't see there are Narwhal dataframes, just ways to use it's API against other already existing df's. If it had some read/write methods that allowed you to use it to create or stream other types of df's using one api that handled any data, then that would make sense to me. I don't see anything like that. So, I'm not sure how this fits. But again, that's a quick look at their doc. Not a full investigation of it.
Tom

[gregorywaynepower]

You are right that they do offer wrappers around existing dataframes. You do have an option of accepting any flavor of dataframe and exporting out it out to pandas. https://narwhals-dev.github.io/narwhals/basics/dataframe_conversion/ Which I think could be useful for interfacing with external libraries.