Skip to content

[Enh]: Add SQL-like string parsing into Narwhals expressions (or document examples) #3310

@danielgafni

Description

@danielgafni

We would like to learn about your use case. For example, if this feature is needed to adopt Narwhals in an open source project, could you please enter the link to it below?

https://github.com/anam-org/metaxy

Please describe the purpose of the new feature or describe the problem to solve.

I would like to be able to use pieces of SQL(-like) - in my case filters - in yaml files and CLI args, and have them parsed into Narwhals expressions. We were able to achieve some success with the help of sqlglot, but that's not finalized yet.

I would also be like to do the opposite operation, but that's not required right now.

Clearly this would be a very useful feature for Narwhals. It will enable so many use cases for partially templated workflows, or for workflows that can make use of user input at launch-time (e.g. having CLIs, config editors).

I'd like to have something like this:

table_a: ~

table_b:
  deps: [table_a]
  filters: 
    - "user_age > 18"

Or:

python script.py --filter "user_age > 18"

I would like to have a standard way (or even just a documented example) to parse "user_age > 18" into nw.col("user_age") > 18.

Suggest a solution if possible.

@geoHeil had success with implementing this (via Pydantic) here: anam-org/metaxy#232

If you have tried alternatives, please describe them below.

No response

Additional information that may help us understand your needs.

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions