RFC: Closure expression optimization for predicate pushdown

[andrewgazelka]

Summary

This RFC proposes enabling nushell to analyze and optimize closures by inspecting their AST structure. This allows optimizations such as:

• Predicate pushdown to data sources (databases, Polars DataFrames)
• Pipeline stage reordering (e.g., filter before sort)

Motivation

Nushell already parses closures into an AST—they aren't compiled to opaque bytecode. The infrastructure exists; we just need to build the analysis pass.

Currently:

open data.db | query "SELECT * FROM users" | where { $in.age > 30 }

Fetches all rows and filters in-memory.

With this RFC:

# Same syntax, but predicate pushed to SQL: WHERE age > 30

Key points

• No syntax changes - users write closures as before
• Conservative fallback - unoptimizable closures execute unchanged
• Escape hatch - --no-optimize flag for when optimization causes issues
• Builds on existing IR work that enables "optimization passes in the future"

Prior art

• LINQ (C#) - Expression trees for database translation
• Spark SQL - Logical plan optimization
• Polars - Lazy expression DSL

---

Note: The RFC process here seems dormant (last merge 2020), but this felt like a good place to document the idea formally. Happy to discuss on Discord or move to a GitHub discussion if preferred.

[fdncred]

This is what I said in discord.

yes, we'd do something like this. in the past, we've talked about query optimization and pipeline optimization. this falls withing that type of thinking so it's something we've long been a fan of if done properly.