ProjectDB Query Parser

[esoergel]

Product Description

Technical Summary

https://docs.google.com/document/d/1ayuYQx7DD6dfc2-zzeLat4R8kXZkTp6WlPcD4rcw0YU/edit?tab=t.8pn5qhzhr6br#heading=h.a9lfwm9d4qth

Initial work on parsing user-provided SQL, interpreting a limited set, and rewriting it in SQLAlchemy.

Feature Flag

Safety Assurance

Safety story

Automated test coverage

QA Plan

Rollback instructions

• This PR can be reverted after deploy with no further considerations

Labels & Review

• Risk label is set correctly
• The set of people pinged as reviewers is appropriate for the level of risk of the change

[esoergel]

@MartinRiese @millerdev (and whoever else is interested) I'd be interested in your big picture take on this idea. Still need to spend more time noodling on it, but this is the core of the work, I think.

My hope is that by using SQL as a DSL for SQL queries, we can express basic table joining and filtering semantics. This would sit underneath Martin's GUI query builder, which would essentially output a WHERE clause on top of this. That is, projects would get a text box to define the shape of the data source for their case search endpoint (and future applications of this dataset), and then use the query builder to do parametrized filtering on top of that.

Beyond what's here, I think I'll add support for UNION and aliasing column names, and I think that should take us most of the way to being able to express what's needed for basic cross-table queries. Once that's done I'll also be looking at real queries from the BHA app to check what I'm missing (with the knowledge that the query builder will do most of the filtering). Then I'd like to add a simple report UI to write and test these queries on HQ, basically a text box for input, then show the resulting SQL and the first N rows of the result. Probably eventually add EXPLAIN ANALYZE too.

Anyways, before I go much farther down this route, I wanted to look for big picture feedback. I'd hoped to avoid building this kind of thing in-house, but didn't see any great alternatives. As you'll see from the code, it's a bit fiddly, but still seems doable. It is only capable of producing queries on the tables that are passed in to the caller, and all functionality must be explicitly implemented by us, which meets what I'd hoped from a safety standpoint. (I'm also working on the side on using per-domain SQL users so postgres roles can provide a second layer of data access control)

[millerdev]

I like the overall direction. Seems very powerful.

I did not do a detailed review looking for edge cases or places where unexpected query patterns might slip through (allowing things that we do not intend).

[millerdev]

What does this string representation look like? Rather than str(expected), would it make sense to have expected be a string literal?

Alternately, would result == expected work?

str(result) == str(expected) seems like it could produce false positives if the string representation is generic (like "<select object>"). Do these select objects always have a unique/meaningful string representation that expresses all query elements?