Skip to content

Commit dba0816

Browse files
committed
feat: parse JsonAccess as a custom binary operator
- Needed in datafusion-contrib/datafusion-variant#26 - `sqlparser-rs` currently exposes the colon operator (`:`) as a special `JsonAccess` expression. So it fails in datafusion's parsing before an `ExprPlanner` is even invoked. - Fix it by converting `JsonAccess` to a normal binary expr, on which the `ExprPlanner` is invoked and custom parsing can be done. I have arrived at this solution iteratively to solve datafusion-contrib/datafusion-variant#26. Some alternatives I considered (but NOT IMPLEMENTED): - Add a new `Operator::Colon` variant instead of `BinaryOperator::Custom`. This will make more sense to do if datafusion plans on supporting the colon operator on other data types (like json strings). - Add a new `Expr::JsonAccess` instead of re-using `BinaryExpr`. Alongside this, we could also introduce datafusion's version of a `JsonPath` struct. `Expr::JsonAccess` could also contain the optional cast that usually comes after it (like in `col_a:field_b::int`). The cast might be needed for performance optimizations. This is a much bigger change, so I have not done it. If either of these make sense to do at this time, please let me know and I will do theose changes.
1 parent 73fbd48 commit dba0816

File tree

1 file changed

+21
-2
lines changed

1 file changed

+21
-2
lines changed

datafusion/sql/src/expr/mod.rs

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,8 @@ use datafusion_expr::planner::{
2222
use sqlparser::ast::{
2323
AccessExpr, BinaryOperator, CastFormat, CastKind, CeilFloorKind,
2424
DataType as SQLDataType, DateTimeField, DictionaryField, Expr as SQLExpr,
25-
ExprWithAlias as SQLExprWithAlias, MapEntry, StructField, Subscript, TrimWhereField,
26-
TypedString, Value, ValueWithSpan,
25+
ExprWithAlias as SQLExprWithAlias, JsonPath, MapEntry, StructField, Subscript,
26+
TrimWhereField, TypedString, Value, ValueWithSpan,
2727
};
2828

2929
use datafusion_common::{
@@ -651,10 +651,29 @@ impl<S: ContextProvider> SqlToRel<'_, S> {
651651
options: Box::new(WildcardOptions::default()),
652652
}),
653653
SQLExpr::Tuple(values) => self.parse_tuple(schema, planner_context, values),
654+
SQLExpr::JsonAccess { value, path } => {
655+
self.parse_json_access(schema, planner_context, value, &path)
656+
}
654657
_ => not_impl_err!("Unsupported ast node in sqltorel: {sql:?}"),
655658
}
656659
}
657660

661+
fn parse_json_access(
662+
&self,
663+
schema: &DFSchema,
664+
planner_context: &mut PlannerContext,
665+
value: Box<SQLExpr>,
666+
path: &JsonPath,
667+
) -> Result<Expr> {
668+
self.build_logical_expr(
669+
BinaryOperator::Custom(":".to_owned()),
670+
self.sql_to_expr(*value, schema, planner_context)?,
671+
// pass json path as a string literal, let the impl parse it when needed.
672+
Expr::Literal(ScalarValue::Utf8(Some(path.to_string())), None),
673+
schema,
674+
)
675+
}
676+
658677
/// Parses a struct(..) expression and plans it creation
659678
fn parse_struct(
660679
&self,

0 commit comments

Comments
 (0)