Skip to content

Conversation

@fivetran-felixhuang
Copy link
Collaborator

For the GREATEST function

In BigQuery:
- if any argument is NULL, the result is NULL
In DuckDB:
- NULLs are ignored, returns greatest non-NULL value.

The workaround here is to make sure that the transpiled DuckDB query returns NULL is any argument is NULL

BigQuery: GREATEST(1, 2, NULL, 3) -> DuckDB: CASE WHEN 1 IS NULL OR 2 IS NULL OR NULL IS NULL OR 3 IS NULL THEN NULL ELSE GREATEST(1, 2, NULL, 3) END

@fivetran-felixhuang fivetran-felixhuang force-pushed the transpile_greatest_bq_duckdb branch from e9cfb7c to f58c999 Compare November 19, 2025 17:53
@geooo109
Copy link
Collaborator

Can we also add tests for this ?

@fivetran-felixhuang
Copy link
Collaborator Author

Can we also add tests for this ?

@geooo109 added some tests, please let me know if they are sufficient

@fivetran-felixhuang fivetran-felixhuang changed the title support transpilation of GREATEST from BigQuery to DuckDB Feature: support transpilation of GREATEST from BigQuery to DuckDB Nov 21, 2025
@fivetran-felixhuang fivetran-felixhuang force-pushed the transpile_greatest_bq_duckdb branch from 4ea1569 to 05a38f6 Compare November 21, 2025 17:24
@georgesittas
Copy link
Collaborator

is greatest null / nan handling preserved for bigquery -> duckdb? perhaps least should be supported in a follow-up pr

@geooo109
Copy link
Collaborator

geooo109 commented Nov 21, 2025

So, @fivetran-felixhuang the LEAST + NaN seems a bug for duckdb (nice finding). We will confirm it.
For now, It's safer to make a separate PR for LEAST as @georgesittas suggested.
As a result, lets keep the build_greatest as is. Just remove the test from transpile.py file.

@fivetran-felixhuang
Copy link
Collaborator Author

is greatest null / nan handling preserved for bigquery -> duckdb? perhaps least should be supported in a follow-up pr

For GREATEST, the NaN handling is the same for both the BigQuery and DuckDB, but NULL handling is not. This PR is to address the NULL handling issue.

I will work on the NaN handling issue for LEAST in a different PR

Copy link
Collaborator

@georgesittas georgesittas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one last comment, I don't think we need _build_greatest for DuckDB.


FUNCTIONS = {
**parser.Parser.FUNCTIONS,
"GREATEST": _build_greatest,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this right? It should be parsed just fine out of the box.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we do. When I removed, the arguments were not parsed correctly for some reason....

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's due to is_var_len_args. Without doing what I did in parser.py, we'd break the AST by spilling args[2:] to null_if_any_null, which is not the right representation.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks!

@georgesittas
Copy link
Collaborator

@fivetran-felixhuang let's not worry about nan for now, I've reached out to the duckdb community to see if this is a bug instead of a feature. We can revisit later.

@georgesittas georgesittas merged commit 5bb1170 into main Nov 21, 2025
8 checks passed
@georgesittas georgesittas deleted the transpile_greatest_bq_duckdb branch November 21, 2025 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants