Added support for json functions by ritwizsinha · Pull Request #546 · duckdb/pg_duckdb

ritwizsinha · 2025-01-17T17:26:16Z

Addresses #516

ritwizsinha · 2025-01-17T17:40:01Z

@JelteF I have added all the json functions present in duckdb to the sql file, is this fine or do we need to break this up. All of this is quite repeated and most of it is tests, so I doubt a big PR is a problem

JelteF · 2025-01-20T17:00:42Z

sql/pg_duckdb--0.2.0--0.3.0.sql

 COMMENT ON DOMAIN pg_catalog.blob IS 'The DuckDB BLOB alias for BYTEA';
+
+-- json_exists
+CREATE FUNCTION @extschema@.json_exists(json VARCHAR , path VARCHAR)


It seems confusing that these functions expect a VARCHAR type for the json argument. I'd expect that argument to be of the json or jsonb (or have functions for both of them).

Just added for json

JelteF · 2025-01-20T17:01:49Z

src/pgduckdb_metadata_cache.cpp

-	                                "iceberg_snapshots", "delta_scan", "read_json",    "approx_count_distinct"};
+	const char *function_names[] = {"read_parquet",      "read_csv",     "iceberg_scan",       "iceberg_metadata",
+	                                "iceberg_snapshots", "delta_scan",   "read_json",          "approx_count_distinct",
+	                                "json_exists",       "json_extract", "json_extract_string"};


It seems this list is missing a bunch of the functions that you added.

JelteF · 2025-01-20T17:08:43Z

test/regression/sql/json_functions_duckdb.sql

@@ -0,0 +1,144 @@
+-- <JSON_EXISTS>
+-- Test 1: Path exists in a simple JSON object
+SELECT json_exists('{"a": {"b": 1}}', '$.a.b'); -- Expected: true


I think most of these tests simply test that DuckDB works correctly, not so much the pg_duckdb intgration.

I think for each function it should give us enough confidence into pg_duckdb to add one test with reasonable arguments, and then tests with the various combinations of NULL arguments. So while I really appreciate the effort to add extensive tests, I don't think they add much value and would in practice only increase test runtime.

Reduced the tests

ritwizsinha · 2025-01-22T13:38:44Z

@JelteF structures are not supported as types in pg_duckdb so not possible to implement functions like:

json_transform(json, structure)	Transform json according to the specified structure.
from_json(json, structure)	Alias for json_transform.
json_transform_strict(json, structure)	Same as json_transform, but throws an error when type casting fails.
from_json_strict(json, structure)	Alias for json_transform_strict.

Added all the other ones

ritwizsinha · 2025-01-23T06:39:01Z

Curious why this fails on postgres 17

JelteF · 2025-01-24T11:58:37Z

sql/pg_duckdb--0.2.0--0.3.0.sql

I just merged #531. Which introduces a more re-usable way of doing this:

AS 'MODULE_PATHNAME', 'duckdb_only_function' LANGUAGE C;

…unctions

JelteF

I played around with this a bunch and added a few more tests and fixed a bunch of issues I found. I now think this is good enough to merge. @Y-- @mkaruza can either of you do another small review.

ritwizsinha added 2 commits January 17, 2025 22:55

Added support for json functions

14f751e

Format file

dcf3db7

ritwizsinha added 2 commits January 17, 2025 23:16

Add json_extract_string

6d7090b

Run format

a6ff51b

JelteF reviewed Jan 20, 2025

View reviewed changes

JelteF requested changes Jan 20, 2025

View reviewed changes

ritwizsinha added 3 commits January 22, 2025 10:14

Add functions to metadata cache

49f75a5

Add all json functions support

63ebf10

format

6a6069c

Remove jsonb tests

5840c50

ritwizsinha requested a review from JelteF January 22, 2025 14:14

Make the tests pass

2563c73

JelteF and others added 3 commits January 23, 2025 15:36

Fix for PG17, in which JSON became a keyword

7f0e183

Remove tests giving different outputs in postgres 17

09ed864

Remove from expected

be1792a

JelteF reviewed Jan 24, 2025

View reviewed changes

Merge branch 'main' into json-functions

4daeed0

ritwizsinha force-pushed the json-functions branch from 8fd291f to 4daeed0 Compare January 24, 2025 12:51

ritwizsinha and others added 5 commits January 24, 2025 18:22

Run formatting

1b347e5

Move json functions to end of sql migration file

211bfbf

Make json functions work with duckdb.unresolved_type and jsonb

ded869a

Also include ruleutils change for PG14-PG16

aff27ac

Merge branch 'main' into json-functions

9a32562

JelteF added the JSON Something related to JSON support label Jan 28, 2025

JelteF added 3 commits January 29, 2025 10:24

Add COMMENT on duckdb.json type

88768f4

Test that we handle JSON of different Postgres types correctly

8430e0a

Change tests a bit and use duckdb.json type for json and json_valid f…

1f2202f

…unctions

JelteF added 4 commits January 29, 2025 13:35

Define JSON aggregate functions as aggregates and fix json_group_object

b715ac5

Tests for json functions that return STRUCT type and fix return types

974c068

Add some more json function variants

3d7d1e0

Add even more json function variants

6c89e16

JelteF approved these changes Jan 29, 2025

View reviewed changes

JelteF and others added 3 commits January 29, 2025 18:06

Add subquery aliases to make PG14/15 happy

13bbf4b

Fix test output

2eba816

Merge branch 'main' into json-functions

4108d17

wuputah requested review from Y-- and mkaruza February 3, 2025 20:09

Y-- approved these changes Feb 3, 2025

View reviewed changes

Y-- merged commit d9bb1a3 into duckdb:main Feb 3, 2025
5 checks passed

JelteF mentioned this pull request Mar 18, 2025

Better support for JSONB #516

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added support for json functions#546

Added support for json functions#546
Y-- merged 28 commits intoduckdb:mainfrom
ritwizsinha:json-functions

ritwizsinha commented Jan 17, 2025 •

edited

Loading

Uh oh!

ritwizsinha commented Jan 17, 2025 •

edited

Loading

Uh oh!

JelteF Jan 20, 2025

Uh oh!

ritwizsinha Jan 22, 2025

Uh oh!

JelteF Jan 20, 2025

Uh oh!

ritwizsinha Jan 22, 2025

Uh oh!

JelteF Jan 20, 2025

Uh oh!

ritwizsinha Jan 22, 2025

Uh oh!

ritwizsinha commented Jan 22, 2025

Uh oh!

ritwizsinha commented Jan 23, 2025

Uh oh!

JelteF Jan 24, 2025 •

edited

Loading

Uh oh!

ritwizsinha Jan 24, 2025

Uh oh!

JelteF left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ritwizsinha commented Jan 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ritwizsinha commented Jan 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JelteF Jan 20, 2025

Choose a reason for hiding this comment

Uh oh!

ritwizsinha Jan 22, 2025

Choose a reason for hiding this comment

Uh oh!

JelteF Jan 20, 2025

Choose a reason for hiding this comment

Uh oh!

ritwizsinha Jan 22, 2025

Choose a reason for hiding this comment

Uh oh!

JelteF Jan 20, 2025

Choose a reason for hiding this comment

Uh oh!

ritwizsinha Jan 22, 2025

Choose a reason for hiding this comment

Uh oh!

ritwizsinha commented Jan 22, 2025

Uh oh!

ritwizsinha commented Jan 23, 2025

Uh oh!

JelteF Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ritwizsinha Jan 24, 2025

Choose a reason for hiding this comment

Uh oh!

JelteF left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ritwizsinha commented Jan 17, 2025 •

edited

Loading

ritwizsinha commented Jan 17, 2025 •

edited

Loading

JelteF Jan 24, 2025 •

edited

Loading