Skip to content

feat: json constructor and operators #7112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 33 commits into
base: main
Choose a base branch
from
Draft

feat: json constructor and operators #7112

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
5901a1a
chore: update datafusion to 50
MichaelScofield Oct 11, 2025
b39a712
fix ci
MichaelScofield Oct 13, 2025
4ac4759
fix: update datafusion_pg_catalog import
sunng87 Oct 13, 2025
0695bf2
chore: fix toml format
sunng87 Oct 13, 2025
eceeed4
chore: fix toml format again
sunng87 Oct 13, 2025
5b5f280
fix nextest
MichaelScofield Oct 14, 2025
6a9f08b
fix sqlness
MichaelScofield Oct 15, 2025
a45c065
chore: switch datafusion-orc to upstream tag
sunng87 Oct 17, 2025
15da1ed
feat: wip to_json udf
sunng87 Oct 17, 2025
b43cdab
chore: add information extension to the plugins in standalone (#7079)
shuiyisong Oct 14, 2025
49aeeb8
refactor: remove unused grpc-expr module and pb conversions (#7085)
sunng87 Oct 14, 2025
4cafd40
ci: update dev-builder image tag (#7073)
github-actions[bot] Oct 14, 2025
c097dab
feat: conversion between struct, value and json (#7052)
sunng87 Oct 14, 2025
ade17f8
feat(parser): ALTER TABLE ... REPARTITION ... (#7082)
waynexia Oct 15, 2025
3a4219b
feat: add updated_on to tablemeta with a default of created_on (#7072)
Standing-Man Oct 15, 2025
a31309c
feat: add Value::Json value type (#7083)
sunng87 Oct 15, 2025
809b556
fix: part cols not in projection (#7090)
discord9 Oct 16, 2025
a8717ef
feat: manual compaction parallelism (#7086)
v0y4g3r Oct 16, 2025
1ee5d5f
feat: memtable seq range read (#6950)
discord9 Oct 16, 2025
abe4a12
refactor: add `peer_hostname` field in `information_schema.cluster_in…
zyy17 Oct 16, 2025
d84888c
fix: fix build warnings (#7099)
WenyXu Oct 16, 2025
000f9b6
chore: pub route_prometheus function (#7101)
v0y4g3r Oct 16, 2025
2009222
refactor: update valueref coerce function name based on its semantics…
sunng87 Oct 16, 2025
719de47
refactor!: unify the API of getting total cpu and memory (#7049)
zyy17 Oct 16, 2025
b4ef018
feat: supports large string (#7097)
killme2008 Oct 17, 2025
333b427
feat: add index cache eviction support (#7064)
zhongzc Oct 17, 2025
f01f9dc
feat(trigger): support "for" and "keep_firing_for" (#7087)
fengys1996 Oct 17, 2025
878c564
feat: new create table syntax for new json datatype (#7103)
MichaelScofield Oct 17, 2025
612a57d
feat(mito2): expose puffin index metadata (#7042)
zhongzc Oct 17, 2025
76308d9
chore: remove unused deps (#7108)
fengys1996 Oct 17, 2025
24df41d
chore: update datafusion to 50
MichaelScofield Oct 11, 2025
3381859
fix: update datafusion_pg_catalog import
sunng87 Oct 13, 2025
0355923
feat: implement a udf to convert json into arrow struct
sunng87 Oct 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
826 changes: 394 additions & 432 deletions Cargo.lock
View file
Open in desktop

Large diffs are not rendered by default.

69 changes: 35 additions & 34 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,12 +99,12 @@ rust.unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tokio_unstable)'] }
# See for more detaiils: https://github.com/rust-lang/cargo/issues/11329
ahash = { version = "0.8", features = ["compile-time-rng"] }
aquamarine = "0.6"
arrow = { version = "56.0", features = ["prettyprint"] }
arrow-array = { version = "56.0", default-features = false, features = ["chrono-tz"] }
arrow-buffer = "56.0"
arrow-flight = "56.0"
arrow-ipc = { version = "56.0", default-features = false, features = ["lz4", "zstd"] }
arrow-schema = { version = "56.0", features = ["serde"] }
arrow = { version = "56.2", features = ["prettyprint"] }
arrow-array = { version = "56.2", default-features = false, features = ["chrono-tz"] }
arrow-buffer = "56.2"
arrow-flight = "56.2"
arrow-ipc = { version = "56.2", default-features = false, features = ["lz4", "zstd"] }
arrow-schema = { version = "56.2", features = ["serde"] }
async-stream = "0.3"
async-trait = "0.1"
# Remember to update axum-extra, axum-macros when updating axum
Expand All @@ -123,18 +123,18 @@ clap = { version = "4.4", features = ["derive"] }
config = "0.13.0"
crossbeam-utils = "0.8"
dashmap = "6.1"
datafusion = "49"
datafusion-common = "49"
datafusion-expr = "49"
datafusion-functions = "49"
datafusion-functions-aggregate-common = "49"
datafusion-optimizer = "49"
datafusion-orc = { git = "https://github.com/GreptimeTeam/datafusion-orc", rev = "a0a5f902158f153119316eaeec868cff3fc8a99d" }
datafusion-pg-catalog = { git = "https://github.com/datafusion-contrib/datafusion-postgres", rev = "3d1b7c7d5b82dd49bafc2803259365e633f654fa" }
datafusion-physical-expr = "49"
datafusion-physical-plan = "49"
datafusion-sql = "49"
datafusion-substrait = "49"
datafusion = "50"
datafusion-common = "50"
datafusion-expr = "50"
datafusion-functions = "50"
datafusion-functions-aggregate-common = "50"
datafusion-optimizer = "50"
datafusion-orc = { git = "https://github.com/datafusion-contrib/datafusion-orc", tag = "v0.5.0" }
datafusion-pg-catalog = "0.11"
datafusion-physical-expr = "50"
datafusion-physical-plan = "50"
datafusion-sql = "50"
datafusion-substrait = "50"
deadpool = "0.12"
deadpool-postgres = "0.14"
derive_builder = "0.20"
Expand All @@ -147,7 +147,7 @@ etcd-client = { git = "https://github.com/GreptimeTeam/etcd-client", rev = "f62d
fst = "0.4.7"
futures = "0.3"
futures-util = "0.3"
greptime-proto = { git = "https://github.com/GreptimeTeam/greptime-proto.git", rev = "d75496d5d09dedcd0edcade57ccf0a522f4393ae" }
greptime-proto = { git = "https://github.com/GreptimeTeam/greptime-proto.git", rev = "69a6089933daa573c96808ec4bbc48f447ec6e8c" }
hex = "0.4"
http = "1"
humantime = "2.1"
Expand Down Expand Up @@ -180,7 +180,7 @@ otel-arrow-rust = { git = "https://github.com/GreptimeTeam/otel-arrow", rev = "2
"server",
] }
parking_lot = "0.12"
parquet = { version = "56.0", default-features = false, features = ["arrow", "async", "object_store"] }
parquet = { version = "56.2", default-features = false, features = ["arrow", "async", "object_store"] }
paste = "1.0"
pin-project = "1.0"
pretty_assertions = "1.4.0"
Expand All @@ -207,6 +207,7 @@ rstest_reuse = "0.7"
rust_decimal = "1.33"
rustc-hash = "2.0"
# It is worth noting that we should try to avoid using aws-lc-rs until it can be compiled on various platforms.
hostname = "0.4.0"
rustls = { version = "0.23.25", default-features = false }
sea-query = "0.32"
serde = { version = "1.0", features = ["derive"] }
Expand All @@ -216,10 +217,7 @@ simd-json = "0.15"
similar-asserts = "1.6.0"
smallvec = { version = "1", features = ["serde"] }
snafu = "0.8"
sqlparser = { git = "https://github.com/GreptimeTeam/sqlparser-rs.git", rev = "39e4fc94c3c741981f77e9d63b5ce8c02e0a27ea", features = [
"visitor",
"serde",
] } # branch = "v0.55.x"
sqlparser = { version = "0.58.0", default-features = false, features = ["std", "visitor", "serde"] }
sqlx = { version = "0.8", features = [
"runtime-tokio-rustls",
"mysql",
Expand Down Expand Up @@ -321,16 +319,19 @@ git = "https://github.com/GreptimeTeam/greptime-meter.git"
rev = "5618e779cf2bb4755b499c630fba4c35e91898cb"

[patch.crates-io]
datafusion = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "7d5214512740b4dfb742b6b3d91ed9affcc2c9d0" }
datafusion-common = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "7d5214512740b4dfb742b6b3d91ed9affcc2c9d0" }
datafusion-expr = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "7d5214512740b4dfb742b6b3d91ed9affcc2c9d0" }
datafusion-functions = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "7d5214512740b4dfb742b6b3d91ed9affcc2c9d0" }
datafusion-functions-aggregate-common = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "7d5214512740b4dfb742b6b3d91ed9affcc2c9d0" }
datafusion-optimizer = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "7d5214512740b4dfb742b6b3d91ed9affcc2c9d0" }
datafusion-physical-expr = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "7d5214512740b4dfb742b6b3d91ed9affcc2c9d0" }
datafusion-physical-plan = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "7d5214512740b4dfb742b6b3d91ed9affcc2c9d0" }
datafusion-sql = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "7d5214512740b4dfb742b6b3d91ed9affcc2c9d0" }
datafusion-substrait = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "7d5214512740b4dfb742b6b3d91ed9affcc2c9d0" }
datafusion = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-common = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-expr = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-functions = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-functions-aggregate-common = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-optimizer = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-physical-expr = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-physical-expr-common = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-physical-plan = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-datasource = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-sql = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
datafusion-substrait = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "78041eabebf014b8661f50d5ce8b3ae2f70105f7" }
sqlparser = { git = "https://github.com/GreptimeTeam/sqlparser-rs.git", rev = "4b519a5caa95472cc3988f5556813a583dd35af1" } # branch = "v0.58.x"

[profile.release]
debug = 1
Expand Down
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ CARGO_BUILD_OPTS := --locked
IMAGE_REGISTRY ?= docker.io
IMAGE_NAMESPACE ?= greptime
IMAGE_TAG ?= latest
DEV_BUILDER_IMAGE_TAG ?= 2025-05-19-f55023f3-20250829091211
DEV_BUILDER_IMAGE_TAG ?= 2025-10-01-8fe17d43-20251011080129
BUILDX_MULTI_PLATFORM_BUILD ?= false
BUILDX_BUILDER_NAME ?= gtbuilder
BASE_IMAGE ?= ubuntu
Expand Down
Loading
Loading