Skip to content

Commit b24ab63

Browse files
authored
chore(deps): upgrade to DataFusion 52 (apache#1997)
## Which issue does this PR close? Validates and adopts DataFusion 52 ## What changes are included in this PR? - Upgrade DataFusion integration from 51.x to 52.x - Keep the DataFusion Python dependency dynamic within major version 52 - Update the Python FFI table provider bridge for DataFusion 52 API/ABI expectations: - session-aware `__datafusion_table_provider__(session)` integration - DF52 compatible FFI table provider construction with task context and logical codec handling - Update sqllogictest physical plan expectations for DataFusion 52 planner output changes - Refresh lockfiles impacted by the upgrade ### DataFusion FFI API change DataFusion 52 expanded table provider FFI construction to include task context and optional logical codec parameters. Updated the Rust/Python bridge accordingly allowing filter/logical expression serialization remains compatible across the FFI boundary. ## Are these changes tested? Yes --------- Signed-off-by: Ethan Urbanski <ethan@urbanskitech.com>
1 parent 4f5274a commit b24ab63

14 files changed

+1169
-1439
lines changed

Cargo.lock

Lines changed: 415 additions & 853 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -42,14 +42,14 @@ rust-version = "1.88"
4242
anyhow = "1.0.72"
4343
apache-avro = { version = "0.21", features = ["zstandard"] }
4444
array-init = "2"
45-
arrow-arith = "57.0"
46-
arrow-array = "57.0"
47-
arrow-buffer = "57.0"
48-
arrow-cast = "57.0"
49-
arrow-ord = "57.0"
50-
arrow-schema = "57.0"
51-
arrow-select = "57.0"
52-
arrow-string = "57.0"
45+
arrow-arith = "57.1"
46+
arrow-array = "57.1"
47+
arrow-buffer = "57.1"
48+
arrow-cast = "57.1"
49+
arrow-ord = "57.1"
50+
arrow-schema = "57.1"
51+
arrow-select = "57.1"
52+
arrow-string = "57.1"
5353
as-any = "0.3.2"
5454
async-trait = "0.1.89"
5555
aws-config = "1.8.7"
@@ -62,9 +62,9 @@ bytes = "1.11"
6262
chrono = "0.4.41"
6363
clap = { version = "4.5.48", features = ["derive", "cargo"] }
6464
dashmap = "6"
65-
datafusion = "51.0"
66-
datafusion-cli = "51.0"
67-
datafusion-sqllogictest = "51.0"
65+
datafusion = "52.1"
66+
datafusion-cli = "52.1"
67+
datafusion-sqllogictest = "52.1"
6868
derive_builder = "0.20"
6969
dirs = "6"
7070
enum-ordinalize = "4.3.0"
@@ -100,7 +100,7 @@ murmur3 = "0.5.2"
100100
once_cell = "1.20"
101101
opendal = "0.55.0"
102102
ordered-float = "4"
103-
parquet = "57.0"
103+
parquet = "57.1"
104104
pilota = "0.11.10"
105105
pretty_assertions = "1.4"
106106
rand = "0.8.5"
@@ -130,4 +130,4 @@ url = "2.5.7"
130130
uuid = { version = "1.18", features = ["v7"] }
131131
volo = "0.10.6"
132132
volo-thrift = "0.10.8"
133-
zstd = "0.13.3"
133+
zstd = "0.13.3"

0 commit comments

Comments
 (0)