0.57.0-rc.0

Changes

⚠️ Breaks

• Do not inline vortex_array into vortex:: namespace (#5584) @gatesn
• Move stats into expr (#5582) @gatesn
• Combine Array VTable with Encoding (#5462) @gatesn
• Consolidate into stat_expression function (#5366) @gatesn
• chore[vortex-array]: remove SerdeVTable (#5387) @joseph-isaacs
• feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs

✨ Features

• feat[scalar_fn]: binary ops (#5617) @joseph-isaacs
• Execute ScalarFn arrays (#5611) @gatesn
• FilterArray (#5610) @gatesn
• Add GetItemFn (#5609) @gatesn
• Add MaskFn (#5606) @gatesn
• feat[scalar_fn]: is_null (#5601) @joseph-isaacs
• Scalar functions (#5561) @gatesn
• feat[buffer]: use BufferHandle in more places (#5567) @joseph-isaacs
• feat: alp decompress to vector (#5548) @0ax1
• feat[gpu]: use BufferHandle to the array vtable build (#5565) @joseph-isaacs
• feat[vortex-expr]: is_fallible analysis (#5546) @joseph-isaacs
• Feature: add take compute for slices (#5537) @connortsui20
• feat[vortex-expr]: is_fallible expression analysis (#5531) @joseph-isaacs
• feat[vortex-expr]: add a is_null_sensitive expression analysis (#5528) @joseph-isaacs
• Feature: add a take compute trait (#5503) @connortsui20
• Feature: add impl owned filter impls (#5493) @connortsui20
• Feature: add filter impl for FixedSizeListVector (#5494) @connortsui20
• Vector cast (#5461) @gatesn
• Scalar arithmetic (#5460) @gatesn
• feat[cli]: add scroll for tree encoding view (#5497) @a10y
• Revert "Revert "chore[ci]: add windows test to CI (#5439)"" (#5486) @joseph-isaacs
• More vector scalars (#5459) @gatesn
• feature: Support std_file on windows as it's used by cxx bindings (#5450) @robert3005
• Feature: implement bitpacking batch execute (#5446) @connortsui20
• feat[vortex-array]: added debug to expr/array reduce rules (#5442) @joseph-isaacs
• feat: publish vortex-spark-all assembly JAR (#5435) @a10y
• feat[vortex-array] add an array session rule registry (#5414) @joseph-isaacs
• Feature: add spare_capacity_mut to PVectorMut (#5287) @connortsui20
• feat: pipeline-pcodec (#5415) @0ax1
• Add VectorOps::clear (#5409) @gatesn
• feat[vortex-array]: expr array that represents lazy computation (#5400) @joseph-isaacs
• feat[vortex-expr]: reduce_parent has typed parent (#5398) @joseph-isaacs
• feat: cast expression (#5389) @danking
• feat: PyExpr::evaluate (#5388) @danking
• feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs
• Faster Pipelines (#5352) @gatesn
• feat[vortex-expr]: better debug print (#5374) @joseph-isaacs

🚀 Performance

• perf[ci]: speed up ci MORE (#5576) @joseph-isaacs
• perf(ci): skip rust-cache on runs-on runners with S3 sccache (#5570) @joseph-isaacs
• chunked array zip (#5530) @onursatici
• varbinview zip kernel (#5526) @onursatici
• chore[vortex-dict]: numeric pushdown & clean up the dict ctor (#5438) @joseph-isaacs
• Handwritten pipeline benchmark and analysis (#5408) @connortsui20
• Fix: incorrect offset for nulls + don't chunk elements (#5423) @connortsui20
• perf[dict]: all_values_referenced to allow pushdown optimisation. (#5416) @joseph-isaacs
• perf[vortex-expr]: replace can stop looking at subtree if it finds a match (#5405) @joseph-isaacs
• actually make ZCTL (#5402) @a10y
• fix: performance regression for filtering ListView (#5390) @a10y
• fix[vortex-dict]: faster min_max for dict array (#5288) @joseph-isaacs
• fix: yo dawg, I heard you like inlining (#5363) @a10y

🐛 Bug Fixes

35 changes

• fix: avoid ListBuilder (#5636) @a10y
• fix: disable flaky bench for codspeed (#5580) @0ax1
• fix: don't bench slice_arrow_buffer on codspeed (#5578) @0ax1
• fix: fixed seed for micro-benchmarks (#5574) @0ax1
• fix: don't clone in micro-benchmarks (#5573) @0ax1
• fix: Decimal sum doesn't panic but returns null on overflow (#5564) @robert3005
• fix[dict]: min_max must not be null (#5560) @joseph-isaacs
• fix[display]: escape strings (#5551) @joseph-isaacs
• fix: benchmark black box handling (#5558) @0ax1
• fix: make micro-benchmark labels stable (#5557) @0ax1
• Fix: Schema projection order mismatch from table/file schema (#5541) @brancz
• fix: remove print (#5547) @joseph-isaacs
• fix: assert_arrays_eq macro reexports format function to hide itertools dep (#5543) @robert3005
• fix[mask]: intersect_by_rank and bit to arrow (#5545) @joseph-isaacs
• fix: Restore fast constant sum behaviour, don't fuzz floating point sum compute function (#5527) @robert3005
• Fix: ConstantArray float sum behaves the same way as primitive array (#5513) @robert3005
• Fix: this may have been a merge conflict (#5510) @connortsui20
• fix[dict]: not (filter/take) set_all_values_referenced (#5496) @joseph-isaacs
• fix: Sum from stats has to account for accumulator (#5445) @robert3005
• fix: benchmark input handling (#5488) @0ax1
• fix: handling of struct fields in Spark (#5453) @a10y
• Revert "chore[ci]: add windows test to CI (#5439)" (#5449) @0ax1
• Fix: remove unreachable cases in bitpack patches (#5444) @connortsui20
• fix: FFI returns a string/binary instead of requiring the caller to guess the necessary memory (#5433) @robert3005
• Fix: Correctly handle inconsistent file and table schema order in vortex-datafusion (#5436) @brancz
• Fix: incorrect offset for nulls + don't chunk elements (#5423) @connortsui20
• fix[ci]: use run_id in the s3 benchmark runs (#5421) @joseph-isaacs
• fix: add missing dependency: typing-extensions (#5420) @danking
• fix: vx_struct_fields_field_dtype takes usize instead of u64 (#5396) @robert3005
• fix: BitView slice filter (#5381) @a10y
• fix: cast must cast its stats too (#5386) @danking
• fix: VarBin overflow on take (#5361) @a10y
• fix[vortex-array]: correct convert to arrow from vortex (#5378) @joseph-isaacs
• Use the specified concurrency value when reading from a ObjectStoreIoSource (#5371) @nrc
• fix: yo dawg, I heard you like inlining (#5363) @a10y

📖 Documentation

• docs: fix typo in readme (#5627) @0ax1
• Chore: allow searching subcrates in vortex rustdoc (#5612) @connortsui20
• Describe upcoming API breaks in docs (#5583) @gatesn
• TSC Agenda 2025-11-25 (#5523) @gatesn
• Docs: document chunk_offsets in Patches (#5429) @connortsui20

🧰 Maintenance

87 changes

• Add advisory semver checks to CI (#5645) @gatesn
• fix(deps): update protobuf monorepo to v4.33.2 (#5644) @renovate[bot]
• chore(deps): update all patch updates (#5643) @renovate[bot]
• fix platform/arch for runs-on config (#5622) @a10y
• ci: add runs-on.yml that points to prebuilt AMI (#5620) @a10y
• ci: add cache-suffix to isolate sanitizer build cache (#5604) @joseph-isaacs
• feat[fuzz]: add WASM fuzzer support for wasmfuzz (#5575) @joseph-isaacs
• chore(deps): update crate-ci/typos action to v1.40.0 (#5596) @renovate[bot]
• chore(deps): update tokio-tracing monorepo (#5595) @renovate[bot]
• chore(deps): update all patch updates (#5593) @renovate[bot]
• chore: inline alp decompress_from_buffer fn (#5587) @0ax1
• Remove Operator VTable (#5588) @gatesn
• Test implementation of cast using ScalarFn (#5586) @gatesn
• Remove ExprArray and ExprOptimizer (#5585) @gatesn
• Move stats into expr (#5582) @gatesn
• chore[ci]: bump codspeed runner (#5579) @joseph-isaacs
• chore: turn byte factory macro into function (#5577) @0ax1
• ci: simplify codspeed sharding (#5572) @0ax1
• chore: speed up taplo format (#5569) @robert3005
• chore: Remove codspeed warning about deprecated mode (#5562) @robert3005
• Chore: use item imports in bench-vortex (#5554) @connortsui20
• Remove clone from array vtable (#5552) @gatesn
• Chore: check taplo formatting in pre-commit (#5549) @connortsui20
• chore[*]: prefer #[expect(..)] over #[allow(..)] p2 (#5533) @joseph-isaacs
• chore[*]: prefer #[expect(..)] over #[allow(..)] (#5498) @joseph-isaacs
• chore: benchmark hygiene (#5499) @0ax1
• chore[ffi]: remove header gen warning not nightly (#5529) @joseph-isaacs
• varbinview zip bench (#5518) @onursatici
• chore: apply imports_granularity = "Item" everywhere expect bench-vortex (#5521) @joseph-isaacs
• chore: fix error message for unwrap_scalar (#5515) @xumingming
• Chore: remove unstable pipeline benchmark (#5512) @connortsui20
• Expression execution to be passed input vectors (#5507) @gatesn
• Chore: Use full item paths for every import is vortex-vector (#5509) @connortsui20
• Chore: Fully qualify imported items (#5504) @connortsui20
• Chore: Even more stable benchmarks (#5506) @connortsui20
• Chore: make pipeline benchmarks less susceptible to noise (#5502) @connortsui20
• Chore: refactor dict module (#5501) @connortsui20
• chore: minimize with_inputs work for benchmarks (#5492) @0ax1
• Preapproved issue template (#5489) @gatesn
• chore[dbp]: remove useless_conversion (#5487) @joseph-isaacs
• Update Rust crate pyo3 to 0.27.0 (#5478) @renovate[bot]
• Update Rust crate goldenfile to v1.9.1 (#5474) @renovate[bot]
• Update Rust crate tokio to v1.48.0 (#5480) @renovate[bot]
• Update Rust crate insta to v1.44.1 (#5475) @renovate[bot]
• Update all patch updates (#5470) @renovate[bot]
• Update Rust crate noodles-bgzf to 0.44.0 (#5476) @renovate[bot]
• Update protobuf monorepo to v4.33.1 (#5472) @renovate[bot]
• Update Rust crate cudarc to 0.18.0 (#5473) @renovate[bot]
• Update plugin com.diffplug.spotless to v8.1.0 (#5471) @renovate[bot]
• Update Rust crate lance to 0.39.0 (#5466) @renovate[bot]
• Update actions/checkout action to v6 (#5468) @renovate[bot]
• Update actions/download-artifact action to v6 (#5469) @renovate[bot]
• Update CodSpeedHQ/action digest to 346a2d8 (#5463) @renovate[bot]
• Update GitHub Artifact Actions to v5 (major) (#5464) @renovate[bot]
• Update Rust crate dyn-hash to v1 (#5465) @renovate[bot]
• Introduces a new set of issue/discussion templates (#5456) @gatesn
• Chore: make apply patches its own method (#5458) @connortsui20
• Chore: refactor rle module (#5455) @connortsui20
• Chore: refactor delta module (#5452) @connortsui20
• Chore: refactor for module (#5451) @connortsui20
• chore[ci]: add windows test to CI (#5439) @joseph-isaacs
• chore: Shard codspeed into 8 runs (#5440) @robert3005
• feat[vortex-array] chunked array has owned chunk_offsets child (#5426) @joseph-isaacs
• Chore: add set functions to mask mut (#5432) @connortsui20
• Consolidate into stat_expression function (#5366) @gatesn
• Chore: rename apply patches (#5430) @connortsui20
• Chore: rename unpack functions (#5428) @connortsui20
• Chore: Use MaybeUninit instead of UninitRange for bit unpacking (#5427) @connortsui20
• Remove selection mask from batch vtable (#5424) @gatesn
• Pipeline use immutable vectors (#5418) @gatesn
• chore: less emojis in top-level readme (#5422) @0ax1
• chore[ci]: unify the benchmark table analysis thresholds (#5413) @joseph-isaacs
• chore[vortex-array]: remove unused experimental compute arrays (#5404) @joseph-isaacs
• chore[vortex-array]: clean up doc str (#5403) @joseph-isaacs
• chore[vortex-expr]: allow Arced ref to the rewrite rule (#5401) @joseph-isaacs
• chore[vortex-array]: remove ser/de type constraint from Metadata (#5397) @joseph-isaacs
• chore[vortex-array]: rename reduce_children -> reduce (#5375) @joseph-isaacs
• chore[vortex-array]: remove SerdeVTable (#5387) @joseph-isaacs
• chore: better expression error messages on failure (#5392) @danking
• chore: suggest using a cast for float-float and int-int comparisons (#5393) @danking
• Revert "chore[vortex-gpu]: remove vortex-gpu" (#5384) @joseph-isaacs
• chore(deps): update codspeedhq/action digest to 6a8e2b8 (#5138) @renovate[bot]
• chore[vortex-gpu]: remove vortex-gpu (#5376) @joseph-isaacs
• chore: Fix compilation of benchmarks if gpu_unstable is not enabled (#5370) @robert3005
• chore: gpu_bitunpack bench is behind cfg flag (#5369) @robert3005
• Mooooaarrrrr generics (#5367) @gatesn
• chore: documentation is a valid PR tag (#5350) @danking