Skip to content

Added disk spilling for merge#4218

Closed
thomasfrederikhoeck wants to merge 3 commits intodelta-io:mainfrom
thomasfrederikhoeck:feat_spill_merge
Closed

Added disk spilling for merge#4218
thomasfrederikhoeck wants to merge 3 commits intodelta-io:mainfrom
thomasfrederikhoeck:feat_spill_merge

Conversation

@thomasfrederikhoeck
Copy link
Contributor

Added disk spilling for merge similar to optimize functions to allow for merges which touches many files in the target

Description

I have added functionality for spilling to disk similar to how it works in the optimize functions. If nothing is provided it works as before.

I have added test similar to those for the other spill functions.

I have tested my cases in #4217 which now successfully completes the merge without OOM.

I have used AI (Opus 4.6) for getting a overview of the project structure and for writing most of the code. I have review and verified the code myself.

Related Issue(s)

Closes #4217

Documentation

Added disk spilling for merge similar to optimize functions
@github-actions github-actions bot added binding/python Issues for the Python package binding/rust Issues for the Rust crate labels Feb 20, 2026
@github-actions
Copy link

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

Thomas Frederik Hoeck added 2 commits February 20, 2026 12:01
Added disk spilling for merge similar to optimize functions

Signed-off-by: Thomas Frederik Hoeck <tfh@norden.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

binding/python Issues for the Python package binding/rust Issues for the Rust crate

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Feature]: Write incrementally to table when merging

1 participant