Add Arrow C streaming, DataFrame iteration, and OOM-safe streaming execution#1222

Open

kosiew wants to merge 64 commits intoapache:mainfrom

kosiew:oom-1206

Commits on Sep 2, 2025

feat: add streaming utilities, range support, and improve async handling in DataFrame
kosiew
committed
refactor: improve DataFrame streaming, memory management, and error handling
kosiew
committed
feat: enhance DataFrame streaming and improve robustness, tests, and docs
kosiew
committed
feat: add testing utilities for DataFrame range generation
kosiew
committed
feat: ensure proper resource management in DataFrame streaming
kosiew
committed
refactor: replace spawn_stream and spawn_streams with spawn_future for consistency
kosiew
committed
feat: add test for Arrow C stream schema selection in DataFrame
kosiew
committed
test: rename and extend test_arrow_c_stream_to_table to include RecordBatchReader validation
kosiew
committed
test: add validation for schema mismatch in Arrow C stream
kosiew
committed
fix Ruff errors
kosiew
committed

Commits on Sep 7, 2025

Update docs/source/user-guide/dataframe/index.rst

kosiew
and
kylebarron
authored

Commits on Sep 10, 2025

Commits on Sep 13, 2025

Commits on Sep 15, 2025

Commits on Sep 20, 2025