Skip to content

Conversation

@XanthosXanthopoulos
Copy link
Collaborator

Issue and/or context:
#4311 aims to provide a zero copy write path but at the python side when setting individual columns with data the user can pass temporary objects which forces to always copy the data on that code path to avoid segfaults

Changes:
This PR stores the references of the object used to set individual columns until the query is submitted and disable memory copy from that code path.

Notes for Reviewer:

@codecov
Copy link

codecov bot commented Dec 12, 2025

Codecov Report

❌ Patch coverage is 80.00000% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.15%. Comparing base (b8a91dd) to head (e87512a).
⚠️ Report is 1 commits behind head on xan/mem_modes.

Additional details and impacted files
@@                Coverage Diff                @@
##           xan/mem_modes    #4350      +/-   ##
=================================================
- Coverage          89.16%   89.15%   -0.02%     
=================================================
  Files                 62       62              
  Lines               7118     7128      +10     
=================================================
+ Hits                6347     6355       +8     
- Misses               771      773       +2     
Flag Coverage Δ
python 89.15% <80.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
python_api 89.15% <80.00%> (-0.02%) ⬇️
libtiledbsoma ∅ <ø> (∅)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@XanthosXanthopoulos XanthosXanthopoulos merged commit 5a81aba into xan/mem_modes Dec 12, 2025
18 checks passed
@XanthosXanthopoulos XanthosXanthopoulos deleted the xan/preserve_temp_objects branch December 12, 2025 18:15
XanthosXanthopoulos added a commit that referenced this pull request Dec 19, 2025
…rrow objects (#4334)

* Add custom allocator for vector backed buffers, implement memory modes for TileDB to Arrow conversions

* Lint fix

* Migrate R to use multithreaded arrow conversion

* Fix compiler warnings

* Rebind buffers after each read operation

* Read memory mode from config

* Include Rcpp header before other headers

* Store object references when setting column data until write is submitted (#4350)
XanthosXanthopoulos added a commit that referenced this pull request Jan 14, 2026
…rrow objects (#4334)

* Add custom allocator for vector backed buffers, implement memory modes for TileDB to Arrow conversions

* Lint fix

* Migrate R to use multithreaded arrow conversion

* Fix compiler warnings

* Rebind buffers after each read operation

* Read memory mode from config

* Include Rcpp header before other headers

* Store object references when setting column data until write is submitted (#4350)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants