Skip to content

Update kerchunk writer refs_to_dataframe to deal with ArrowStringArray from new ZarrParser deps.#907

Merged
TomNicholas merged 2 commits intomainfrom
kerchunk_parquet_writer_pyarrow_fx
Mar 7, 2026
Merged

Update kerchunk writer refs_to_dataframe to deal with ArrowStringArray from new ZarrParser deps.#907
TomNicholas merged 2 commits intomainfrom
kerchunk_parquet_writer_pyarrow_fx

Conversation

@norlandrhagen
Copy link
Collaborator

What I did

#892 added two new deps, arro3-core and pyarrow These broke the Kerchunk parquet tests:

ValueError: Error converting column "path" to bytes using encoding UTF8. Original error: Unable to avoid copy while creating an array as requested.

Adding a pandas context seemed to resolve this. pd.option_context("future.infer_string", False)

Acceptance criteria:

  • Tests passing
  • Full type hint coverage

@norlandrhagen norlandrhagen added Kerchunk Relating to the kerchunk library / specification itself dependencies Updates a dependency labels Mar 6, 2026
@norlandrhagen norlandrhagen marked this pull request as ready for review March 6, 2026 20:56
@codecov
Copy link

codecov bot commented Mar 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.34%. Comparing base (3287d82) to head (98a612c).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #907      +/-   ##
==========================================
- Coverage   89.38%   89.34%   -0.04%     
==========================================
  Files          34       34              
  Lines        1997     1999       +2     
==========================================
+ Hits         1785     1786       +1     
- Misses        212      213       +1     
Files with missing lines Coverage Δ
virtualizarr/accessor.py 95.69% <100.00%> (+0.09%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Member

@TomNicholas TomNicholas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch!!

@TomNicholas TomNicholas merged commit 8a7a73b into main Mar 7, 2026
15 checks passed
@TomNicholas TomNicholas deleted the kerchunk_parquet_writer_pyarrow_fx branch March 7, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Updates a dependency Kerchunk Relating to the kerchunk library / specification itself

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants