Fix backpressure

[aseyboldt]

fixes pymc-devs/nutpie#262

[codecov-commenter]

Codecov Report

❌ Patch coverage is 0% with 80 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.97%. Comparing base (394c629) to head (6c44a27).
⚠️ Report is 95 commits behind head on main.

Files with missing lines	Patch %	Lines
src/storage/zarr/async_impl.rs	0.00%	80 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (394c629) and HEAD (6c44a27). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (394c629)	HEAD (6c44a27)
	2	1

Additional details and impacted files

@@             Coverage Diff             @@
##             main      #46       +/-   ##
===========================================
- Coverage   83.77%   55.97%   -27.81%     
===========================================
  Files           8       29       +21     
  Lines        1923     7048     +5125     
===========================================
+ Hits         1611     3945     +2334     
- Misses        312     3103     +2791     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This pull request fixes a backpressure issue in the async Zarr storage implementation by introducing a bounded queue for write operations. The main changes include replacing the unbounded Vec<JoinHandle> with a bounded JoinSet protected by a mutex, and implementing a backpressure mechanism that blocks when too many writes are queued.

Key changes:

• Introduced a bounded write queue using Arc<Mutex<JoinSet>> with configurable limit
• Implemented queue_write function that enforces backpressure by waiting for tasks when limit is reached
• Updated finalize and flush methods to properly join all pending writes
• Bumped pulp dependency from 0.21.4 to 0.22.2

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File	Description
src/storage/zarr/async_impl.rs	Implements backpressure mechanism with bounded write queue, replaces Vec of JoinHandles with Arc<Mutex>, adds queue_write helper function
Cargo.toml	Updates pulp dependency to version 0.22.2

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.