Skip to content

Commit 870b4af

Browse files
committed
feature: optionally support uvloop
1 parent 060da4a commit 870b4af

File tree

4 files changed

+79
-3
lines changed

4 files changed

+79
-3
lines changed

changes/xxxx.feature.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Add optional uvloop support for improved async performance.
2+
3+
When uvloop is available, Zarr will automatically use it as the event loop implementation
4+
for better I/O performance. This can be controlled via the ``async.use_uvloop`` configuration
5+
setting or the ``ZARR_ASYNC__USE_UVLOOP`` environment variable. uvloop can be installed
6+
with ``pip install 'zarr[uvloop]'``.

docs/user-guide/performance.rst

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -270,6 +270,76 @@ E.g., pickle/unpickle an local store array::
270270
>>> np.all(z1[:] == z2[:])
271271
np.True_
272272

273+
.. _user-guide-uvloop:
274+
275+
Event loop optimization with uvloop
276+
-----------------------------------
277+
278+
Zarr can optionally use `uvloop <https://github.com/MagicStack/uvloop>`_, a fast,
279+
drop-in replacement for the default Python asyncio event loop implementation.
280+
uvloop is written in Cython and built on top of libuv, providing significantly
281+
better performance for I/O-intensive operations.
282+
283+
When uvloop is available, Zarr will use it by default for better performance.
284+
This is particularly beneficial when working with remote storage backends or
285+
performing many concurrent operations.
286+
287+
Installation
288+
~~~~~~~~~~~~
289+
290+
To enable uvloop support, install it as an optional dependency::
291+
292+
pip install 'zarr[uvloop]'
293+
294+
Or install uvloop directly::
295+
296+
pip install uvloop
297+
298+
Configuration
299+
~~~~~~~~~~~~~
300+
301+
uvloop usage can be controlled via Zarr's configuration system:
302+
303+
.. code-block:: python
304+
305+
import zarr
306+
307+
# Enable uvloop (default when available)
308+
zarr.config.set({"async.use_uvloop": True})
309+
310+
# Disable uvloop (use standard asyncio)
311+
zarr.config.set({"async.use_uvloop": False})
312+
313+
You can also control this via environment variables::
314+
315+
# Disable uvloop
316+
export ZARR_ASYNC__USE_UVLOOP=false
317+
318+
Platform Support
319+
~~~~~~~~~~~~~~~~~
320+
321+
uvloop is supported on:
322+
323+
- Linux
324+
- macOS
325+
- Other Unix-like systems
326+
327+
uvloop is **not** supported on Windows. On Windows, Zarr will automatically
328+
fall back to the standard asyncio event loop regardless of the configuration setting.
329+
330+
Performance Benefits
331+
~~~~~~~~~~~~~~~~~~~~
332+
333+
uvloop can provide performance improvements for:
334+
335+
- Remote storage operations (S3, GCS, etc.)
336+
- Concurrent array operations
337+
- Large numbers of small I/O operations
338+
- Network-bound workloads
339+
340+
The performance improvement varies depending on the workload, but can be
341+
substantial for I/O-intensive operations.
342+
273343
.. _user-guide-tips-blosc:
274344

275345
Configuring Blosc

pyproject.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -91,8 +91,7 @@ remote_tests = [
9191
"moto[s3,server]",
9292
"requests",
9393
]
94-
optional = ["rich", "universal-pathlib"]
95-
uvloop = ["uvloop"]
94+
optional = ["rich", "universal-pathlib", "uvloop"]
9695
docs = [
9796
# Doc building
9897
'sphinx==8.1.3',
@@ -230,6 +229,7 @@ dependencies = [
230229
'typing_extensions==4.9.*',
231230
'donfig==0.8.*',
232231
'obstore==0.5.*',
232+
'uvloop==0.20.0',
233233
# test deps
234234
'zarr[test]',
235235
'zarr[remote_tests]',

tests/test_config.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ def test_config_defaults_set() -> None:
5555
"order": "C",
5656
"write_empty_chunks": False,
5757
},
58-
"async": {"concurrency": 10, "timeout": None},
58+
"async": {"concurrency": 10, "timeout": None, "use_uvloop": True},
5959
"threading": {"max_workers": None},
6060
"json_indent": 2,
6161
"codec_pipeline": {

0 commit comments

Comments
 (0)