Skip to content

Commit e96be80

Browse files
normanrzdstansby
andauthored
Apply suggestions from code review
Co-authored-by: David Stansby <[email protected]>
1 parent 827cff0 commit e96be80

File tree

2 files changed

+5
-5
lines changed

2 files changed

+5
-5
lines changed

docs/user-guide/arrays.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -580,8 +580,8 @@ With Zarr format 3, a new sharding feature has been added to address this issue.
580580

581581
With sharding, multiple chunks can be stored in a single storage object (e.g. a file).
582582
Within a shard, chunks are compressed and serialized separately.
583-
This allows to read individual chunks independently.
584-
However, when writing data, a full shard must be written for optimal performance and to
583+
This allows individual chunks to be read independently.
584+
However, when writing data, a full shard must be written in one go for optimal performance and to
585585
avoid concurrency issues.
586586
That means that shards are the units of writing and chunks are the units of reading.
587587
Users need to configure the chunk and shard shapes accordingly.
@@ -607,7 +607,7 @@ Sharded arrays can be created by providing the ``shards`` parameter to :func:`za
607607
Chunks Initialized : 100
608608

609609
In this example a shard shape of (1000, 1000) and a chunk shape of (100, 100) is used.
610-
This means that 10*10 chunks are stored in each shard.
610+
This means that 10*10 chunks are stored in each shard, and there are 10*10 shards in total.
611611

612612
Missing features in 3.0
613613
-----------------------

docs/user-guide/performance.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,14 +69,14 @@ Sharding
6969
If you have large arrays but need small chunks to efficiently access the data, you can
7070
use sharding. Sharding provides a mechanism to store multiple chunks in a single
7171
storage object or file. This can be useful because traditional file systems and object
72-
storage systems may have issues with many small files.
72+
storage systems may have issues storing and accessing many files.
7373

7474
Picking a good combination of chunk shape and shard shape is important for performance.
7575
The chunk shape determines what unit of your data can be read independently, while the
7676
shard shape determines what unit of your data can be written efficiently.
7777

7878
For an example, consider you have a 100 GB array and need to read small chunks of 1 MB.
79-
Without sharding, each chunk would be one file resulting in 10000 files. That can
79+
Without sharding, each chunk would be one file resulting in 100,000 files. That can
8080
already cause performance issues on some file systems.
8181
With sharding, you could use a shard size of 1 GB. This would result in 1000 chunks per
8282
file and 100 files in total, which seems manageable for most storage systems.

0 commit comments

Comments
 (0)