Skip to content

Commit 494ac96

Browse files
aliddellaliddell
authored
Update documentation for acquire-zarr v0.5.1 (part 2 / 2) (#104)
* Emphasize acquire-zarr and deëmphasize acquire-python in README * Add a core concepts page * Add note about streaming-only design * Get started -> Getting started * Update Getting started with Python * Update deploy_docs.yml * Update getting started page * Point to Getting Started page for acquire-zarr installation
1 parent 3b7dae1 commit 494ac96

File tree

3 files changed

+224
-80
lines changed

3 files changed

+224
-80
lines changed

‎.github/workflows/deploy_docs.yml‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ jobs:
5353
with:
5454
path: acquire-zarr # place in a named directory
5555
repository: acquire-project/acquire-zarr
56+
submodules: true
5657

5758
- name: Install Doxygen
5859
run: sudo apt-get install -y doxygen
@@ -69,9 +70,7 @@ jobs:
6970
- name: Install acquire-zarr # For collecting the API reference
7071
run: |
7172
cd acquire-zarr
72-
python -m pip install pybind11
73-
cmake --preset=default -B build -DBUILD_PYTHON=ON .
74-
cmake --build build
73+
python -m pip install pybind11[global]
7574
python -m pip install .
7675
7776
- name: Build website

‎docs/get_started.md‎

Lines changed: 221 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,18 @@
1-
# Get Started
2-
The `acquire-zarr` library provides both Python and C interfaces.
1+
# Getting started
32

4-
## Get Started with Python
3+
You can use `acquire-zarr` to stream data in either Python or C/C++ programs.
4+
This document provides a quick overview of how to get started with both languages.
55

6-
### Install the Python library
6+
> [!TIP]
7+
> New to Zarr streaming? Check out our [Core concepts](core_concepts.md) page to understand the key terminology used throughout this guide.
78
8-
The `acquire-zarr` Python library is compatible with Python versions 3.9-3.13.
9+
## Getting started with Python
910

10-
To install the library on Windows, macOS, or Ubuntu, run the following command:
11+
### Installation
12+
13+
The `acquire-zarr` Python library is supported for Python versions 3.9-3.13.
14+
15+
To install the library on Windows, macOS, or Linux, run the following command, after ensuring that the `python` command points to the correct Python version you want to use:
1116

1217
```bash
1318
python -m pip install acquire-zarr
@@ -17,7 +22,7 @@ We recommend installing `acquire-zarr` in a fresh conda environment or virtualen
1722
For example, to install `acquire-zarr` in a conda environment named `acquire`:
1823

1924
```shell
20-
conda create -n acquire python #compatible with python 3.9-3.13
25+
conda create -n acquire python=3.13 # or python=3.12, python=3.11, etc.
2126
conda activate acquire
2227
python -m pip install acquire-zarr
2328
```
@@ -26,98 +31,231 @@ or with virtualenv:
2631

2732
```shell
2833
$ python -m venv venv
29-
$ . ./venv/bin/activate # or on Windows: .\venv\Scripts\Activate.bat or .\venv\Scripts\Activate.ps1
34+
$ . ./venv/bin/activate # or on Windows: .\venv\Scripts\Activate
3035
(venv) $ python -m pip install acquire-zarr
3136
```
3237

33-
Once you have `acquire-zarr` installed, simply call `import acquire_zarr` in your script, notebook, or module to start utilizing the package.
38+
Now in your scripts or notebooks, import acquire-zarr with:
3439

3540
```python
36-
import acquire_zarr
41+
import acquire_zarr as aqz # or simply `import acquire_zarr`
3742
```
3843

39-
#### Usage
44+
### Usage
4045

41-
The library provides two main classes. First, `ZarrStream`, representing an output stream to a Zarr dataset.
42-
Second, `ZarrStreamSettings` to configure a Zarr stream.
46+
The typical workflow for acquiring data in Python is to create a stream by configuring it with the desired settings, and then to append data to the stream.
4347

44-
A typical use case for a 4-dimensional acquisition in Python might look like this:
48+
The *stream* represents a Zarr dataset that can be written to.
49+
You can write data to one or more *arrays* in the stream, where each array corresponds to a separate subset of the data.
50+
For example, you might have a stream for a multi-camera acquisition, where each camera's data is stored in a separate array.
51+
52+
Here's how you might configure a stream for a 4-dimensional acquisition (time, channel, height, width) with a single array and append data to it:
4553

4654
```python
4755
import acquire_zarr as aqz
4856
import numpy as np
4957

5058
settings = aqz.StreamSettings(
5159
store_path="my_stream.zarr",
52-
data_type=aqz.DataType.UINT16,
53-
version=aqz.ZarrVersion.V3
60+
version=aqz.ZarrVersion.V3,
61+
overwrite=True, # this will remove any existing data at my_stream.zarr
62+
arrays=[
63+
aqz.ArraySettings(
64+
output_key="array1",
65+
data_type=np.uint16,
66+
dimensions = [
67+
aqz.Dimension(
68+
name="t",
69+
type=aqz.DimensionType.TIME,
70+
array_size_px=0,
71+
chunk_size_px=32,
72+
shard_size_chunks=10
73+
),
74+
aqz.Dimension(
75+
name="c",
76+
type=aqz.DimensionType.CHANNEL,
77+
array_size_px=3,
78+
chunk_size_px=1,
79+
shard_size_chunks=1
80+
),
81+
aqz.Dimension(
82+
name="y",
83+
type=aqz.DimensionType.SPACE,
84+
array_size_px=1080,
85+
chunk_size_px=270,
86+
shard_size_chunks=2
87+
),
88+
aqz.Dimension(
89+
name="x",
90+
type=aqz.DimensionType.SPACE,
91+
array_size_px=1920,
92+
chunk_size_px=480,
93+
shard_size_chunks=2
94+
)
95+
]
96+
)
97+
]
5498
)
5599

56-
settings.dimensions.extend([
57-
aqz.Dimension(
58-
name="t",
59-
type=aqz.DimensionType.TIME,
60-
array_size_px=0,
61-
chunk_size_px=100,
62-
shard_size_chunks=10
63-
),
64-
aqz.Dimension(
65-
name="c",
66-
type=aqz.DimensionType.CHANNEL,
67-
array_size_px=3,
68-
chunk_size_px=1,
69-
shard_size_chunks=1
70-
),
71-
aqz.Dimension(
72-
name="y",
73-
type=aqz.DimensionType.SPACE,
74-
array_size_px=1080,
75-
chunk_size_px=270,
76-
shard_size_chunks=2
77-
),
78-
aqz.Dimension(
79-
name="x",
80-
type=aqz.DimensionType.SPACE,
81-
array_size_px=1920,
82-
chunk_size_px=480,
83-
shard_size_chunks=2
84-
)
85-
])
86-
87100
# Generate some random data: one time point, all channels, full frame
88-
my_frame_data = np.random.randint(0, 2**16, (3, 1080, 1920), dtype=np.uint16)
101+
my_frame_data = np.random.randint(0, 2 ** 16, (3, 1080, 1920), dtype=np.uint16)
89102

90103
stream = aqz.ZarrStream(settings)
91104
stream.append(my_frame_data)
105+
106+
# ... append more data as needed ...
107+
108+
# When done, close the stream to flush any remaining data
109+
stream.close()
110+
```
111+
112+
When all is said and done, the data will be stored in a Zarr dataset on disk at `my_stream.zarr`, which can be read by any Zarr-compatible library.
113+
It will contain one array named `array1` with the data organized in the specified dimensions according to the chunk and shard layout you specified.
114+
115+
If you instead had a multichannel acquisition with both brightfield and fluorescence channels, you could create a stream with two arrays, one for each channel type:
116+
117+
```python
118+
import acquire_zarr as aqz
119+
import numpy as np
120+
121+
# configure the stream with two arrays
122+
settings = aqz.StreamSettings(
123+
store_path="experiment.zarr",
124+
version=aqz.ZarrVersion.V3,
125+
overwrite=True, # this will remove any existing data at experiment.zarr
126+
arrays=[
127+
aqz.ArraySettings(
128+
output_key="sample1/brightfield",
129+
data_type=np.uint16,
130+
dimensions=[
131+
aqz.Dimension(
132+
name="t",
133+
type=aqz.DimensionType.TIME,
134+
array_size_px=0,
135+
chunk_size_px=100,
136+
shard_size_chunks=1
137+
),
138+
aqz.Dimension(
139+
name="c",
140+
type=aqz.DimensionType.CHANNEL,
141+
array_size_px=1,
142+
chunk_size_px=1,
143+
shard_size_chunks=1
144+
),
145+
aqz.Dimension(
146+
name="y",
147+
type=aqz.DimensionType.SPACE,
148+
array_size_px=1080,
149+
chunk_size_px=270,
150+
shard_size_chunks=2
151+
),
152+
aqz.Dimension(
153+
name="x",
154+
type=aqz.DimensionType.SPACE,
155+
array_size_px=1920,
156+
chunk_size_px=480,
157+
shard_size_chunks=2
158+
)
159+
]
160+
),
161+
aqz.ArraySettings(
162+
output_key="sample1/fluorescence",
163+
data_type=np.uint16,
164+
dimensions=[
165+
aqz.Dimension(
166+
name="t",
167+
type=aqz.DimensionType.TIME,
168+
array_size_px=0,
169+
chunk_size_px=100,
170+
shard_size_chunks=1
171+
),
172+
aqz.Dimension(
173+
name="c",
174+
type=aqz.DimensionType.CHANNEL,
175+
array_size_px=2, # two fluorescence channels
176+
chunk_size_px=1,
177+
shard_size_chunks=1
178+
),
179+
aqz.Dimension(
180+
name="y",
181+
type=aqz.DimensionType.SPACE,
182+
array_size_px=1080,
183+
chunk_size_px=270,
184+
shard_size_chunks=2
185+
),
186+
aqz.Dimension(
187+
name="x",
188+
type=aqz.DimensionType.SPACE,
189+
array_size_px=1920,
190+
chunk_size_px=480,
191+
shard_size_chunks=2
192+
)
193+
]
194+
)
195+
]
196+
)
197+
198+
stream = aqz.ZarrStream(settings)
199+
200+
# ... append data ...
201+
brightfield_frame_data = ... # define your brightfield frame data here
202+
fluorescence_frame_data = ... # define your fluorescence frame data here
203+
204+
stream.append(brightfield_frame_data, key="sample1/brightfield")
205+
stream.append(fluorescence_frame_data, key="sample1/fluorescence")
206+
207+
# ... append more data as needed ...
208+
209+
# When done, close the stream to flush any remaining data
210+
stream.close()
92211
```
93-
### Build Python Bindings from Source
94212

95-
The library must be built from source to contribute to the latest development version or to customize the installation for your system.
96-
To build the Python bindings from source, follow [these instructions](https://github.com/acquire-project/acquire-zarr/blob/main/README.md#building).
213+
See the [API reference](api_reference/index.md) for more details on the available settings and methods.
214+
215+
### Building the Python library from source
216+
217+
If you want to [contribute](for_contributors/index.md) to acquire-zarr, or customize the installation for your own system, you'll need to be able to build the Python bindings from source.
218+
The first step is to install the system dependencies as found in the ["Installing dependencies" section](https://github.com/acquire-project/acquire-zarr/blob/main/README.md#installing-dependencies) of the README.md file in the `acquire-zarr` repository.
219+
In your Python environment, you also need the following dependencies:
220+
221+
- cmake<4.0.0
222+
- ninja
223+
- pybind11[global]
224+
- setuptools
225+
- wheel
226+
227+
You can install these dependencies with:
228+
229+
```bash
230+
python -m pip install cmake<4.0.0 ninja pybind11[global] setuptools wheel
231+
```
97232

98-
## Get Started with C Bindings
233+
Then, clone the `acquire-zarr` repository and install acquire-zarr in your Python environment:
99234

100-
### Install the C Library
235+
```bash
236+
git clone --recursive https://github.com/acquire-project/acquire-zarr.git
237+
cd acquire-zarr
238+
python -m pip install .
239+
```
101240

102-
The `acquire-zarr` C library is distributed as a binary and headers, which you can download for your system from our [Releases page](https://github.com/acquire-project/acquire-zarr/releases). You will also need to install the following dependencies:
241+
## Getting started with C/C++
103242

104-
- [c-blosc](https://github.com/Blosc/c-blosc) >= 1.21.5
105-
- [nlohmann-json](https://github.com/nlohmann/json) >= 3.11.3
106-
- [minio-cpp](https://github.com/minio/minio-cpp) >= 0.3.0
107-
- [crc32c](https://github.com/google/crc32c) >= 1.1.2
243+
### Install the library
108244

109-
We suggest using [vcpkg](https://github.com/microsoft/vcpkg) or another package manager to handle dependencies.
245+
`acquire-zarr` provides a C API that works with both C and C++ projects.
246+
You can download the library for your system from our [Releases page](https://github.com/acquire-project/acquire-zarr/releases).
247+
The library ships with header files and precompiled binaries for Windows x86, macOS (x86 or ARM), or Linux (x86 or ARM), as well as source code examples and a sample CMakeLists.txt file.
110248

111-
[Here](https://github.com/acquire-project/acquire-zarr/blob/main/examples/CMakeLists.txt) is an example CMakeLists.txt file of C executables using acquire-zarr.
249+
### Usage
112250

113-
#### Usage
251+
Similarly with Python, the typical workflow for acquiring data in C is to create a stream by configuring it with the desired settings, and then to append data to the stream.
114252

115253
The library provides two main structs. First, `ZarrStream`, representing an output stream to a Zarr dataset.
116254
Second, `ZarrStreamSettings` to configure a Zarr stream.
117255

118256
A typical use case for a 4-dimensional acquisition in C might look like this:
119257

120-
```c
258+
```C
121259
#include "acquire.zarr.h"
122260
#include "assert.h"
123261

@@ -171,7 +309,25 @@ int main() {
171309

172310
Look at [acquire.zarr.h](include/acquire.zarr.h) for more details.
173311

174-
### Building the C Library from Source
312+
### Building the library from source
313+
314+
If you want to [contribute](for_contributors/index.md) to acquire-zarr, or customize the installation for your own system, you'll need to be able to build the library from source.
315+
The first step is to install the system dependencies as found in the ["Installing dependencies" section](https://github.com/acquire-project/acquire-zarr/blob/main/README.md#installing-dependencies) of the README.md file in the `acquire-zarr` repository.
316+
As in the README, we recommend using vcpkg to install them.
317+
You will also need CMake to build the library.
318+
319+
Once you have built the library, try running the tests to ensure everything is working correctly.
320+
321+
```bash
322+
cd build # or wherever you built the library
323+
ctest -C ${BUILD_TYPE} -L acquire-zarr --output-on-failure
324+
```
325+
326+
where `${BUILD_TYPE}` can be `Debug`, `Release`, or `RelWithDebInfo`, depending on how you built the library.
327+
(If you did not specify a build type, it defaults to `Debug`.)
175328

176-
The library must be built from source to contribute to the latest development version or to incorporate the library into an existing program.
177-
To build the C library from source, follow [these instructions](https://github.com/acquire-project/acquire-zarr/blob/main/README.md#building).
329+
If the tests pass, you can install the library to your system with:
330+
331+
```bash
332+
cmake --install . --config ${BUILD_TYPE}
333+
```

‎docs/index.md‎

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -16,18 +16,7 @@ acquisition workflows since it does not rely on runtime or hardware support.
1616

1717
## Installation
1818

19-
### Install the Python library
20-
21-
To install the `acquire-zarr` Python library on Windows, macOS, or Ubuntu, run the following command:
22-
23-
```bash
24-
python -m pip install acquire-zarr
25-
```
26-
For more details, check out the [Get Started page](get_started.md).
27-
28-
### Build the C Library from Source
29-
30-
To build the C Library, follow [these instructions](https://github.com/acquire-project/acquire-zarr/blob/main/README.md).
19+
See our [Getting Started](get_started.md) page for installation instructions.
3120

3221
## Guides
3322

0 commit comments

Comments
 (0)