Skip to content

Commit 468ef58

Browse files
markbadernormanrz
andauthored
Update CLI docs (#1299)
* Update docs. * Remove contribute.md. * Implement requested changes. * parse url in download command. * Apply suggested changes. * unify args * Refer to pypi and run format command. --------- Co-authored-by: Norman Rzepka <[email protected]>
1 parent 8f05050 commit 468ef58

File tree

17 files changed

+1927
-882
lines changed

17 files changed

+1927
-882
lines changed

docs/mkdocs.yml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -282,6 +282,19 @@ nav:
282282
- Task: api/webknossos/administration/task.md
283283
- CLI Reference:
284284
- Overview: cli/index.md
285+
- Install and Run: cli/install.md
286+
- Environment Variables: cli/environment_variables.md
287+
- Distribution Strategies: cli/distribution_strategies.md
288+
- CLI Commands:
289+
- Convert Images: cli/convert.md
290+
- Compress: cli/compress.md
291+
- Copy Datasets: cli/copy.md
292+
- Download: cli/download.md
293+
- Upload: cli/upload.md
294+
- Downsample: cli/downsample.md
295+
- Upsample: cli/upsample.md
296+
- Export as Tiff: cli/export-as-tiff.md
297+
- Merge Fallback: cli/merge-fallback.md
285298
- Version Changelog:
286299
- webknossos-py/stability_policy.md
287300
- webknossos-py/changelog.md

docs/src/cli/compress.md

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# `compress`
2+
3+
The `compress` command allows you to compress a WEBKNOSSOS dataset. It supports compressing specific layers and magnifications (mags) while leveraging parallel execution and customizable distribution strategies.
4+
5+
## Usage
6+
7+
```bash
8+
compress [OPTIONS] TARGET
9+
```
10+
11+
### Arguments
12+
13+
- **TARGET**
14+
Path to the WEBKNOSSOS dataset to be compressed.
15+
Example: `/path/to/dataset`.
16+
17+
### Options
18+
19+
- `--layer-name`
20+
Name of the layer to be compressed. If not provided, all layers will be compressed.
21+
Example: `--layer-name segmentation`.
22+
23+
- `--mag`
24+
Magnifications (mags) to be compressed. Specify as a number or a hyphen-separated string (e.g., `2` or `2-2-2`).
25+
For multiple mags, use the option multiple times.
26+
Example: `--mag 1 --mag 2 --mag 4-4-2`.
27+
28+
- `--jobs`
29+
Number of processes to be spawned for parallel execution.
30+
Default: Number of CPU cores.
31+
32+
- `--distribution-strategy`
33+
Strategy to distribute the task across CPUs or nodes.
34+
Options: `multiprocessing`, `slurm`, `kubernetes`, `sequential`.
35+
Default: `multiprocessing`.
36+
37+
- `--job-resources`
38+
Specify resources for jobs when using the SLURM distribution strategy.
39+
Example: `--job-resources '{"mem": "10M"}'`.
40+
41+
## Example Commands
42+
43+
### Compress all layers in a dataset:
44+
```bash
45+
compress /path/to/dataset
46+
```
47+
This command compresses all layers in the specified dataset.
48+
49+
### Compress a specific layer:
50+
```bash
51+
compress --layer-name segmentation /path/to/dataset
52+
```
53+
This command compresses only the `segmentation` layer in the dataset.
54+
55+
### Compress specific magnifications:
56+
```bash
57+
compress --mag 1 --mag 2 /path/to/dataset
58+
```
59+
This command compresses the `1` and `2` magnifications in all layers of the dataset.
60+
61+
### Compress using SLURM with custom job resources:
62+
```bash
63+
compress --distribution-strategy slurm --job-resources '{"mem": "10M"}' /path/to/dataset
64+
```
65+
This command uses the SLURM distribution strategy with custom memory allocation for each job.
66+
67+
## Notes
68+
69+
- Ensure that the target dataset path is accessible and properly configured.
70+
- Use the `--layer-name` option to target specific layers for compression.
71+
- For SLURM distribution, make sure the required resources are specified using the `--job-resources` option.
72+
- This command is designed to handle large datasets efficiently by leveraging parallel processing and customizable distribution strategies.

0 commit comments

Comments
 (0)