You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This repository contains specifications defined by the bioimage.io community. These specifications are used for defining fields in YAML files which we called `Resource Description Files` or `RDF`. The RDFs can be downloaded or uploaded to the [bioimage.io website](https://bioimage.io), produced or consumed by bioimage.io-compatible consumers(e.g. image analysis software or other website). Currently we defined two types of RDFs: a dedicated RDF specification for AI models (i.e. `model description`) and a generic description specification. The model description is a RDF with additional fields that specifically designed for describing AI models.
7
+
This repository contains specifications defined by the bioimage.io community. These specifications are used for defining fields in YAML 1.2 files which should be named `bioimageio.yaml`. Such a bioimageio.yaml --- along with files referenced in it --- can be downloaded from or uploaded to the [bioimage.io website](https://bioimage.io) and may be produced or consumed by bioimage.io-compatible consumers(e.g. image analysis software like ilastik).
8
8
9
-
All the bioimage.io-compatible RDF must fulfill the following rules:
9
+
bioimage.io-compatible resources must fulfill the following rules:
10
10
11
-
* Must be a YAML file encoded as UTF-8; If yaml syntax version is not specified to be 1.1 in the first line by `% YAML 1.1` it must be equivalent in yaml 1.1 and yaml 1.2. For differences see <https://yaml.readthedocs.io/en/latest/pyyaml.html#differences-with-pyyaml>.
12
-
* The RDF file extension must be `.yaml` (not `.yml`)
13
-
* The RDF file can be saved in a folder (or virtual folder) or in a zip package, the following additional rules must apply:
14
-
1. When stored in a local file system folder, github repo, zenodo deposition, blob storage virtual folder or similar kind, the RDF file name should match the pattern of `*.rdf.yaml`, for example `my-model.rdf.yaml`.
15
-
2. When the RDF file and other files are zipped into a RDF package, it must be named as `rdf.yaml`.
11
+
Note that the Python package PyYAML does not support YAML 1.2 .
12
+
We therefore use and recommend [ruamel.yaml](https://ruamelyaml.readthedocs.io/en/latest/).
13
+
For differences see <https://ruamelyaml.readthedocs.io/en/latest/pyyaml>.
16
14
17
-
As a general guideline, please follow the model description spec to describe AI models and use the generic description spec for other resource types including `dataset`, `application`. You will find more details about these two specifications in the following sections. Please also note that the best way to check whether your RDF file is bioimage.io-compliant is to run the bioimage.io validator against it.
15
+
Please also note that the best way to check whether your `bioimageio.yaml` file is bioimage.io-compliant is to call `bioimageio.core.validate` from the [bioimageio.core](https://github.com/bioimage-io/core-bioimage-io-python) Python package.
16
+
The [bioimageio.core](https://github.com/bioimage-io/core-bioimage-io-python) Python package also provides the bioimageio command line interface (CLI) with the `validate` command:
17
+
18
+
```terminal
19
+
bioimageio validate path/to/your/bioimageio.yaml
20
+
```
18
21
19
22
## Format version overview
20
23
21
24
All bioimage.io description formats are defined as [Pydantic models](https://docs.pydantic.dev/latest/).
22
-
To fully verify any resource description `bioimageio.core.validate` from the [bioimageio.core]() Python package should be used.
23
25
24
26
| type | format version | documentation |
25
27
| --- | --- | --- |
@@ -32,7 +34,7 @@ To fully verify any resource description `bioimageio.core.validate` from the [bi
32
34
33
35
## JSON schema
34
36
35
-
Simplified descriptions are available as [JSON schema]():
37
+
Simplified descriptions are available as [JSON schema](https://json-schema.org/):
36
38
37
39
| bioimageio.spec version | JSON schema |
38
40
| --- | --- |
@@ -41,136 +43,32 @@ Simplified descriptions are available as [JSON schema]():
41
43
42
44
These are primarily intended for syntax highlighting and form generation.
43
45
44
-
## Resource Description Specification
45
-
46
-
A bioimage.io-compatible Resource Description File (RDF) is a YAML file with a set of [specifically defined fields](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/generic_spec_latest.md).
47
-
48
-
We provide some [examples for using RDFs to describe applications, notebooks, datasets etc](https://github.com/bioimage-io/spec-bioimage-io/blob/main/example_specs/rdf-examples.md).
49
-
50
-
## Model Description Specification
51
-
52
-
Besides the generic description spec, the `model description spec` defines a file format based on the [YAML 1.2](https://en.wikipedia.org/wiki/YAML) for representing pretrained AI models. This format is used to describe models hosted on the [bioimage.io](https://bioimage.io) model repository site.
53
-
54
-
You can find documentation of the `model description`[here](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/model_spec_latest.md).
The [`Collection Description Specification`](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/collection_spec_latest.md)(`collection description`) defines a file format for representing collections of resources for the [bioimage.io](https://bioimage.io) website.
63
-
64
-
You can find the latest `collection description`[here](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/collection_spec_latest.md).
65
-
66
-
## Linking resource items
67
-
68
-
todo: outdated
69
-
You can create links to connect resource items by adding another resource item id to the `links` field. For example, if you want to associate an applicaiton with a model, you can set the links field of the models like the following:
In order to make a resource description file (RDF) available on <https://bioimage.io>, you can use the [bioimage.io uploader](https://bioimage.io/#/upload/), which assists you in uploading it and any associated files to [Zenodo](https://zenodo.org/).
87
-
Alternatively you can upload directly to [Zenodo](https://zenodo.org/). In this case keep in mind to create an `rdf.yaml` file and add the keyword `bioimage.io` to your zenodo record for our CI to discover it.
48
+
We provide some [examples for using bioimageio.yaml files to describe models, applications, notebooks and datasets](https://github.com/bioimage-io/spec-bioimage-io/blob/main/example_specs/examples.md).
88
49
89
50
## 💁 Recommendations
90
51
91
-
todo: outdated
92
-
93
-
* For AI models, consider using the model-specific spec (i.e. [model description](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/model_spec_latest.md)) instead of the generic description. Only fallback to the generic description if writing model specific RDF is not possible for some reason.
94
-
* The RDF or package file name should not contain spaces or special characters, it should be concise, descriptive, in kebab case or camel case.
95
-
* Due to the limitations of storage services such as Zenodo, which does not support subfolders, it is recommended to place other files in the same directory level of the RDF file and try to avoid using subdirectories.
96
-
* Use the [bioimage.io spec validator](#bioimageio-spec-validator) to verify your YAML file
97
-
* Store the yaml file in a version controlled Git repository (e.g. Github or Gitlab)
98
-
* Use or upgrade to the latest format version
99
-
100
-
# ⌨ bioimageio command-line interface (CLI)
101
-
102
-
todo: outdated
103
-
The bioimage.io command line tool makes it easy to work with bioimage.io RDFs.
104
-
A basic version of it, documented here, is provided by the [bioimageio.spec package](bioimageio-python-package), which is extended by the [bioimageio.core](https://github.com/bioimage-io/core-bioimage-io-python) package.
105
-
106
-
## 🧪 validate
107
-
108
-
todo: outdated
109
-
It is recommended to use this validator to verify your models when you write it manually or develop tools for generating RDF files.
52
+
* Due to the limitations of storage services such as Zenodo, which does not support subfolders, it is recommended to place other files in the same directory level of the `bioimageio.yaml` file and try to avoid using subdirectories.
53
+
* Use the [bioimageio.core Python package](https://github.com/bioimage-io/core-bioimage-io-python) to validate your `bioimageio.yaml` file.
54
+
* bioimageio.spec keeps evolving. Try to use and upgrade to the most current format version!
110
55
111
-
Use the `validate` command to check for formatting errors like missing or invalid values:
56
+
## ⌨ bioimageio command-line interface (CLI)
112
57
113
-
```
114
-
bioimageio validate <MY-MODEL-SOURCE>
115
-
116
-
```
117
-
118
-
`<MY-MODEL-SOURCE>`may be a local RDF yaml "`<MY-MODEL>/rdf.yaml`" or a DOI / URL to a zenodo record, or a URL to an rdf.yaml file.
119
-
120
-
To see if your model is compatible to the [latest bioimage.io model format](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/model_spec_latest.md) use the spec validator with the `--update-format` flag:
The output of the `validate` command will indicate missing or invalid fields in the model file. For example, if the field `timestamp` was missing it would print the following:
129
-
130
-
```
131
-
132
-
{'timestamp': ['Missing data for required field.']}
133
-
134
-
```
135
-
136
-
or if the field `test_inputs` does not contain a list, it would print:
137
-
138
-
```
139
-
140
-
{'test_inputs': ['Not a valid list.']}.
141
-
142
-
```
143
-
144
-
## update-format
145
-
146
-
Similar to the `validate` command with `--update-format` flag the `update-format` command attempts to convert an RDF
147
-
to the latest applicable format version, but saves the result in a file for further manual editing:
Copy file name to clipboardExpand all lines: example_specs/examples.md
+14-9Lines changed: 14 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,14 @@
1
1
# Examples for describing the
2
2
3
-
### Describing applications
4
-
The RDF can be used to describe applications. To do so set the `type` field to `application`.\
3
+
## Describing applications
4
+
5
+
A bioimageio.yaml file can be used to describe applications. To do so set the `type` field to `application`.\
5
6
For regular software package with a downloadable file, you can set `download_url` to the downloadable file, for example, you can upload the executable files as Github release, deposit it on Zenodo, or even generate a sharable url from Dropbox/Google Drive.\
6
7
For web application, set `source` to the url of the web application. Users can then click and redirect to your web application. However, simple integration will not support features such as opening dataset or models with your application.
7
8
8
9
It is recommended to build BioEngine Apps such that users can directly try and use them in bioimage.io. See [here](https://github.com/bioimage-io/bioimage.io/blob/main/docs/bioengine_apps/build-bioengine-apps.md) for more details.\
9
10
Below is an example for [Kaibu](https://kaibu.org), which is a BioEngine/ImJoy compatible web application:
For more application examples, see the [manifest for ImJoy](https://github.com/imjoy-team/bioimage-io-models/blob/master/manifest.bioimage.io.yaml).
27
30
28
-
### Describing notebooks and scripts
31
+
## Describing notebooks and scripts
29
32
30
33
Jupyter notebooks, Google Colab or other types of executable notebooks or scripts are considered as applications, therefore, you should use `type=application` and add additional tags. For example:
@@ -58,26 +62,27 @@ Jupyter notebooks, Google Colab or other types of executable notebooks or script
58
62
- Dataset_fnet_3D_ZeroCostDL4Mic
59
63
```
60
64
61
-
62
65
### Describing datasets and other types
66
+
63
67
The RDF allows for the description of datasets (type=`dataset`) and other potential resources, you can use set `source` and/or `download_url` to point to the resource, or use `attachments` to specify a list of associated files.
64
68
65
69
For examples, see entries `dataset`/`notebook` in the [ZeroCostDL4Mic](https://github.com/HenriquesLab/ZeroCostDL4Mic/blob/master/manifest.bioimage.io.yaml) collection.
66
70
67
-
68
71
### Describing models with the unspecific RDF(not recommended, use the Model RDF instead)
69
-
In general, it is discouraged to use the general RDF to describe AI models and we recommend to follow the [model RDF spec](#model-resource-description-file-specification) instead. However, in some cases, it is not possible to provide detailed fields defined in the [model RDF spec](#model-resource-description-file-specification), the general RDF can be used for discribing AI models.
72
+
73
+
In general, it is discouraged to use the general RDF to describe AI models and we recommend to follow the [model spec](#model-resource-description-file-specification) instead. However, in some cases, it is not possible to provide detailed fields defined in the [model spec](#model-resource-description-file-specification), the general RDF can be used for discribing AI models.
70
74
To do that, you need to first set the `type` field to `model`.\
71
75
A basic integration would be simply provide a `download_url` to a zip file (for example, with the model weights, source code or executable binary file) hosted on Github releases, Dropbox, Google Drive etc. For example:
0 commit comments