Skip to content

annotation.rst

Latest commit

 

History

History
235 lines (179 loc) · 10.1 KB

annotation.rst

File metadata and controls

235 lines (179 loc) · 10.1 KB

Annotation format

The precomputed annotation format defines an annotation collection for a given n-dimensional coordinate space and one of the following five geometry types:

  • Points (represented by a single position)
  • Line segments (represented by the two endpoint positions)
  • Axis-aligned bounding boxes (represented by two positions)
  • Axis-aligned ellipsoids (represented by a center position and radii vector)
  • Polylines (represented by the number of points in the line, followed by the positions of the points)

All annotations within the annotation collection have the same geometry type.

Each annotation is defined by:

An annotation collection is represented as a directory tree consisting of the following:

:file:`info` metadata file

The :file:`info` file is a JSON-format text file with the following schema:

.. json:schema:: PrecomputedAnnotation

Annotation id index

The annotation id index supports efficient retrieval of individual annotations by their uint64 id, and is used by Neuroglancer when selecting or hovering over an annotation.

The annotation id index maps each uint64 annotation id to the [encoded representation](#single-annotation-encoding) of the single corresponding annotation. Depending on whether :json:schema:`~PrecomputedAnnotation.by_id.sharding` parameters are specified, the index is stored either in the :ref:`unsharded uint64 index format<precomputed-annotation-format-unsharded-uint64-index>` or the :ref:`sharded uint64 index format<precomputed-annotation-format-sharded-uint64-index>`.

Note that the geometry and property data is duplicated in all indices, but only the annotation id index encodes the complete lists of related object ids.

Single annotation encoding

Within the annotation id index, each annotation is encoded in the following binary format:

Unsharded uint64 index

The data corresponding to each uint64 annotation id or related object id is stored in a file named :file:`{<id>}` within the directory indicated by the :json:`"key"` member, where :file:`{<id>}` is the base-10 string representation of the uint64 id.

Sharded uint64 index

The uint64 annotation id or related object id is used directly as the key within the sharded representation within the directory indicated by the :json:`"key"` member.

Related object id index

The related object id index supports efficient retrieval of the list of annotations associated via a given relationship with a given object id, and is used by Neuroglancer when filtering by segment ids.

The related object id index maps each uint64 object id to the :ref:`encoded representation<precomputed-annotation-format-multiple-annotation-encoding>` of the list of related annotations. Depending on whether :json:schema:`~PrecomputedAnnotation.relationships.sharding` parameters are specified, the index is stored either in the :ref:`unsharded uint64 index format<precomputed-annotation-format-unsharded-uint64-index>` or the :ref:`sharded uint64 index format<precomputed-annotation-format-sharded-uint64-index>`.

Multiple annotation encoding

Both the related object id index and the spatial index encode lists of annotations in the following binary format:

For the related object id index, the order of the annotations does not matter. For the spatial index, the annotations should be ordered randomly.

Spatial index

The spatial index supports efficient retrieval of the set of annotations that intersects a given bounding box, with optional subsampling down to a desired maximum density.

The spatial index is used by Neuroglancer when not filtering by related segment ids.

Each spatial index level maps cell positions within the grid specified by the :json:schema:`~PrecomputedAnnotation.spatial.chunk_size` and :json:schema:`~PrecomputedAnnotation.spatial.grid_shape` to a spatially uniform subsample of annotations intersecting that grid cell.

  • A grid cell with coordinates cell corresponds to a spatial interval in dimension d of [lower_bound[d] + cell[d] * chunk_size[d], lower_bound[d] + (cell[d] + 1) * chunk_size[d]]
  • The chunk_size for spatial index level i+1 should evenly divide the chunk_size for spatial index level i. The grid cells within level i+1 that are contained within a single level i grid cell are considered the child cells. For each level, the elementwise product of the grid_shape and the chunk_size should equal upper_bound - lower_bound.
  • Typically the grid_shape for level 0 should be a vector of all 1 (with chunk_size equal to upper_bound - lower_bound), and each component of chunk_size of each successively level should be either equal to, or half of, the corresponding component of the prior level chunk_size, whichever results in a more spatially isotropic chunk.

The spatial index levels should be computed as follows:

  • For each grid position cell at the coarsest level, compute the set remaining_annotations(0, cell) of annotations that intersect the cell. Note that a single annotation may intersect multiple cells.
  • Sequentially generate spatial index level, starting at level=0 (the coarsest level):
    • Define maxCount(level) to be the maximum over all cell positions of the size of remaining_annotations(level, cell).
    • For each cell:
    • Continue generating successively finer spatial index levels until no annotations remain.

Unsharded spatial index

The encoded annotation list corresponding to a grid cell cell is stored within the directory indicated by the :json:schema:`~PrecomputedAnnotation.spatial.key` member in a file named cell.join('_'), i.e. the base-10 string representations of the grid cell coordinates separated by the '_' character. For example, cell (1, 2, 3) is stored in the file named 1_2_3.

Sharded spatial index

The :ref:`compressed Morton code<precomputed-compressed-morton-code>` of the grid cell is used as the key within the sharded representation stored in the directory indicated by the :json:schema:`~PrecomputedAnnotation.spatial.key` member.