Skip to content

Commit be4f68b

Browse files
authored
Merge pull request #1636 from AllenNeuralDynamics/release-v2.2.0
Release v2.2.0
2 parents 06e703d + fa7e7dc commit be4f68b

40 files changed

+2860
-230
lines changed

docs/base/core/acquisition.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,10 @@ For example, in the `"Brain Computer Interface"` project name, good acquisition
2727

2828
You should use the `Code.parameters` field to store your stimulus properties for each [StimulusEpoch](#stimulusepoch). We have pre-existing parameter schemas for a subset of stimuli defined [here](components/stimulus.md) or you can define your own schema.
2929

30+
## Manipulations
31+
32+
Procedures that occur during an acquisition (between the start_time and end_time) should be recorded in the `Acquisition.manipulations` using a [Manipulation](#manipulation).
33+
3034
## FAQs
3135

3236
### When should a DataStream be split in two
4.77 KB
Loading
98.2 KB
Loading

docs/source/acquisition.md

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,10 @@ For example, in the `"Brain Computer Interface"` project name, good acquisition
2727

2828
You should use the `Code.parameters` field to store your stimulus properties for each [StimulusEpoch](#stimulusepoch). We have pre-existing parameter schemas for a subset of stimuli defined [here](components/stimulus.md) or you can define your own schema.
2929

30+
## Manipulations
31+
32+
Procedures that occur during an acquisition (between the start_time and end_time) should be recorded in the `Acquisition.manipulations` using a [Manipulation](#manipulation).
33+
3034
## FAQs
3135

3236
### When should a DataStream be split in two
@@ -88,6 +92,7 @@ while the StimulusEpoch represents all stimuli being presented.
8892
| `maintenance` | List[[Maintenance](components/measurements.md#maintenance)] | Maintenance (List of maintenance on instrument prior to acquisition.) |
8993
| `data_streams` | List[[DataStream](acquisition.md#datastream)] | Data streams (A data stream is a collection of devices that are acquiring data simultaneously. Each acquisition can include multiple streams. Streams should be split when configurations are changed.) |
9094
| `stimulus_epochs` | List[[StimulusEpoch](acquisition.md#stimulusepoch)] | Stimulus (A stimulus epoch captures all stimuli being presented during an acquisition. Epochs should be split when the purpose of the stimulus changes.) |
95+
| `manipulations` | List[[Manipulation](acquisition.md#manipulation)] | Manipulations (Procedures performed during the acquisition.) |
9196
| `subject_details` | Optional[[AcquisitionSubjectDetails](acquisition.md#acquisitionsubjectdetails)] | Subject details |
9297

9398

@@ -102,7 +107,7 @@ Details about the subject during an acquisition
102107
| `animal_weight_prior` | `Optional[decimal.Decimal]` | Animal weight (g) (Animal weight before procedure) |
103108
| `animal_weight_post` | `Optional[decimal.Decimal]` | Animal weight (g) (Animal weight after procedure) |
104109
| `weight_unit` | [MassUnit](aind_data_schema_models/units.md#massunit) | Weight unit |
105-
| `anaesthesia` | Optional[[Anaesthetic](components/surgery_procedures.md#anaesthetic)] | Anaesthesia |
110+
| `anaesthesia` | Optional[[Anaesthetic](components/surgery_procedures.md#anaesthetic)] | Anaesthesia (Anaesthesia present during entire acquisition, use Manipulation for partial anaesthesia) |
106111
| `mouse_platform_name` | `str` | Mouse platform |
107112
| `reward_consumed_total` | `Optional[decimal.Decimal]` | Total reward consumed (mL) |
108113
| `reward_consumed_unit` | Optional[[VolumeUnit](aind_data_schema_models/units.md#volumeunit)] | Reward consumed unit |
@@ -125,6 +130,19 @@ same time.
125130
| `connections` | List[[Connection](components/connections.md#connection)] | Connections (Connections are links between devices that are specific to this acquisition (i.e. not already defined in the Instrument)) |
126131

127132

133+
### Manipulation
134+
135+
Description of procedures performed during an acquisition.
136+
137+
| Field | Type | Title (Description) |
138+
|-------|------|-------------|
139+
| `start_time` | `datetime (timezone-aware)` | Manipulation start time (Must be between the acquisition start and end times) |
140+
| `end_time` | `datetime (timezone-aware)` | Manipulation end time (Must be between the acquisition start and end times) |
141+
| `procedures` | Optional[List[[Injection](components/injection_procedures.md#injection) or [BrainInjection](components/surgery_procedures.md#braininjection)]] | Procedures (Procedures performed during the manipulation) |
142+
| `anaesthesia` | Optional[[Anaesthetic](components/surgery_procedures.md#anaesthetic)] | Anaesthesia |
143+
| `notes` | `Optional[str]` | Notes |
144+
145+
128146
### PerformanceMetrics
129147

130148
Summary of a StimulusEpoch

docs/source/aind_data_schema_models/external.md

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,20 @@ External registries act as an application programming interfaces (API). They all
66

77
### MouseAnatomyModel
88

9-
Base model for mouse anatomy
9+
[EMAPA](https://www.ebi.ac.uk/ols4/ontologies/emapa)
10+
11+
Base model for mouse anatomy. Some examples:
1012

1113
| Name | Registry | Registry Identifier |
1214
|------|-------|--------|
13-
| `todo` | `todo` | `todo` |
15+
| `heart` | `Registry.EMAPA` | `16105` |
16+
17+
### Gene
18+
19+
[GenBank](https://www.ncbi.nlm.nih.gov/genbank/)
20+
21+
Base model for genes. One example:
22+
23+
| Name | Description | Registry | Registry Identifier |
24+
|------|------|-------|--------|
25+
| `gfp` | `Human adenovirus B isolate 340-2010 DNA, complete genome` | `Registry.GENBANK` | `LN515608` |

docs/source/components/devices.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -529,7 +529,7 @@ Description of a lick spout
529529
| `spout_diameter` | `decimal.Decimal` | Spout diameter (mm) |
530530
| `spout_diameter_unit` | [SizeUnit](../aind_data_schema_models/units.md#sizeunit) | Spout diameter unit |
531531
| `solenoid_valve` | [Device](#device) | Solenoid valve |
532-
| `lick_sensor` | [Device](#device) | Lick sensor |
532+
| `lick_sensor` | [Device](#device) or [HarpDevice](#harpdevice) | Lick sensor |
533533
| `lick_sensor_type` | Optional[[LickSensorType](../aind_data_schema_models/devices.md#licksensortype)] | Lick sensor type |
534534
| `name` | `str` | Device name |
535535
| `serial_number` | `Optional[str]` | Serial number |

docs/source/components/identifiers.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Code or script identifier
1212
| `name` | `Optional[str]` | Name |
1313
| `version` | `Optional[str]` | Code version |
1414
| `container` | Optional[[Container](#container)] | Container |
15-
| `run_script` | `Optional[pathlib._local.Path]` | Run script (Path to run script) |
15+
| `run_script` | `Optional[pathlib.Path]` | Run script (Path to run script) |
1616
| `language` | `Optional[str]` | Programming language (Programming language used) |
1717
| `language_version` | `Optional[str]` | Programming language version |
1818
| `input_data` | Optional[List[[DataAsset](#dataasset) or [CombinedData](#combineddata)]] | Input data (Input data used in the code or script) |

docs/source/coordinate_systems.md

Lines changed: 52 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,18 @@ An [Origin](aind_data_schema_models/coordinates.md#origin) is a point in space,
4444

4545
Each [Axis](components/coordinates.md#axis) is a combination of an [AxisName](aind_data_schema_models/coordinates.md#axisname) and [Direction](aind_data_schema_models/coordinates.md#direction).
4646

47+
### Units
48+
49+
Each [CoordinateSystem](components/coordinates.md#coordinatesystem) defines its origin, axis direction, and units. All of this information is inherited by the [Translation](components/coordinates.md#translation), [Rotation](components/coordinates.md#rotation), and [Scale](components/coordinates.md#scale) transforms that are applied. The only exception is for rotations, where we ask you to specify for each rotation the units (degrees or radians).
50+
51+
#### 3D vs 4D and Depth
52+
53+
Because skull shapes vary across animals the most useful coordinates to re-create insertions across animals are often the AP/ML position of the entry coordinate and then the "depth", i.e. the insertion distance of the tip of the inserted device from the brain (or dura) surface, whether a probe, fiber, needle, whatever. To support these kinds of insertions we include a depth axis option.
54+
55+
In most cases users should report the coordinates of the *entry coordinate* at the brain/dura surface using the first three (AP, ML, SI) values and then the depth *from the brain/dura surface* in the fourth depth coordinate. Recording all three coordinates disambiguates between the two ways that a probe can be "dropped" to the brain surface (either along the SI axis or down the probe depth axis). You can also use a 3-dimensional coordinate system (AP, ML, Depth) but we don't recommend it, since you need to either report in a protocol or in the notes how you dropped from the AP/ML plane down to the brain surface.
56+
57+
Note that in general, the process by which you perform an insertion should be recorded in a protocol, especially if there are specific details a user would need to know about how to interpret coordinates.
58+
4759
### CoordinateSystemLibrary
4860

4961
We know that allowing complete flexibility with coordinate systems will be a source of confusion. With that in mind, we encourage everybody to use the `CoordinateSystemLibrary` class, which comes with a pre-defined set of standard coordinate systems. For example, you can import the `BREGMA_ARI` coordinate system and then re-use it as follows:
@@ -59,45 +71,53 @@ coordinate_system_name = CoordinateSystemLibrary.BREGMA_ARI.name
5971

6072
You can always define your own coordinate system. If you find yourself re-using a coordinate system that isn't available in the library across multiple projects, please request an update to the library by opening an [issue](https://github.com/AllenNeuralDynamics/aind-data-schema/issues).
6173

74+
## Measured Coordinates
75+
76+
During a [Surgery](components/subject_procedures.md#surgery) requiring stereotaxic coordinates the surgeon will typically reference the stereotax or insertion device to a known coordinate, almost always Bregma. It's useful at this time to also measure the relative position of other known landmarks, like Lambda. The `Surgery.measured_coordinates` field is intended to store this data, for example for a surgery in the BREGMA_ARI coordinate system and where Lambda is measured 4.1 mm posterior to Bregma you would include:
77+
78+
```{code} python
79+
measured_coordinates = {
80+
Origin.LAMBDA: Translation(translation=[-4.1, 0, 0])
81+
}
82+
```
83+
84+
Some notes: the position is *negative* because in BREGMA_ARI the AP axis points positive in the anterior direction, the other two axes are zero because this skull has apparently already been leveled, and finally no units are included in the translation itself because they are implied by the coordinate system, see [units](#units) above.
85+
86+
## Rotations
87+
88+
Rotations are applied using the [scipy Euler angle conventions](https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.transform.Rotation.from_euler.html#scipy.spatial.transform.Rotation.from_euler), in "xyz" order. Positive angles rotate counter-clockwise.
89+
90+
It can be complicated to translate your rotations into the default conventions in situations where you aren't in control of the coordinate system definition. In that situation, it is preferable to construct an affine rotation matrix directly and pass it using the [Affine](components/coordinates.md#affine) object.
91+
6292
## Device transforms
6393

6494
To understand the position and orientation of a **device** in an instrument requires knowing three things: (1) the coordinate system for the instrument, (2) the coordinate system for the device, and (3) the coordinate system transform i.e. how a point in one coordinate system is translated, rotated, and scaled to the other. For example, a [CameraAssembly](components/devices.md#cameraassembly) is a positioned device: it has three special fields `relative_position`, `coordinate_system`, and `transform`. The relative position is required for all positioned devices while the transform and coordinate system are only required when a device's exact position will have an impact on the interpretation/analysis of data.
6595

96+
### Relative Position
97+
98+
For devices where the exact position is not important or is unknown, simply tell us where the device is *roughly* relative to the origin. By combining several [AnatomicalRelative](aind_data_schema_models/coordinates.md#anatomicalrelative) directions in a list, for example `[AnatomicalRelative.ANTERIOR, AnatomicalRelative.SUPERIOR]`, etc, you can describe the position.
99+
100+
### Exact Position
101+
66102
The transform we require for devices is the device to instrument transform. I.e. given the origin of the device (0, 0, 0) and the three axis directions, what will be the position of the origin and what direction will the three axes point in the instrument's coordinate system.
67103

68-
Here is an example of an affine rotation matrix and a translation applied to a [Monitor](components/devices.md#monitor) device being positioned in the standard `BREGMA_ARI` coordinate system, see above for the definition. Note that the affine matrix and translation could be composed together, we're keeping them separate here for interpretability.
104+
#### Building an exact position from scratch
69105

70-
```
71-
Monitor(
72-
relative_position=[AnatomicalRelative.ANTERIOR, AnatomicalRelative.LEFT, AnatomicalRelative.INFERIOR],
73-
coordinate_system=CoordinateSystemLibrary.SIPE_MONITOR_RTF,
74-
transform=[
75-
Affine(
76-
affine_transform=[
77-
[
78-
-0.80914,
79-
-0.58761,
80-
0,
81-
],
82-
[
83-
-0.12391,
84-
0.17063,
85-
0.97751,
86-
],
87-
[
88-
-0.5744,
89-
0.79095,
90-
-0.21087,
91-
],
92-
],
93-
),
94-
Translation(
95-
translation=[
96-
0.08751,
97-
-0.12079,
98-
0.02298,
99-
],
100-
),
106+
The easiest way to construct the exact position is to start by drawing two pictures of your device. First, draw your device in the instrument coordinate system at the origin. Define a "neutral" position: for example, a monitor at neutral might be facing posterior as if the mouse is looking at it. Select an origin coordinate for the device, a logical point for a monitor is the center of the screen. Then, define this device coordinate system by adding axis direction information matching the picture you drew so that the X, Y, and Z axes are matched between your instrument coordinate system and the device. Assuming that our instrument coordinate system is BREGMA_ARI, i.e. +X = +Anterior, +Y = +Right, and +Z = +Inferior, then our monitor should be defined as:
107+
108+
```{code} python
109+
CoordinateSystem(
110+
name="MONITOR_BRU",
111+
origin=Origin.FRONT_CENTER,
112+
axis_unit=SizeUnit.MM,
113+
axes=[
114+
Axis(name=AxisName.X, direction=Direction.FB),
115+
Axis(name=AxisName.Y, direction=Direction.LR),
116+
Axis(name=AxisName.Z, direction=Direction.DU),
101117
],
102118
)
103119
```
120+
121+
The device is now defined in the instrument coordinate system, but at a physically impossible location overlapping the mouse. Now draw a second picture, which is the actual location of your monitor relative to the mouse. In our case, lets assume that this monitor is facing the mouse's right eye, at a 45 degree angle to the axis of the mouse's body (i.e. the eye to monitor vector is perpendicular to the monitor surface.), and at a viewing distance of 100 mm.
122+
123+
Now we calculate the transforms needed to correctly position the device in the acquisition coordinate system. First we translate the monitor to the correct position by applying `Translation(translation=70.7, 70.7, 0)`. Then, we rotate *clockwise* around the Z axis by applying `Rotation(angles=[0, 0, -45], angles_unit=AngleUnit.DEG)`.

docs/source/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,4 +80,5 @@ I want to...
8080

8181
general
8282
coordinate_systems
83+
validation
8384
related_standards

docs/source/validation.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Validation
2+
3+
To ensure that your metadata is valid you need to construct the full [Metadata](metadata.md#metadata) object. *Validition* does not guarantee in any way that your metadata is *complete*. Many fields are marked as `Optional[]` because not all situations require them, but they may be expected for your use case. If you have a piece of metadata accessible, it should be reported!
4+
5+
## Instrument and Acquisition
6+
7+
If you are validating your `Instrument` and `Acquisition` on your rig, you may not have access to the other metadata files like the procedures. To allow for partial validation of these files we include an `InstrumentAcquisitionCompatibility` class. Construct it as follows:
8+
9+
```{python}
10+
from aind_data_schema.utils.compatibility_check import InstrumentAcquisitionCompatibility
11+
12+
# Construct your Instrument and Acquisition objects
13+
14+
compatibility_check = InstrumentAcquisitionCompatibility(instrument, acquisition)
15+
compatibility_check.run_compatibility_check(raise_for_missing_devices=True)
16+
```
17+
18+
The `raise_for_missing_devices` will raise a `ValidationError` if `Acquisition.active_devices` can't be found in the instrument. Note that if your situation includes implanted devices in the procedures, then errors will be raised because the procedures are not available. In that case, you should construct the a full `Metadata` object for validation.

0 commit comments

Comments
 (0)