Multi-channel, multi-plane TwoPhotonSeries / ImageSeries

[ehennestad]

Is there a way of adding multi-channel, multi-plane TwoPhotonSeries to NWB? Should I make one TwoPhotonSeries per channel? Are there any examples for how to convert a multichannel, multiplane image series?

[CodyCBakerPhD]

Hi there,

This answer has two parts: one for the multi-channel, one for the multi-plane

Multi-channel: Due to the current design of the ImagingPlane object, which includes OpticalChannel metadata, you would currently create one unique ImagingPlane per unique OpticalChannel, and separate TwoPhotonSeries linking to their appropriate ImagingPlane (and by extension, the appropriate color filter)

We realize this seems a bid redundant since both OpticalChannels share the same physical space; stay tuned with the ndx-microscopy extension and subsequent NEP proposal to have this metadata structure improved in the core NWB schema

Multi-plane: this one is less straightforward and has a lot to do with how you usually interpret your own data.

If the grid spacing of each plane overlaps to the point where it can be considered, effectively, one contiguous volume, then you would have a 3-D ImagingPlane and 4-D TwoPhotonSeries with the last axis being concatenation over depths. This is most common for whole-brain imaging such as this dandiset: https://dandiarchive.org/dandiset/000350?search=ahrens&pos=1

Even if the grid spacing overlaps, but you usually analyze each plane separately independent of the volume, then I would create a separate 2-D ImagingPlane per depth and separate 3-D TwoPhotonSeries per plane

If the grid spacing between planes doesn't overlap at all, then again go with separate ImagingPlane objects per plane and separate TwoPhotonSeries for each plane

[ehennestad]

Multi-channel: Due to the current design of the ImagingPlane object, which includes OpticalChannel metadata, you would currently create one unique ImagingPlane per unique OpticalChannel, and separate TwoPhotonSeries linking to their appropriate ImagingPlane (and by extension, the appropriate color filter)

As far as I can see, it is possible to add multiple optical channels per imaging plane. Should I refrain from doing that and instead create something like this:

"imagine_plane1_channel1" with optical channel "channel1"
"imaging_plane1_channel2" with optical channel "channel2"
"imagine_plane2_channel1" with optical channel "channel1"
...

Another question: for the indicator property on the imaging plane, is it ok to add a list of indicators if multiple indicators are used?

[CodyCBakerPhD]

As far as I can see, it is possible to add multiple optical channels per imaging plane.

Sorry for the confusion - while it is possible to add the multiple channels to the ImagingPlane object itself, the main reason for my above suggestion is because then when you make the TwoPhotonSeries that links to the ImagingPlane, it's unable to link or refer to which particular Optic Channel was used by that series (aside from perhaps the choice of name of the series, which is not that explicit of a link)

Should I refrain from doing that and instead create something like this:

"imagine_plane1_channel1" with optical channel "channel1"
"imaging_plane1_channel2" with optical channel "channel2"
"imagine_plane2_channel1" with optical channel "channel1"

That looks good to me

Another question: for the indicator property on the imaging plane, is it ok to add a list of indicators if multiple indicators are used?

Another reason for separating optic channels to different planes; are you saying that a single optical channel was reliant on or utilized more than one indicator?

What indicators were these?

A list would technically not be formally possible (the field is simply a free-text string) but you can certainly separate each indicator with either a comma or semi-colon

[ehennestad]

Sorry for the confusion - while it is possible to add the multiple channels to the ImagingPlane object itself, the main reason for my above suggestion is because then when you make the TwoPhotonSeries that links to the ImagingPlane, it's unable to link or refer to which particular Optic Channel was used by that series (aside from perhaps the choice of name of the series, which is not that explicit of a link)

This relates to a question I am encountering now. I am trying to write code to detect a TwoPhotonSeries within an NWB file and then resolve how many optical channels and planes it has in order to provide it as a 5D image stack object. For the files I have written myself, this is straight forward because I can rely on the naming of each TwoPhotonSeries, but I also want to generalise this, and then it seems a bit more complicated.

Since each plane has an optical channel (and these are embedded, not links), it seems that I have to find all optical channels from all planes, and then compare if these have the same description/emission values in order to resolve whether they are the same channel or not.

Are there a more obvious way to go about this?

Another reason for separating optic channels to different planes; are you saying that a single optical channel was reliant on or utilized more than one indicator?

I would argue that the indicator should not be coupled to an optical channel. True, optical channels should be optimised to capture signal only from the indicator of interest and block out other indicators but this might not always be true (bleedthrough is a common problem). Furthermore, multiple indicators are present in an imaging plane, regardless of the configuration (filters) of the optical channel and whether or not an optical channel is active or not. On the other hand, I understand that the imaging plane is an abstraction and in the ideal case it is coupled to a "perfect" optical channel and thus capturing only one indicator. I don't have a strong opinion either way, I was just curious what is common practice.

PS: I am using matnwb and have not checked how this works in pynwb

[CodyCBakerPhD]

Thanks for the elaborate response!

You should definitely check out the work in progress on the ndx-micrsocopy extension design which is intended to resolve a large number of concerns just like our current discussion

I am trying to write code to detect a TwoPhotonSeries within an NWB file and then resolve how many optical channels and planes it has in order to provide it as a 5D image stack object.

This is no doubt mentioned somewhere on the ndx-microscopy repo but I'll post it here for clarity

A big part of my suggestions throughout our discussion are actually to attempt to move away from a single 5D data representation to a collection of 3D (for single plane over time) or 4D (volume over time, applies only if all imaging planes can be considered to constitute a 'contiguous' volume) representations

A big reason for this is, while a 5D structure certainly can make sense on its own (time x width x height x planes x channels), it allows for ambiguity when a 4D structure is observed (does the fourth dimension represent planes or optic channels?) which can be a source of confusion

Whereas if we instead peel out the optic channels and have one series/data structure per channel, we can now immediately and intuitively determine if that series is of a single plane or a single volume

P.S.: I only use PyNWB but hopefully there are no differences between the two languages

[ehennestad]

You should definitely check out the work in progress on the catalystneuro/ndx-microscopy#4 which is intended to resolve a catalystneuro/ndx-microscopy#1 just like our current discussion

Thanks for reminding me, this looks very interesting, and I look forward to seeing it in action!

A big reason for this is, while a 5D structure certainly can make sense on its own (time x width x height x planes x channels), it allows for ambiguity when a 4D structure is observed (does the fourth dimension represent planes or optic channels?) which can be a source of confusion

I am not sure I agree that this has to be ambiguous. No matter how you define your structure, you also have to specify exactly what each dimension is. The least elegant solution is to allow 2D, 3D, 4D and 5D structures and then for each structure explicitly define what each dimension is. The better solution is to treat everything as 5D structures and then operate with singleton dimensions. I assume this would anyway be done for e.g z-stacks or linescans, in which case either the time or the height dimension would be a singleton.

I don't see why grouping dimensions XYZ should be preferred to grouping dimensions XYC. In practice, I assume the main deciding factor would be how the data should be retrieved. If I am interested in reading all optical channels for a single slice/plane of my data, I imagine (although I have not tested) this would be more computationally costly if I read from multiple datasets which are organised according to TXYZ instead of say a 5D dataset which is organised according to XYCZT.