Mappings between ImageSequence and AnalogSignal objects?

[rgutzen]

I think the new ImageSequence object, implemented by @teogale, is a great addition to Neo! So, I'm wondering how to best make use of this kind of object.

On the one hand, each pixel in the ImageSequence is in principle also an analog signal, and in some scenarios (having Elephant functions in mind) it would be helpful to represent them accordingly.
On the other hand, many analog signals are recorded with some kind of electrode array, and thus have a spatial aspect that is not really represented in the AnalogSignal object. In these cases, it would be helpful to be able to represent them in a spatially arranged way like the ImageSequence.
So for some applications with a temporal focus data would be better represented as an AnalogSignal (e.g. for spectral analysis, filtering, spike-coupling, ...), while the same data for another application with a spatial focus is better represented as ImageSequence (e.g. for ROI selection, spatially tracking a wavefront, plotting a movie, ...).
Thus ideally, there would be a unique mapping between the two object types.
I guess such a mapping would be rather straight-forward with mainly two things to consider:

1. Where to store the spatial information with the AnalogSignal? (as annotations with a specific Namespace, or as linked Channel objects, ...)
2. How to store pixel-wise annotations with the ImageSequence object?

Would such a mapping be in spirit of Neo?

[samuelgarcia]

Hi Robin.

You are right. The spatial information is the big missing piece of neo.

For multi electrode array case, storing geomtery in AnalogSignal.array_annotations is OK but not standarized at API scale. Note that for mea, the geometry is not necessary a grid of pixel but can have many exotic layout (even in 3D). So mapping can be diffulult to an "image"

For pixel-wise annotation we need to extend array_annoations in 2D (or more) for that case I think.

[apdavison]

The RegionOfInterest class and its subclasses, together with the ImageSequence.signal_from_region() method, are intended to provide such a mapping.

@rgutzen I guess for your use case, you would need:

1. a PixelRegionOfInterest or PixelArrayRegionOfInterest class
2. a flag for signal_from_region() to allow returning a multi-channel AnalogSignal rather than an average over pixels
3. support in NixIO for storing RegionOfInterest objects

[rgutzen]

The points you list would definitely be very helpful. Regarding that,
2. could there be a relation link indicating that within a neo segment an AnalogSignal was created from an ImageSequence?
3. when the geometry information are stored in RegionOfInterest could there also be a function to transform an AnalogSignal to an ImageSequence object?

[apdavison]

2. needs some thought. Could use annotations, or the new Group class.
3. that could perhaps be a method of PixelArrayRegionOfInterest, it doesn't really make sense for other ROI types, which will not in general cover the image field.

[apdavison]

I think there's quite a lot of material here to cover, so I'm going to postpone until the next milestone.