Problems with using Stream2 interface with more than 1 threshold and not decompressing data in the driver

[MarkRivers]

This issue is a follow-on to #76. That issue got off track several times due to incorrect configuration of the driver and plugins.

The Stream2 interface in the driver currently does not work correctly when the number of thresholds is greater than 1 and when the driver does not decompress the data. The reason is that the multiple arrays received over the Stream2 interface are individually compressed, but the driver is putting these into a 3-D array. The codecs in NDPluginCodec and HDF5 cannot decode these.

There are additional problems even when the driver does decompress the data. The PVA plugin receives a 3-D array, but the ImageJ and other viewers cannot really handle 3-D arrays, so the user cannot control which threshold is displayed. Similarly the STATS plugin produces the statistics on the entire 3-D array, while the user may want to see the statistics for each threshold individually. This could be arranged with an ROI plugin, but a simpler solution is desirable.

I propose to change the driver to do callbacks using the asyn address for a detector configured to collect N thresholds as follows:

• Address 0 sends the 2-D NDArrays for all enabled thresholds in order from 1 to N.
• Address J sends the 2-D NDArray for the J'th enabled threshold (J=1-N).
• 2 new NDAttributes are created for each NDArray.
  • ThresholdName is an NDAttrString attribute containing the name of the threshold as reported by the Stream2 interface. These are "threshold_1", "threshold_2", etc.
  • ThresholdEnergy is an NDAttrFloat64 attribute containing the energy of the threshold as reported by the Stream2 interface in units of eV.

Note that the first enabled threshold might be "threshold_2" if "threshold_1" is disabled.

If the HDF5 plugin receives its data from the detector using address 0 then is will receive NDArrays for all thresholds in order. For example if 2 thresholds are enabled then first NDArray will be threshold 1, the second will be threshold 2, the third threshold1, etc. The ThresholdName and ThresholdEnergy attributes for each 2-D array in the dataset can be used to identify which threshold it is. Clients that read the HDF5 file can select the desired threshold data by selecting every other array, for example, if 2 thresholds are enabled.

Plugins can use asyn address 0 to receive all NDArrays, address 1 to receive only the first enabled threshold, address 2 to receive only the second enabled threshold, etc. This allows the PVA plugin, for example, to be switched between displaying just threshold 1 or just threshold 2.

This scheme works both when the driver decompresses the data, and when it does not decompress and compressed data is passed directly to the plugins. In the latter case the PVA, HDF5, and Codec plugins are capable of receiving the compressed data directly. Other plugins must be configured to get their data from the Codec plugin. The address of the Codec plugin can be configured to select all thresholds (0), or just a single threshold (1, 2 ...).