Feature Weighting II

[kkappler]

Future Work

1. The processing configuration template that uses weights stored in aurora should be automatically generated from the processing configuration template that does not use weights, together with the sample weights block in mt_metadata. It can be written, but there are two issues

• inconsistency in formats (both valid, but one uses pure nesting, and the other uses dot-extensions, for exmaple:

"channel_nomenclature": {
            "ex": "ex",
            "ey": "ey",
            "hx": "hx",
            "hy": "hy",
            "hz": "hz"
        },

versus

"channel_nomenclature.ex": "ex",
"channel_nomenclature.ey": "ey",
"channel_nomenclature.hx": "hx",
"channel_nomenclature.hy": "hy",
"channel_nomenclature.hz": "hz",

• The automatically output version does not express all the details, -- this should wait until after the mt_metadata pydantic upgrade.

2. Add capability (with tests and examples) to pack features into the mth5 before weights calculation (currently only compute on the fly is supported)
3. Add tests and examples for FCCoherence feature, allowing calculation of coherence directly from FCs.

• It will be an xarray with dims time, and frequency.
• The frequency axis will come from mth5.processing.spectre.frequency_band_helpers, using ( log_spaced_frequencies and bands_of_constant_q to build broad coherence bands.
• The metadata for specifying this is in mt_metadata/features/standards/fc_coherence.json
• In either case, we need to write the parameters that define the bands, and the feature name into the processing_config_template

4. aurora's xarray_helpers module contains a handle_nan function, and transfer_function_helpers has a remove_nan. These can be consolidated into a single method.
5. Investigate best way to extract the weights for band:
   This may involve finding the nearest frequency bin to the band center period and then applying the weights for that bin, or some tapered region around it. For now, we will just use the mean of the weights for the band. This is a temporary solution and should be replaced with a more robust method.
   The code to examine is the TODO in process_transfer_functions_with_weights in transfer-function_helpers.py module.
6. Now that we have feature_weights in config, the edf_weights parameters should also be added to the processing config and documented ( a cleanup of the statistical notes in edf_weights.py would also be appropriate here).
7. A flow diagram of how the weights enter the processing can be added to the docs.