Observations feature branch

[mchantry]

Planned medium-term feature branch to enable the development of observation dataset training.

Key components:

• Call structure/inference.
• First sources.
• Dataset creation from sources.
• Dataset loading.

As a contributor to the Anemoi framework, please ensure that your changes include unit tests, updates to any affected dependencies and documentation, and have been tested in a parallel setting (i.e., with multiple GPUs). As a reviewer, you are also responsible for verifying these aspects and requesting changes if they are not adequately addressed. For guidelines about those please refer to https://anemoi.readthedocs.io/en/latest/

By opening this pull request, I affirm that all authors agree to the Contributor License Agreement.

[radiradev]

Hi @b8raoult, as I mentioned during the MLPP meeting I was wondering if tensordict would be a good alternative to dicts of torch.Tensor. afaik tensordict is a dict of tensors but with support for tensor operations, so it might make downstream applications easier to implement.

[anaprietonem]

Suggested change

	"refinement_level_c": input_refinement_level_c,
	"refinement_level_c": None,

From @MeraX

---

Thanks for fixing this inconsistency. I think in general there is no need to set refinement_level_c for the icon grid_definition as the grid must always correspond to the grib data. Thus I would make the following suggestion:

If I understood the earthkit team correctly, they are working on supporting ICON grids. I.e. it will be possible to ask EKD for the grid definition, making the icon grid_definition in anemoi-datatests unnecessary.