feat(models,training)!: multi dataset integration#594
Merged
Conversation
mchantry
changed the title
This was referenced Oct 9, 2025
JPXKQX
reviewed
Oct 21, 2025
JPXKQX
reviewed
Oct 21, 2025
Contributor
|
Hi @mchantry, I'm planning on doing some training with this branch. Could you let me know what I should expect to work and not work? |
MeraX
reviewed
Nov 11, 2025
radiradev
reviewed
Nov 11, 2025
radiradev
reviewed
Nov 12, 2025
radiradev
reviewed
Nov 12, 2025
for more information, see https://pre-commit.ci
pass edges and edge attributes to mapper and processor
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
anaprietonem
previously approved these changes
Jan 21, 2026
Contributor
anaprietonem
left a comment
anaprietonem
left a comment
There was a problem hiding this comment.
Thanks for addressing my previous comments, LGTM!
github-project-automation
bot
moved this from Now In Progress
to For merging
in Anemoi-dev
gmertes
added a commit
to ecmwf/anemoi-inference
that referenced
this pull request
Jan 21, 2026
… `data` (#403) ## Description Implements step 1 (checkpoint support) and part of step 1.5 (new metadata) of #402 Add support for the new multi-dataset checkpoints, but **only** the single-dataset case where the dataset name defaults to "data". Real multi dataset support will follow later in step 2. This PR should be merged before ecmwf/anemoi-core#594 , but does not need to be merged exactly at the same time. This PR is forwards and backwards compatible. I started using some of the new metadata but not all, to keep things simple. Improvements to the metadata will also be deferred to the next round. ***As a contributor to the Anemoi framework, please ensure that your changes include unit tests, updates to any affected dependencies and documentation, and have been tested in a parallel setting (i.e., with multiple GPUs). As a reviewer, you are also responsible for verifying these aspects and requesting changes if they are not adequately addressed. For guidelines about those please refer to https://anemoi.readthedocs.io/en/latest/*** By opening this pull request, I affirm that all authors agree to the [Contributor License Agreement.](https://github.com/ecmwf/codex/blob/main/Legal/contributor_license_agreement.md)
anaprietonem
approved these changes
Jan 21, 2026
HCookie
approved these changes
Jan 21, 2026
Merged
HCookie
changed the title
HCookie
changed the title
OpheliaMiralles
pushed a commit
to OpheliaMiralles/anemoi-core
that referenced
this pull request
Feb 16, 2026
Update the metadata we store in checkpoints in order to provide more information to inference The checkpoint mentioned [here](ecmwf#594 (comment)) was created based on this branch. ***As a contributor to the Anemoi framework, please ensure that your changes include unit tests, updates to any affected dependencies and documentation, and have been tested in a parallel setting (i.e., with multiple GPUs). As a reviewer, you are also responsible for verifying these aspects and requesting changes if they are not adequately addressed. For guidelines about those please refer to https://anemoi.readthedocs.io/en/latest/*** By opening this pull request, I affirm that all authors agree to the [Contributor License Agreement.](https://github.com/ecmwf/codex/blob/main/Legal/contributor_license_agreement.md) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BEGIN_COMMIT_OVERRIDE
feat(models,training)!: multi dataset integration
END_COMMIT_OVERRIDE
Description
Supports multiple time-aligned datasets as inputs and outputs for training.
e.g.
where inputs/outputs each use their own encoder/decoder.
As a contributor to the Anemoi framework, please ensure that your changes include unit tests, updates to any affected dependencies and documentation, and have been tested in a parallel setting (i.e., with multiple GPUs). As a reviewer, you are also responsible for verifying these aspects and requesting changes if they are not adequately addressed. For guidelines about those please refer to https://anemoi.readthedocs.io/en/latest/
By opening this pull request, I affirm that all authors agree to the Contributor License Agreement.
📚 Documentation preview 📚: https://anemoi-training--594.org.readthedocs.build/en/594/
📚 Documentation preview 📚: https://anemoi-graphs--594.org.readthedocs.build/en/594/
📚 Documentation preview 📚: https://anemoi-models--594.org.readthedocs.build/en/594/