Skip to content

add test data for each ilamb or iomb metric#31

Merged
lewisjared merged 5 commits intomainfrom
ilamb-iomb-data
May 9, 2025
Merged

add test data for each ilamb or iomb metric#31
lewisjared merged 5 commits intomainfrom
ilamb-iomb-data

Conversation

@nocollier
Copy link
Copy Markdown
Contributor

@nocollier nocollier commented Apr 29, 2025

Description

Checklist

Please confirm that this pull request has done the following:

  • Data registry up to date (regenerate if necessary with a comment on this PR of /regenerate)
  • Documentation added (where applicable)
  • Changelog item added to changelog/

@nocollier
Copy link
Copy Markdown
Contributor Author

/regenerate

@github-actions
Copy link
Copy Markdown

PR comment handling
Something went wrong!

Details: https://github.com/Climate-REF/ref-sample-data/actions/runs/14736729906

@nocollier
Copy link
Copy Markdown
Contributor Author

@lewisjared The decimation on the variable msftmz failed because its dimensions are time, basin, lev, lat. The portion I am using as test data (2000-2014) is not too big. I am not sure how to decimate it. We could, instead of raising a ValueError if there isn't both lat and lon, just not decimate. Any preference?

@lewisjared
Copy link
Copy Markdown
Contributor

I'm happy with the approach of not decimating this particular dataset

@nocollier
Copy link
Copy Markdown
Contributor Author

@lewisjared It is not clear to me how to declare a dataset that I intend to not decimate. I see the boolean provided to the command create_sample_data, but this seems to be a constant for all the datasets in DATASETS_TO_FETCH.

My naive approach would be to:

  1. Remove that option from create_sample_data and set decimate=True in the current loop over DATASETS_TO_FETCH.
  2. Create a new constant DATASETS_NOT_DECIMATED with just my msftmz variable in a CMIP6Request.
  3. Write a second loop in create_sample_data over this new list and set decimate=False

Am I missing something? Another approach would be to add a flag to each request to decimate or not, default to True, and then explicitly flag my request as false.

@lewisjared
Copy link
Copy Markdown
Contributor

That functionality hasn't been added. The intended use-case for the CLI argument was to produce a directory with the same datasets, but not decimated for local testing. Ignoring the decimation at a request level would require some rework.

Honestly for this particular grid, I would add update CMIP6Request.decimate_dataset to support the dimensions and just return the dataset unchanged. That is the lightest touch approach to this.

@nocollier
Copy link
Copy Markdown
Contributor Author

/regenerate

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 9, 2025

PR comment handling
Something went wrong!

Details: https://github.com/Climate-REF/ref-sample-data/actions/runs/14927895928

@nocollier
Copy link
Copy Markdown
Contributor Author

/regenerate

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 9, 2025

PR comment handling
The regenerate task is done!

You can find the workflow here:
https://github.com/Climate-REF/ref-sample-data/actions/runs/14928085755

@nocollier nocollier requested a review from lewisjared May 9, 2025 11:56
@lewisjared lewisjared merged commit 7c4a29a into main May 9, 2025
3 checks passed
@lewisjared lewisjared deleted the ilamb-iomb-data branch May 9, 2025 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants