Skip to content

Conversation

@bouweandela
Copy link
Member

@bouweandela bouweandela commented Oct 9, 2025

Description

Record attributes in provenance via the interface provided by esmvalcore.local.LocalFile instead of directly accessing the file from the provenance module. This is needed to facilitate loading other data than NetCDF files while still maintaining informative provenance records.

This is a first step towards supporting different data sources in #2765.


Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.


To help with the number pull requests:

@bouweandela bouweandela changed the title Record input cube instead of NetCDF file attributes in provenance Record attributes in provenance via the LocalFile interface instead of directly accessing the file Oct 9, 2025
@codecov
Copy link

codecov bot commented Oct 9, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.46%. Comparing base (4b3e8b0) to head (104efd7).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2854      +/-   ##
==========================================
- Coverage   95.46%   95.46%   -0.01%     
==========================================
  Files         260      260              
  Lines       15526    15519       -7     
==========================================
- Hits        14822    14815       -7     
  Misses        704      704              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bouweandela bouweandela force-pushed the use-cube-attrs-for-provenance branch from cef36b0 to 104efd7 Compare October 9, 2025 15:43
@bouweandela

This comment was marked as outdated.

@bouweandela

This comment was marked as outdated.

@bouweandela

This comment was marked as outdated.

@bouweandela bouweandela marked this pull request as ready for review October 10, 2025 07:14
Copy link
Contributor

@valeriupredoi valeriupredoi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good and useful indeed! Things may get hairy when we access eg Zarr attributes etc, but we'll bridge that bridge when we get there; just one (dumb) question about having to reload the data in local.py buds 🍺

iris.cube.CubeList
The loaded data.
"""
cubes = _load_from_file(self, ignore_warnings=ignore_warnings)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where does this sit inside the stack - as in, can we not pass already loaded cube(lists) sitting in memory, do we have to load separately? It's just that iris I/O is costly

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The preprocessor function load calls this method instead of acessing the files itself. That makes it possible to introduce more classes like LocalFile for different data sources in #2765 without further modifications of the existing code.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok - am not 100% sold on that but let's see the final structure when we use it via #2765 - definitely not the place to complain here 😃

@valeriupredoi
Copy link
Contributor

you should leave your introspective comments and not obsolete them out - those were interesting notes!

@bouweandela
Copy link
Member Author

Thanks for reviewing @valeriupredoi!

@bouweandela bouweandela merged commit 0f6489a into main Oct 13, 2025
4 checks passed
@bouweandela bouweandela deleted the use-cube-attrs-for-provenance branch October 13, 2025 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants