-
Couldn't load subscription status.
- Fork 43
Record attributes in provenance via the LocalFile interface instead of directly accessing the file #2854
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…reading them from disk directly
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2854 +/- ##
==========================================
- Coverage 95.46% 95.46% -0.01%
==========================================
Files 260 260
Lines 15526 15519 -7
==========================================
- Hits 14822 14815 -7
Misses 704 704 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
cef36b0 to
104efd7
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good and useful indeed! Things may get hairy when we access eg Zarr attributes etc, but we'll bridge that bridge when we get there; just one (dumb) question about having to reload the data in local.py buds 🍺
| iris.cube.CubeList | ||
| The loaded data. | ||
| """ | ||
| cubes = _load_from_file(self, ignore_warnings=ignore_warnings) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where does this sit inside the stack - as in, can we not pass already loaded cube(lists) sitting in memory, do we have to load separately? It's just that iris I/O is costly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The preprocessor function load calls this method instead of acessing the files itself. That makes it possible to introduce more classes like LocalFile for different data sources in #2765 without further modifications of the existing code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok - am not 100% sold on that but let's see the final structure when we use it via #2765 - definitely not the place to complain here 😃
|
you should leave your introspective comments and not obsolete them out - those were interesting notes! |
|
Thanks for reviewing @valeriupredoi! |
Description
Record attributes in provenance via the interface provided by
esmvalcore.local.LocalFileinstead of directly accessing the file from the provenance module. This is needed to facilitate loading other data than NetCDF files while still maintaining informative provenance records.This is a first step towards supporting different data sources in #2765.
Before you get started
Checklist
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.
To help with the number pull requests: