Utilize wgrib2 index if available #443
Description
Is your feature request related to a problem? Please describe.
Hello,
I'm looking to gauge interest in leveraging wgrib2 index files if they are present. Large GRIB2 files suffer from very slow opening times as the library scans the entire file. While GRIB2 isn't a particularly good file format, it is ubiquitous. I'm sure there is nuance to the different bands and attributes that can be heterogeneous throughout the file. Is there an opportunity to kick-start the metadata reading operations by jumping through the file using those index offsets? This wouldn't have to be something that's enabled by default.
Describe the solution you'd like
When opening a file, I can provide a path to a wgrib2 index file, ideally improving the opening time from cfgrib.
Describe alternatives you've considered
Using cfgrib's index is fine once the file has been opened, but I have concerns on the stability of the pickled format. The spec for that doesn't seem locked down, the hash in the index implies that cfgrib version changes would change the format of that file. I'd like to index grib2 files in a large archive for a long time, so I need a standard "spec" of sorts to lean on.
Additional context
Just looking to gauge interest and get thoughts written down. I searched the repo and didn't find much. Thanks a lot for your contributions to this project, and your consideration.
Organisation
No response