For each NWP parameter (temperature, wind speed, etc.) we could define the range of correct values (for example temperature should be between -100°C and +60°C, and should never be NaN). These "data contracts" could be specified in YAML. And hypergrib could check for correctness as the data is loaded. The user could select whether "bad" data should be silently converted to NaN; or if hypergrib should fail noisily when it finds bad data (failing noisily should probably be the default).
Perhaps we could define two "levels" of data contracts: At the base level, we define contracts for all NWP params, independent of the NWP provider. But we could also allow for these specs to be modified / overridden by contracts specifics to an NWP provide.
Questions
- Should
hypergrib do the validation? This is quite a general problem. So perhaps this should be a separate project? Maybe in Python land? Maybe using Pydantic?