Skip to content

Data contracts: Define what each parameter should look like & check for correctness at runtime #32

@JackKelly

Description

@JackKelly

For each NWP parameter (temperature, wind speed, etc.) we could define the range of correct values (for example temperature should be between -100°C and +60°C, and should never be NaN). These "data contracts" could be specified in YAML. And hypergrib could check for correctness as the data is loaded. The user could select whether "bad" data should be silently converted to NaN; or if hypergrib should fail noisily when it finds bad data (failing noisily should probably be the default).

Perhaps we could define two "levels" of data contracts: At the base level, we define contracts for all NWP params, independent of the NWP provider. But we could also allow for these specs to be modified / overridden by contracts specifics to an NWP provide.

Questions

  • Should hypergrib do the validation? This is quite a general problem. So perhaps this should be a separate project? Maybe in Python land? Maybe using Pydantic?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions