Skip to content

Configuration refactor#97

Merged
garciampred merged 9 commits intomainfrom
configuration-refactor
Feb 24, 2026
Merged

Configuration refactor#97
garciampred merged 9 commits intomainfrom
configuration-refactor

Conversation

@garciampred
Copy link
Collaborator

This PR migrates dataset-specific runtime configuration out of cdsobs and into external service definition files from cads-obs-config.

What changed

  • Moved dataset config fields (tile sizes, reader, ingestion DB, available CDM tables, disabled fields, etc.) into ServiceDefinition.
  • Simplified CDSObsConfig:
    • Removed embedded datasets/DatasetConfig.
    • Renamed config path key to cads_obs_config_location.
  • Updated service-definition loading to read cads-obs-config/service-definitions/.yml
  • Refactored ingestion/retrieve/forms/api flows to use ServiceDefinition as the source of truth.
  • Added optional service_definition injection in key API functions (run_ingestion_pipeline, run_make_cdm, get_forms_jsons) for the tests.
  • Updated CI to clone cads-obs-config instead of cads-forms-insitu.
  • Expanded type hints across modules and tightened coverage config (pyproject.toml, Makefile).
  • Updated tests/fixtures to build dataset behavior from service-definition files and removed legacy dataset config test data.
  • Remove setup.cfg, use only pyproject.toml.

@garciampred garciampred merged commit 83e0027 into main Feb 24, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants