Skip to content

Commit 3de3af0

Browse files
refactor: Generate metadata models as single file with JSON schema output
- Modified build script to generate all models into a single models.py file - Added consolidated JSON schema generation (metadata_schema.json) - Updated imports to reference generated.models module - Removed 30+ individual model files in favor of single-file approach - Updated README to document new structure and outputs - Added GitHub token support to avoid rate limiting Co-Authored-By: AJ Steers <[email protected]>
1 parent 62902f6 commit 3de3af0

35 files changed

+197
-3245
lines changed

airbyte_cdk/sources/declarative/models/declarative_component_schema.py

Lines changed: 86 additions & 58 deletions
Large diffs are not rendered by default.

airbyte_cdk/test/models/connector_metadata/README.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ The models are automatically generated from JSON Schema YAML files maintained in
99
airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src/
1010
```
1111

12-
During the CDK build process (`poetry run poe build`), these schemas are downloaded from GitHub and used to generate Pydantic models via `datamodel-code-generator`.
12+
During the CDK build process (`poetry run poe build`), these schemas are downloaded from GitHub and used to generate Pydantic models via `datamodel-code-generator`. All models are generated into a single Python file for simplicity and easier imports.
1313

1414
## Usage
1515

@@ -48,10 +48,10 @@ print(f"Support level: {metadata.data.supportLevel}")
4848

4949
### Accessing other models
5050

51-
All generated models are available in the `generated` submodule:
51+
All generated models are available in the `generated.models` module:
5252

5353
```python
54-
from airbyte_cdk.test.models.connector_metadata.generated import (
54+
from airbyte_cdk.test.models.connector_metadata.generated.models import (
5555
ConnectorBreakingChanges,
5656
ConnectorReleases,
5757
ReleaseStage,
@@ -82,8 +82,11 @@ poetry run poe build
8282

8383
This command:
8484
1. Downloads the latest schema YAML files from the airbyte repository
85-
2. Generates Pydantic models using `datamodel-code-generator`
86-
3. Outputs models to `airbyte_cdk/test/models/connector_metadata/generated/`
85+
2. Generates all Pydantic models into a single file using `datamodel-code-generator`
86+
3. Generates a consolidated JSON schema file for external validation tools
87+
4. Outputs to `airbyte_cdk/test/models/connector_metadata/generated/`:
88+
- `models.py` - All Pydantic models in a single file
89+
- `metadata_schema.json` - Consolidated JSON schema
8790

8891
## Schema Source
8992

airbyte_cdk/test/models/connector_metadata/__init__.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
1-
from .generated.ConnectorMetadataDefinitionV0 import ConnectorMetadataDefinitionV0
2-
from .generated.ConnectorTestSuiteOptions import ConnectorTestSuiteOptions
1+
from .generated.models import ConnectorMetadataDefinitionV0, ConnectorTestSuiteOptions
32

43
__all__ = [
54
"ConnectorMetadataDefinitionV0",

airbyte_cdk/test/models/connector_metadata/generated/ActorDefinitionResourceRequirements.py

Lines changed: 0 additions & 48 deletions
This file was deleted.

airbyte_cdk/test/models/connector_metadata/generated/AirbyteInternal.py

Lines changed: 0 additions & 39 deletions
This file was deleted.

airbyte_cdk/test/models/connector_metadata/generated/AllowedHosts.py

Lines changed: 0 additions & 18 deletions
This file was deleted.

airbyte_cdk/test/models/connector_metadata/generated/ConnectorBreakingChanges.py

Lines changed: 0 additions & 70 deletions
This file was deleted.

airbyte_cdk/test/models/connector_metadata/generated/ConnectorBuildOptions.py

Lines changed: 0 additions & 15 deletions
This file was deleted.

airbyte_cdk/test/models/connector_metadata/generated/ConnectorIPCOptions.py

Lines changed: 0 additions & 36 deletions
This file was deleted.

0 commit comments

Comments
 (0)