|
2 | 2 |
|
3 | 3 | This package contains Pydantic models for validating Airbyte connector `metadata.yaml` files. |
4 | 4 |
|
5 | | -## Overview |
6 | | - |
7 | | -The models are automatically generated from JSON Schema YAML files maintained in the [airbytehq/airbyte](https://github.com/airbytehq/airbyte) repository at: |
8 | | -``` |
9 | | -airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src/ |
10 | | -``` |
11 | | - |
12 | | -During the CDK build process (`poetry run poe build`), these schemas are downloaded from GitHub and used to generate Pydantic models via `datamodel-code-generator`. All models are generated into a single Python file for simplicity and easier imports. |
13 | | - |
14 | 5 | ## Usage |
15 | 6 |
|
16 | | -### Validating a metadata.yaml file |
17 | | - |
18 | 7 | ```python |
19 | | -from pathlib import Path |
20 | | -import yaml |
21 | 8 | from airbyte_cdk.test.models import ConnectorMetadataDefinitionV0 |
| 9 | +import yaml |
22 | 10 |
|
23 | | -# Load metadata.yaml |
24 | | -metadata_path = Path("path/to/metadata.yaml") |
25 | | -metadata_dict = yaml.safe_load(metadata_path.read_text()) |
26 | | - |
27 | | -# Validate using Pydantic |
28 | | -try: |
29 | | - metadata = ConnectorMetadataDefinitionV0(**metadata_dict) |
30 | | - print("✓ Metadata is valid!") |
31 | | -except Exception as e: |
32 | | - print(f"✗ Validation failed: {e}") |
33 | | -``` |
34 | | - |
35 | | -### Accessing metadata fields |
36 | | - |
37 | | -```python |
38 | | -from airbyte_cdk.test.models import ConnectorMetadataDefinitionV0 |
39 | | - |
40 | | -metadata = ConnectorMetadataDefinitionV0(**metadata_dict) |
41 | | - |
42 | | -# Access fields with full type safety |
43 | | -print(f"Connector: {metadata.data.name}") |
44 | | -print(f"Docker repository: {metadata.data.dockerRepository}") |
45 | | -print(f"Docker image tag: {metadata.data.dockerImageTag}") |
46 | | -print(f"Support level: {metadata.data.supportLevel}") |
47 | | -``` |
48 | | - |
49 | | -### Accessing other models |
50 | | - |
51 | | -All generated models are available in the `generated.models` module: |
52 | | - |
53 | | -```python |
54 | | -from airbyte_cdk.test.models.connector_metadata.generated.models import ( |
55 | | - ConnectorBreakingChanges, |
56 | | - ConnectorReleases, |
57 | | - ReleaseStage, |
58 | | - SupportLevel, |
59 | | -) |
| 11 | +metadata = ConnectorMetadataDefinitionV0(**yaml.safe_load(metadata_yaml)) |
60 | 12 | ``` |
61 | 13 |
|
62 | | -### Available models |
63 | | - |
64 | | -The main model is `ConnectorMetadataDefinitionV0`, which includes nested models for: |
65 | | - |
66 | | -- `ConnectorType` - Source or destination |
67 | | -- `ConnectorSubtype` - API, database, file, etc. |
68 | | -- `SupportLevel` - Community, certified, etc. |
69 | | -- `ReleaseStage` - Alpha, beta, generally_available |
70 | | -- `ConnectorBreakingChanges` - Breaking change definitions |
71 | | -- `ConnectorReleases` - Release information |
72 | | -- `AllowedHosts` - Network access configuration |
73 | | -- And many more... |
74 | | - |
75 | 14 | ## Regenerating Models |
76 | 15 |
|
77 | | -Models are regenerated automatically when you run: |
78 | | - |
79 | | -```bash |
80 | | -poetry run poe build |
81 | | -``` |
82 | | - |
83 | | -This command: |
84 | | -1. Downloads the latest schema YAML files from the airbyte repository |
85 | | -2. Generates all Pydantic models into a single file using `datamodel-code-generator` |
86 | | -3. Generates a consolidated JSON schema file for external validation tools |
87 | | -4. Outputs to `airbyte_cdk/test/models/connector_metadata/generated/`: |
88 | | - - `models.py` - All Pydantic models in a single file |
89 | | - - `metadata_schema.json` - Consolidated JSON schema |
90 | | - |
91 | | -## Schema Source |
92 | | - |
93 | | -The authoritative schemas are maintained in the [airbyte monorepo](https://github.com/airbytehq/airbyte/tree/master/airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src). |
94 | | - |
95 | | -Any changes to metadata validation should be made there, and will be automatically picked up by the CDK build process. |
| 16 | +See the [Contributing Guide](../../../docs/CONTRIBUTING.md#regenerating-connector-metadata-models) for information on regenerating these models. |
0 commit comments