Skip to content

Commit a3679a3

Browse files
author
Oleksandr Bazarnov
committed
add README.md
1 parent 4085c8e commit a3679a3

File tree

1 file changed

+66
-0
lines changed
  • airbyte_cdk/sources/declarative/migrations/manifest

1 file changed

+66
-0
lines changed
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Manifest Migrations
2+
3+
This directory contains the logic and registry for manifest migrations in the Airbyte CDK. Migrations are used to update or transform manifest components to newer formats or schemas as the CDK evolves.
4+
5+
## Adding a New Migration
6+
7+
1. **Create a Migration File:**
8+
- Add a new Python file in the `migrations/` subdirectory.
9+
- Name the file using the pattern: `<description>_v<major>_<minor>_<patch>__<order>.py`.
10+
- Example: `http_requester_url_base_to_url_v6_45_2__0.py`
11+
- The `<order>` integer is used to determine the order of migrations for the same version.
12+
13+
2. **Define the Migration Class:**
14+
- The migration class must inherit from `ManifestMigration`.
15+
- Name the class using the pattern: `V_<major>_<minor>_<patch>_ManifestMigration_<Description>`.
16+
- Example: `V_6_45_2_ManifestMigration_HttpRequesterUrlBaseToUrl`
17+
- Implement the following methods:
18+
- `should_migrate(self, manifest: ManifestType) -> bool`: Return `True` if the migration should be applied to the given manifest.
19+
- `migrate(self, manifest: ManifestType) -> None`: Perform the migration in-place.
20+
21+
3. **Migration Versioning:**
22+
- The migration version is extracted from the class name and used to determine applicability.
23+
- Only manifests with a version less than or equal to the migration version will be migrated.
24+
25+
4. **Component Type:**
26+
- Use the `TYPE_TAG` constant to check the component type in your migration logic.
27+
28+
5. **Examples:**
29+
- See `migrations/http_requester_url_base_to_url_v6_45_2__0.py` and `migrations/http_requester_path_to_url_v6_45_2__1.py` for reference implementations.
30+
31+
## Migration Registry
32+
33+
- All migration classes in the `migrations/` folder are automatically discovered and registered in `migrations_registry.py`.
34+
- Migrations are applied in order, determined by the `<order>` suffix in the filename.
35+
36+
## Testing
37+
38+
- Ensure your migration is covered by unit tests.
39+
- Tests should verify both `should_migrate` and `migrate` behaviors.
40+
41+
## Example Migration Skeleton
42+
43+
```python
44+
from airbyte_cdk.sources.declarative.migrations.manifest.manifest_migration import TYPE_TAG, ManifestMigration, ManifestType
45+
46+
class V_1_2_3_ManifestMigration_Example(ManifestMigration):
47+
component_type = "ExampleComponent"
48+
original_key = "old_key"
49+
replacement_key = "new_key"
50+
51+
def should_migrate(self, manifest: ManifestType) -> bool:
52+
return manifest[TYPE_TAG] == self.component_type and self.original_key in manifest
53+
54+
def migrate(self, manifest: ManifestType) -> None:
55+
manifest[self.replacement_key] = manifest[self.original_key]
56+
manifest.pop(self.original_key, None)
57+
```
58+
59+
## Additional Notes
60+
61+
- Do not modify the migration registry manually; it will pick up all valid migration classes automatically.
62+
- If you need to skip certain component types, use the `NON_MIGRATABLE_TYPES` list in `manifest_migration.py`.
63+
64+
---
65+
66+
For more details, see the docstrings in `manifest_migration.py` and the examples in the `migrations/` folder.

0 commit comments

Comments
 (0)