-
Notifications
You must be signed in to change notification settings - Fork 71
Open
Description
Steps to reproduce
import airbyte as ab
source = ab.get_source(
"example-test",
config={"hello": "world"},
source_manifest=Path(os.getcwd()) / "mymanifest.yaml",
streams="*"
)
source.check()
Effects
The YAML is built with {{ config['hello'] }} as an empty string.
Root cause
PyAirbyte/airbyte/_executors/declarative.py
Line 125 in 7eb746b
| source_entrypoint = AirbyteEntrypoint(self.declarative_source) |
self.declarative_source is accessing self._config_dict that is always empty, apart from components_py. Since the arg parsing is performed later in the execute() before calling the source_entrypoint.run(args)
def declarative_source(self) -> ConcurrentDeclarativeSource:
"""Get the declarative source object.
Notes:
1. Since Sep 2025, the declarative source class used is `ConcurrentDeclarativeSource`.
2. The `ConcurrentDeclarativeSource` object sometimes doesn't want to be read from twice,
likely due to threads being already shut down after a successful read.
3. Rather than cache the source object, we recreate it each time we need it, to
avoid any issues with re-using the same object.
"""
return ConcurrentDeclarativeSource(
config=self._config_dict,
source_config=self._manifest_dict,
)
Proposed fix
Promote config_dict in the initialization params of DeclarativeExecutor and use it instead of initializing it as an empty dict.
Metadata
Metadata
Assignees
Labels
No labels