Skip to content

DeclarativeExecutor passing down empty configs #868

@adileo

Description

@adileo

Steps to reproduce

import airbyte as ab
source = ab.get_source(
    "example-test",
    config={"hello": "world"},
    source_manifest=Path(os.getcwd()) / "mymanifest.yaml",
    streams="*"
)
source.check()

Effects

The YAML is built with {{ config['hello'] }} as an empty string.

Root cause

source_entrypoint = AirbyteEntrypoint(self.declarative_source)

self.declarative_source is accessing self._config_dict that is always empty, apart from components_py. Since the arg parsing is performed later in the execute() before calling the source_entrypoint.run(args)

    def declarative_source(self) -> ConcurrentDeclarativeSource:
        """Get the declarative source object.

        Notes:
        1. Since Sep 2025, the declarative source class used is `ConcurrentDeclarativeSource`.
        2. The `ConcurrentDeclarativeSource` object sometimes doesn't want to be read from twice,
           likely due to threads being already shut down after a successful read.
        3. Rather than cache the source object, we recreate it each time we need it, to
           avoid any issues with re-using the same object.
        """
        return ConcurrentDeclarativeSource(
            config=self._config_dict,
            source_config=self._manifest_dict,
        )

Proposed fix

Promote config_dict in the initialization params of DeclarativeExecutor and use it instead of initializing it as an empty dict.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions