Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions airbyte_cdk/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,16 @@
- `airbyte_cdk.models.airbyte_protocol`
- `airbyte_cdk.models.airbyte_protocol_serializers`

## Using the CLI (`airbyte_cdk.cli`)

The Airbyte CDK provides two command-line interfaces (CLIs) for interacting with the framework.

- `airbyte-cdk`: This is the main CLI for the Airbyte CDK. It provides commands for building
and testing connectors, as well as other utilities. See the `airbyte_cdk.cli.airbyte_cdk` module
for more details.
- `source-declarative-manifest`: This command allows you to run declarative manifests directly.
See the `airbyte_cdk.cli.source_declarative_manifest` module for more details.

---

API Reference
Expand Down
65 changes: 65 additions & 0 deletions airbyte_cdk/cli/source_declarative_manifest/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Source Declarative Manifest CLI

The source-declarative-manifest CLI is included in the airbyte-cdk package.
This CLI enables connector interfaces to be run locally on manifest-only connectors,
much like we already do with Python connectors.

## Installation

The airbyte-cdk library can be installed globally using pipx:

```bash
pipx install airbyte-cdk
```

If you are using a cloned airbyte-python-cdk repo locally,
you can also create a virtual environment to enable the CLI.
From the root directory of airbyte-python-cdk:

```bash
python -m venv .venv
source .venv/bin/activate
pip install -e .
```

## Usage

### Options

--help: displays the list of available commands

### Commands

- spec: Outputs the JSON configuration specification. NOTE: This currently just outputs the base source-declarative-manifest spec
- check: Runs a connection_check to verify a connection can be made with the passed config
- discover: Outputs a catalog describing the source's schema
- read: Reads the source using the passed config and catalog, and outputs messages to STDOUT

### Command options

- --config: The relative path to the config to inject into SDM.
- --catalog: The relative path to the configured catalog.
- --state: The relative path to the state object to pass. Only used when running an incremental read.
- --manifest-path: The relative path to the local YAML manifest to inject into SDM.
- --components-path: The relative path to the custom components to mount, if they exist.

| Option | spec | check | discover | read |
| ------------------- | ---- | -------- | -------- | -------- |
| `--config` | ❌ | required | required | required |
| `--catalog` | ❌ | ❌ | required | required |
| `--state` | ❌ | ❌ | ❌ | optional |
| `--manifest-path` | ❌ | required | required | required |
| `--components-path` | ❌ | optional | optional | optional |

### Examples

Here are some basic examples of how to run source-declarative-manifest commands locally.
Note that the paths are relative. These examples assume the user is currently at the root level of a connector dir:

```bash
source-declarative-manifest check --config secrets/config.json --manifest-path manifest.yaml
```

```bash
source-declarative-manifest read --config secrets/config.json --catalog integration_tests/configured_catalog.json --manifest-path manifest.yaml --components-path components.py
```
5 changes: 5 additions & 0 deletions airbyte_cdk/cli/source_declarative_manifest/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
"""
.. include:: ./README.md
:start-line: 2
"""

from airbyte_cdk.cli.source_declarative_manifest._run import run

__all__ = [
Expand Down
4 changes: 4 additions & 0 deletions airbyte_cdk/cli/source_declarative_manifest/_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -299,5 +299,9 @@ def _register_components_from_file(filepath: str) -> None:


def run() -> None:
"""Run the `source-declarative-manifest` CLI.

Args are detected from the command line, and the appropriate command is executed.
"""
args: list[str] = sys.argv[1:]
handle_command(args)
Loading