Skip to content

Latest commit

 

History

History
172 lines (126 loc) · 6.53 KB

source.md

File metadata and controls

172 lines (126 loc) · 6.53 KB
page_title subcategory description
airbyte_source - terraform-provider-airbyte
Manages an Airbyte source connector. This is the generic resource for all source types.

airbyte_source (Resource)

Manages an Airbyte source connector.

This is the generic source resource that works with any Airbyte source connector type. Pass the connector's definition_id and a JSON configuration blob. Use the airbyte_connector_configuration data source to resolve connector names to definition IDs and to get clean Terraform diffs on non-sensitive values.

~> Migrating from typed resources? If you are upgrading from a pre-1.0 provider version or moving from a typed resource like airbyte_source_postgres, see the Migrating to 1.0 guide for step-by-step instructions using Terraform's moved block.

Example Usage

Using the airbyte_connector_configuration data source (recommended)

Use the airbyte_connector_configuration data source to resolve the connector's definition_id automatically, validate configuration at plan time, and separate sensitive from non-sensitive values for clean diffs:

data "airbyte_connector_configuration" "postgres" {
  connector_name    = "source-postgres"
  connector_version = "3.6.28"

  configuration = {
    host     = "db.example.com"
    port     = 5432
    database = "mydb"
    schemas  = ["public"]
    ssl_mode = { mode = "prefer" }
  }

  configuration_secrets = {
    username = var.pg_user
    password = var.pg_password
  }
}

resource "airbyte_source" "postgres" {
  name          = "Postgres Production"
  workspace_id  = var.workspace_id
  definition_id = data.airbyte_connector_configuration.postgres.definition_id
  configuration = data.airbyte_connector_configuration.postgres.configuration_json
}

Using inline JSON configuration

For simpler cases where you don't need the data source, pass JSON configuration directly. The entire configuration attribute is sensitive, so all values are hidden in plan output:

resource "airbyte_source" "github" {
  name          = "GitHub"
  workspace_id  = var.workspace_id
  definition_id = "ef69ef6e-aa7f-4af1-a01d-ef775033524e"

  configuration = jsonencode({
    repositories = ["airbytehq/airbyte"]
    credentials = {
      option_title          = "PAT Credentials"
      personal_access_token = var.github_pat
    }
  })
}

Migrating from a typed resource

Typed source resources (airbyte_source_postgres, airbyte_source_github, etc.) are replaced by this generic airbyte_source resource in provider 1.0+. Use a moved block (Terraform >= 1.8) for a zero-downtime migration:

moved {
  from = airbyte_source_pardot.my_source
  to   = airbyte_source.my_source
}

resource "airbyte_source" "my_source" {
  name          = "Pardot"
  workspace_id  = var.workspace_id
  definition_id = "5e6175e5-68e1-4c17-bff9-56103bbb0d80"

  configuration = jsonencode({
    client_id               = var.pardot_client_id
    client_secret           = var.pardot_client_secret
    refresh_token           = var.pardot_refresh_token
    pardot_business_unit_id = "0Uv5g0000008OT2CAM"
  })
}

For full details including alternative methods for older Terraform versions, see the Migrating to 1.0 guide.

Schema

Required

  • configuration (String) The values required to configure the source. The schema for this must match the schema return by source_definition_specifications/get for the source. Parsed as JSON.
  • name (String) Name of the source e.g. dev-mysql-instance.
  • workspace_id (String) Requires replacement if changed.

Optional

  • definition_id (String) The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
  • resource_allocation (Attributes) actor or actor definition specific resource requirements. if default is set, these are the requirements that should be set for ALL jobs run for this actor definition. it is overriden by the job type specific configurations. if not set, the platform will use defaults. these values will be overriden by configuration at the connection level. (see below for nested schema)
  • secret_id (String) Optional secretID obtained through the OAuth redirect flow. Requires replacement if changed.

Read-Only

  • created_at (Number)
  • source_id (String)
  • source_type (String)

Nested Schema for resource_allocation

Optional:

Nested Schema for resource_allocation.default

Optional:

  • cpu_limit (String)
  • cpu_request (String)
  • ephemeral_storage_limit (String)
  • ephemeral_storage_request (String)
  • memory_limit (String)
  • memory_request (String)

Nested Schema for resource_allocation.job_specific

Optional:

  • job_type (String) enum that describes the different types of jobs that the platform runs. Not Null; must be one of ["get_spec", "check_connection", "discover_schema", "sync", "reset_connection", "connection_updater", "replicate"]
  • resource_requirements (Attributes) optional resource requirements to run workers (blank for unbounded allocations). Not Null (see below for nested schema)

Nested Schema for resource_allocation.job_specific.resource_requirements

Optional:

  • cpu_limit (String)
  • cpu_request (String)
  • ephemeral_storage_limit (String)
  • ephemeral_storage_request (String)
  • memory_limit (String)
  • memory_request (String)

Import

Import is supported using the following syntax:

terraform import airbyte_source.my_airbyte_source "..."

In Terraform v1.5.0 and later, the import block can be used:

import {
  to = airbyte_source.my_airbyte_source
  id = "..."
}