Airbyte Terraform Problem Creating a Connection

[marcosmarxm]

Hi all, I'm making all the configurations programmatically. I was able to do it for the source and destination, but I'm having issues creating the connection.
To test it out, I used the PokeAPI as source & Bigquery as destination. Also I was able to create the connection manually through the UI, and it syncs just fine.
I'm using Airbyte OSS deployed on GCP using compute engine. The 8006 port is exposed.
Here are the 3 options that i used to create the connection via code:
Airbyte Terraform Provider
code:

resource "airbyte_connection" "pokeapi_bigquery_connection" {
  name = "Poke API Connection"
  source_id = airbyte_source_pokeapi.my_source_pokeapi.source_id
  destination_id = airbyte_destination_bigquery.my_destination_bigquery.destination_id
  configurations = {
    streams = [
      {
        name      = "pokemon"
        sync_mode = "full_refresh_overwrite"
      }
    ]
  }
  schedule = {
    schedule_type = "manual"
  }
  namespace_definition = "custom_format"
  status = "inactive"
} 

response:

│ Error: failure to invoke API
│ 
│   with airbyte_connection.pokeapi_bigquery_connection,
│   on main.tf line 50, in resource "airbyte_connection" "pokeapi_bigquery_connection":
│   50: resource "airbyte_connection" "pokeapi_bigquery_connection" {
│ 
│ error sending request: Post "http://<server_url>:8006/v1/connections": context deadline
│ exceeded (Client.Timeout exceeded while awaiting headers)

2. POST Request API
   code:

import requests
from requests.auth import HTTPBasicAuth

url = "http://<server_url>:8006/v1/connections/create"

payload = {
    "name": "PokeAPI-to-Bigquery",
    "sourceId": "<source_id>",
    "destinationId": "<destination_id>",
    "workspaceId": "<workspace_id>",
    "status": "active",
    "schedule": { "scheduleType": "manual" },
    "configurations": {
        'streams': [
            {
                'name': 'pokemon',
                'syncMode': 'full_refresh_overwrite'
            }
        ],
    },
    "dataResidency": "auto",
    "namespaceDefinition": "destination",
    "namespaceFormat": None,
    "nonBreakingSchemaUpdatesBehavior": "ignore",
}

headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.post(
    url,
    json=payload,
    headers=headers,
    auth=HTTPBasicAuth('<username>', '<password>')
) 

response:

{"type":"about:blank","status":405}

3. airbyte-api-python-sdk
   code:

import airbyte
from airbyte.models import shared

s = airbyte.Airbyte(
    server_url='http://<server_url>:8006/v1/connections',
    security=shared.Security(
        basic_auth=shared.SchemeBasicAuth(
            password="<password>",
            username="<username>",
        ),
    ),
)

req = shared.ConnectionCreateRequest(
    configurations=shared.StreamConfigurations(
        streams=[
            shared.StreamConfiguration(
                name='pokemon',
                sync_mode=shared.ConnectionSyncModeEnum.FULL_REFRESH_OVERWRITE,
            ),
        ],
    ),
    data_residency=shared.GeographyEnum.AUTO,
    destination_id='<destination_id>',
    name='PokeApi-to-BigQuery',
    namespace_definition=shared.NamespaceDefinitionEnum.SOURCE,
    namespace_format='destination',
    non_breaking_schema_updates_behavior=shared.NonBreakingSchemaUpdatesBehaviorEnum.PROPAGATE_FULLY,
    # prefix='ipsam',
    schedule=shared.ConnectionSchedule(
        cron_expression='0 0/30 * * * ?',
        schedule_type=shared.ScheduleTypeEnum.CRON,
    ),
    source_id='<source_id>',
    status=shared.ConnectionStatusEnum.ACTIVE,
)

res = s.connections.create_connection(req)

response:

SDKError: API error occurred: Status 405
{"type":"about:blank","status":405}

Could someone help to see what I've been missing?
Thanks,
Nico

Original Slack Thread

[marcosmarxm]

Last update from @JonsSpaghetti

Update: We think we’ve isolated the problem and have a fix that is currently in testing. Once that’s been validated in our test environment, we’ll release to both Cloud and OSS. I’ll post here to let you know so that you can give it a try and let us know if it resolves the timeout issues.

[abkebir]

Hello @marcosmarxm, will be glad if you can share how you configure the source and destination because when I am programmatically creating them , i face d 405 error message. Thank you.