Destination connector fails with 'Check failed' error

[Necmttn]

Description

When attempting to use the SurrealDB destination connector with a Notion source connector, the check operation fails with exit code 1. The connector appears to be built and launched successfully, but the actual connection check fails.

Error Details

Initial error:

2025-06-30 14:34:16 platform INFO Connector exited, processing output
2025-06-30 14:34:16 platform INFO Output file jobOutput.json found
2025-06-30 14:34:16 platform INFO Connector exited with exit code 1
2025-06-30 14:34:16 platform INFO Reading messages from protocol version 0.2.0
2025-06-30 14:34:16 platform ERROR Check failed
2025-06-30 14:34:16 platform INFO Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, successivePartialFailureLimit=1000, totalCompleteFailureLimit=10, totalPartialFailureLimit=20, successiveCompleteFailures=1, successivePartialFailures=0, totalCompleteFailures=1, totalPartialFailures=0)

Specific Error Identified

The error has been identified as a KeyError when accessing the 'type' field in the schema properties. Here's the detailed traceback:

2025-06-30 14:34:47 destination INFO Removing table for overwrite: users
2025-06-30 14:34:47 destination ERROR 'type'
Traceback (most recent call last):
  File "/airbyte/integration_code/main.py", line 10, in <module>
    DestinationSurrealDB().run(sys.argv[1:])
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/destinations/destination.py", line 153, in run
    for message in output_messages:
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/destinations/destination.py", line 143, in run_cmd
    yield from self._run_write(
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/destinations/destination.py", line 66, in _run_write
    yield from self.write(
  File "/airbyte/integration_code/destination_surrealdb/destination.py", line 166, in write
    tpe = props["type"]
KeyError: 'type'

Root Cause

The error occurs in destination.py at line 166 where the code attempts to access props["type"] directly without checking if the "type" key exists in the properties dictionary. This happens when processing the stream schema during the write operation.

Impact

This prevents the connector from working with any streams where the JSON schema properties don't have an explicit "type" field, which can occur with:

• Union types using anyOf, oneOf, or allOf
• Referenced schemas using ``
• Properties that inherit type from parent schemas

The Notion connector appears to use schemas that don't always include explicit "type" fields for all properties, which triggers this error.

Suggested Fix

Replace the direct dictionary access with defensive programming:

# Instead of:
tpe = props["type"]

# Use:
tpe = props.get("type", "string")  # Default to string if type is not specified

Or implement a more robust type extraction function that handles complex JSON schema structures.

Steps to Reproduce

1. Set up a Notion source connector
2. Configure the SurrealDB destination connector
3. Attempt to sync data from Notion to SurrealDB
4. The sync fails during the write operation with a KeyError

Expected Behavior

The connector should handle schemas from various sources (including Notion) that may not have explicit "type" fields for all properties.

Environment

• Airbyte platform version: (from logs) workload-init-container:1.7.1
• SurrealDB connector version: Latest from repository
• Source connector: Notion

Additional Context

The connector is being launched in a Kubernetes environment. The workload appears to be created successfully but fails during the actual connection check phase when processing the Notion schema.