Skip to content

Sync mode selection should validate against stream's supported_sync_modes #917

@devin-ai-integration

Description

@devin-ai-integration

Summary

When building the configured catalog, the _get_sync_mode helper function unconditionally falls back to SyncMode.incremental, even for streams that only support full_refresh. This can cause connector failures for streams that don't support incremental sync.

Background

This issue was identified during code review of PR #899 (force_full_refresh fix). The current behavior predates that PR, so it's not a regression - but it's a valid edge case that should be addressed.

Requested by: Aaron ("AJ") Steers (@aaronsteers) (AJ Steers)

Current Behavior

def _get_sync_mode(stream: AirbyteStream) -> SyncMode:
    supported_modes = getattr(stream, "supported_sync_modes", None)

    if force_full_refresh:
        if supported_modes and SyncMode.full_refresh in supported_modes:
            return SyncMode.full_refresh
        return SyncMode.incremental  # BUG: stream might not support incremental

    return SyncMode.incremental  # BUG: stream might only support full_refresh

Expected Behavior

The selected sync mode should always be validated against supported_sync_modes when that attribute is present:

def _get_sync_mode(stream: AirbyteStream) -> SyncMode:
    supported_modes = getattr(stream, "supported_sync_modes", None)

    if force_full_refresh:
        if supported_modes and SyncMode.full_refresh in supported_modes:
            return SyncMode.full_refresh
        if supported_modes and SyncMode.incremental in supported_modes:
            return SyncMode.incremental
        return SyncMode.full_refresh  # fallback for unknown/empty

    # Default: prefer incremental if supported, otherwise full_refresh
    if supported_modes and SyncMode.incremental in supported_modes:
        return SyncMode.incremental
    if supported_modes and SyncMode.full_refresh in supported_modes:
        return SyncMode.full_refresh
    return SyncMode.incremental  # preserve backwards compat for unknown

Affected Code

  • airbyte/sources/base.py - _get_sync_mode helper function (around line 449)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions