Skip to content

feat: Add scheduling capabilities for cloud connections #843

@devin-ai-integration

Description

@devin-ai-integration

Feature Request: Add Scheduling Capabilities for Cloud Connections

Requested by

@aaronsteers (AJ Steers)

Context

Currently, PyAirbyte's Cloud capabilities allow creating and managing connections, but there's no interface for setting or updating sync schedules. This feature request proposes adding scheduling capabilities that align with Airbyte's API primitives.

Proposed Interface

Based on investigation of the Airbyte API, the scheduling system uses:

  • schedule_type: Either CRON or MANUAL
  • cron_expression: A standard cron string when using CRON schedule type

This is compatible with a simple cron string interface. The proposed interface would:

  1. During Connection Creation (CloudWorkspace.deploy_connection()):

    • Add optional schedule parameter accepting either:
      • None (default): Manual sync only
      • A cron expression string (e.g., "0 */6 * * *" for every 6 hours)
    • Internally converts cron string to AirbyteAPIConnectionSchedule object
  2. For Existing Connections (CloudConnection class):

    • Add set_schedule(cron_expression: str | None) method
      • None sets schedule to MANUAL
      • String sets schedule to CRON with the provided expression
    • Add get_schedule() method returning current schedule info
    • These methods are non-destructive (unlike config updates)
  3. MCP Tools:

    • set_cloud_connection_schedule(connection_id, cron_expression) - marked as non-destructive
    • get_cloud_connection_schedule(connection_id) - read-only

Example Usage

import airbyte as ab
from airbyte import cloud

workspace = cloud.CloudWorkspace(...)

# Create connection with schedule
connection = workspace.deploy_connection(
    name="My Connection",
    source=source,
    destination=destination,
    schedule="0 */6 * * *",  # Every 6 hours
)

# Update schedule on existing connection
connection.set_schedule("0 0 * * *")  # Daily at midnight

# Disable automatic syncs
connection.set_schedule(None)  # Manual only

# View current schedule
schedule_info = connection.get_schedule()
print(f"Type: {schedule_info.schedule_type}")
print(f"Cron: {schedule_info.cron_expression}")

Implementation Notes

  • The Airbyte API already supports scheduling via AirbyteAPIConnectionSchedule in PatchConnectionRequest
  • Cron expressions should be validated before sending to API
  • Consider adding helper constants for common schedules (hourly, daily, weekly)
  • Documentation should include cron expression examples and link to cron syntax reference

Related PR

This feature request was created during work on #841, where an initial update_schedule() method was removed to avoid exposing internal API models in the public interface.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions