Skip to content

Deployments and scheduling#209

Draft
caparker wants to merge 7 commits intomainfrom
deployments-and-scheduling
Draft

Deployments and scheduling#209
caparker wants to merge 7 commits intomainfrom
deployments-and-scheduling

Conversation

@caparker
Copy link
Copy Markdown
Collaborator

@caparker caparker commented Feb 8, 2026

  • New fetcher schema: Core tables for handlers, fetcher_clients, adapters, deployments, and deployment_adapters
  • Cron-based scheduling: Custom domain type with validation for cron expressions, enabling pg_cron to trigger
    queue_deployments() every minute
  • Queue-based execution: Uses public.fetchlogs as both job queue and audit log with unique keys for idempotency
  • Runtime configuration: All scheduling and deployment settings stored in database tables, enabling changes without
    application redeployment
  • Temporal offset support: Accommodates data sources that publish with delay

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant