Skip to content

Add escalations support#39

Merged
leonore merged 3 commits intomasterfrom
leo/add-escalations-support-in-singer-tap
Jun 20, 2025
Merged

Add escalations support#39
leonore merged 3 commits intomasterfrom
leo/add-escalations-support-in-singer-tap

Conversation

@leonore
Copy link
Contributor

@leonore leonore commented Jun 19, 2025

This PR adds Singer tap support for the new Escalations API, allowing users to export escalation data to their data warehouses.

Changes
Regenerated client code from the latest OpenAPI spec to include Escalations V2 endpoints

Added model definitions for escalation data structures:

  • model/escalation_v2.go - Main escalation model with fields for id, title, status, creator, priority, events, and related alerts/incidents
  • model/escalation_creator_v2.go - Handles the creator field which can be a user, alert, or workflow (reuses existing UserV2 model)
  • model/escalation_priority_v2.go - Simple priority model with name field
  • model/escalation_event_v2.go - Event model tracking notification history with channels and users
  • model/chat_channel_slim_v2.go - Models Slack and Microsoft Teams channel information
  • model/alert_actor_v2.go - Models alerts that can create escalations
  • model/workflow_actor_v2.go - Models workflows that can create escalations
  • model/alert_slim_v2.go - Models related alerts in escalations
  • model/incident_slim_v2.go - Models related incidents in escalations

🌊 New Stream

  • tap/stream_escalations.go - Implements the escalations stream with:
    • Stream name: escalations
    • Primary key: id
    • Full table replication
    • Pagination support (50 records per page)

📚 Documentation

  • Updated docs/README.md with escalations table information
  • Enhanced development.md with instructions for updating the API client when the incident.io API changes

🔑 Environment Variable Support

  • Modified cmd/tap-incident/cmd/app.go to support INCIDENT_API_KEY and INCIDENT_ENDPOINT environment variables
  • Config file is now optional if environment variables are set
  • Updated development.md with environment variable usage instructions

Your own testing

1. Verify the stream appears in discovery mode:
  ./bin/tap-incident --discover | jq '.streams[] | select(.stream == "escalations") | .stream'

2.   # Create a catalog with only escalations selected
  ./bin/tap-incident --discover | jq '
    .streams |= map(
      if .stream == "escalations" then
        .metadata[0].metadata.selected = true
      else
        .metadata[0].metadata.selected = false
      end
    )
  ' > escalations-only-catalog.json

3.  # Extract only escalations data
  ./bin/tap-incident --catalog=escalations-only-catalog.json | grep '"type":"RECORD"'

"urgency": Optional(Property{
Types: []string{"string"},
}),
"channels": Optional(ArrayOf(ChatChannelSlimV2.Schema())),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean channels goes in a separate table?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it shouldn't (see explanation below). Based on this, maybe we'd want events to be a separate table, but it'd require adding a reference to the escalation ID.

  In Singer taps:
  - Only top-level streams become separate tables (e.g., escalations, incidents, users)
  - Nested objects and arrays within a stream are stored as JSON/structured data within that same table

  In this case:
  - escalations is the stream/table
  - events is an array field within the escalations table
  - channels is a nested array within each event object

  So the data structure in your data warehouse would look like:
  escalations table:
  - id
  - title
  - status
  - events (JSON array containing:
      - id
      - event
      - occurred_at
      - channels (nested JSON array)
      - users (nested JSON array)
    )
  - ... other fields

  Most data warehouses (Snowflake, BigQuery, Redshift) handle nested JSON well, so this structure allows you to query channels using JSON path expressions without needing
  separate tables and joins.

@leonore leonore merged commit 1e0521d into master Jun 20, 2025
1 check passed
@leonore leonore deleted the leo/add-escalations-support-in-singer-tap branch June 20, 2025 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments