Skip to content

Conversation

@TommyNationPol1984
Copy link

@TommyNationPol1984 TommyNationPol1984 commented Jan 7, 2026

Summary
Implement production-ready webhook endpoints and handlers for GitHub, Deepgram, and MyVocal. Handlers must verify signatures, deduplicate deliveries, enqueue work for asynchronous processing, and expose observability for deliveries and failures.

Scope

  • Add endpoints:
    • POST /webhooks/github
    • POST /webhooks/deepgram
    • POST /webhooks/myvocal
  • Signature verification (HMAC or provider token) for each provider.
  • Idempotency/deduplication using delivery IDs (persisted in Redis or DB).
  • Quick accept response (202 Accepted) and enqueue heavy work to job queue (SQS/RabbitMQ/Redis).
  • Minimal payload storage: raw payloads encrypted at rest for a configurable retention window.
  • Monitoring: metrics for delivery count, verification failures, processing latency, and retry rate.
  • Tests: unit tests for verification logic, integration tests for end‑to‑end signed deliveries, and CI smoke test for streaming synth trigger.
  • Docs: update /docs/webhooks.md with endpoint specs, expected headers, example payloads, and secret rotation guidance.

Acceptance criteria

  • Each endpoint verifies provider signatures and rejects invalid requests with 401.
  • Duplicate deliveries are detected and ignored (return 202 with duplicate status).
  • Handlers respond within 1s for typical payloads and enqueue processing jobs.
  • Enqueued jobs include delivery metadata and are processed by a worker that records outcome.
  • Raw payloads are stored encrypted and automatically purged after the retention window.
  • Automated tests cover signature verification, deduplication, and enqueueing logic.
  • /docs/webhooks.md contains example curl and test scripts for generating signed payloads.

Implementation notes

  • Use body raw bytes for HMAC verification; do not rely on parsed JSON for signature checks.
  • Store provider secrets in the ops secret manager; support secret rotation by allowing multiple active secrets.
  • Use Redis (or equivalent) for short‑term dedupe keys: webhook:delivery:<id> with 24h TTL.
  • Enqueue lightweight job messages; perform heavy tasks (transcription storage, model triggers) asynchronously.
  • Expose a simple deliveries dashboard (recent deliveries, verification result, processing status).
  • Default to application/json content type and enforce POST only.

Testing checklist

  • Unit tests for HMAC verification functions (Express and FastAPI variants).
  • Integration test that posts a signed payload and asserts 202 Accepted.
  • Test for duplicate delivery idempotency.
  • End‑to‑end test that enqueues a job and worker processes it successfully.
  • Security test verifying raw payloads are encrypted at rest.

Suggested labels
feature, backend, security, webhooks, needs-review

Suggested assignees and milestone

  • Assignee: backend lead or @backend-team
  • Milestone: Webhook Integration (or MVP UI & Consent per roadmap)

@TommyNationPol1984
Copy link
Author

@TommyNationPol1984
Copy link
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant