GrievanceFlow

GrievanceFlow is a production-oriented, AWS-native grievance intelligence platform designed for high-scale public-service workflows. It supports multilingual voice/text intake, AI-powered triage, semantic de-duplication, policy-grounded auto-resolution, human-in-the-loop adjudication, hotspot analytics, and enterprise security controls.

---

1) What This Project Solves

Traditional grievance systems are highly manual and reactive. GrievanceFlow adds an intelligence layer that:

• accepts citizen complaints from WhatsApp and web channels;
• converts vernacular voice/text into structured grievance signals;
• identifies duplicates and clusters related incidents;
• auto-resolves low-risk informational cases with citation-backed responses;
• escalates high-risk/complex cases to officers with pre-drafted context;
• provides real-time analytics for operations and policy interventions.

---

2) High-Level Architecture

flowchart LR
  Citizen[Citizen WhatsApp/PWA] --> APIGW[API Gateway]
  APIGW --> Intake[Lambda intake-api]
  Intake --> SQS[SQS workflow queue]
  SQS --> Worker[Lambda pipeline-worker]
  Worker --> AI[Lambda ai-orchestrator]
  Worker --> Dedup[Lambda dedup-engine]
  Worker --> RAG[Lambda rag-resolver]
  Worker --> Workflow[Lambda workflow-api]
  Workflow --> DDB[(DynamoDB grievances)]
  AI --> AWSAI[Transcribe + Translate + Comprehend]
  RAG --> Bedrock[Bedrock KB + Model]
  Workflow --> External[Lambda external-integration]
  External --> CPGRAMS[CPGRAMS/State adapters]
  Analytics[Lambda analytics-api] --> DDB
  Dashboard[React Admin Dashboard] --> APIGW

Loading

---

3) Repository Structure

• infra/terraform/ - all AWS infrastructure as code.
• services/intake-api/ - webhook/PWA intake entrypoint.
• services/ai-orchestrator/ - multilingual NLP orchestration + RAG resolver.
• services/dedup-engine/ - similarity matching and cluster decisioning.
• services/workflow-api/ - ticket lifecycle, SLA, officer actions.
• services/analytics-pipeline/ - hotspot detection and operational aggregations.
• services/pipeline-worker/ - queue consumer orchestration across AI/workflow services.
• services/external-integration/ - downstream portal sync and fallback export.
• services/common/ - shared runtime helpers (for example AWS SDK bootstrap).
• apps/admin-dashboard/ - React frontend.
• policy/conftest/ - policy-as-code guardrails.
• scripts/ - deployment and verification scripts.
• docs/ - ADRs, runbooks, governance, rollout docs.

Key detailed docs:

• docs/ARCHITECTURE.md - C4-style architecture and runtime sequence diagrams.
• docs/RUNBOOK.md - on-call operations and incident procedures.
• README_HACKATHON.md - 48-hour low-cost build and demo playbook.

---

4) Core Runtime Flow

1. Citizen submits grievance via WhatsApp webhook or PWA intake.
2. intake-api validates signature/idempotency and enqueues a workflow message.
3. pipeline-worker consumes queue messages.
4. ai-orchestrator performs translation, sentiment/entity extraction, categorization, urgency scoring.
5. dedup-engine checks semantic similarity and cluster linkage.
6. rag-resolver attempts citation-backed Tier-1 resolution where eligible.
7. workflow-api persists/update tickets and handles SLA-tracked state transitions.
8. external-integration syncs resolved/escalated cases to target systems.
9. analytics-api supports trend/hotspot views for operations dashboard.

---

5) Security Model

GrievanceFlow follows zero-trust and defense-in-depth principles:

• Least privilege IAM: dedicated Lambda role per service in infra/terraform/iam_lambda.tf.
• Secrets management: webhook signing secrets in AWS Secrets Manager.
• Encryption:
  • KMS-backed encryption for data stores where configured.
  • encrypted CloudWatch log groups for Lambda/API logs.
• Perimeter controls:
  • AWS WAF managed rule set + IP rate limiting.
  • API throttling at stage/default route settings.
• Admin route auth:
  • JWT authorizer support.
  • jwt_mandatory_mode hard-fails Terraform if JWT config is required but missing.

---

6) Compliance, Audit, and Threat Detection

Implemented infrastructure controls:

• AWS CloudTrail (multi-region, validation enabled).
• AWS Config recorder + delivery channel.
• GuardDuty detector.
• Security Hub enablement.
• IAM Access Analyzer.
• account-level S3 public access blocking.
• dedicated security logs bucket with lifecycle controls.

---

7) Reliability, DR, and Resilience

• SQS workflow queue + DLQ redrive policy.
• Lambda tracing via X-Ray.
• CloudWatch alarms:
  • API 5xx
  • intake Lambda errors
  • DLQ message depth
• DynamoDB PITR + TTL support.
• Multi-region DR scaffolding:
  • secondary provider alias
  • optional DynamoDB replica region
  • S3 replication scaffolding
  • DR region variable controls

---

8) Observability and SIEM Hooks

• Centralized logging via CloudWatch log groups (encrypted).
• Optional OpenTelemetry hooks:
  • otel_enabled
  • otel_layer_arn
  • otel_collector_endpoint
• Optional SIEM export path:
  • CloudWatch log subscription filters
  • Kinesis Firehose delivery stream
  • export into security log bucket / SIEM pipeline

---

9) CI/CD and Supply-Chain Security

CI

• Python lint/test workflow in .github/workflows/ci.yml.
• Terraform init -backend=false + validate.
• Conftest policy checks on workflow definitions.

Security CI

• .github/workflows/security.yml:
  • CodeQL
  • tfsec
  • pip-audit
  • Trivy scans

Dependency governance

• .github/dependabot.yml for weekly updates across actions/pip/npm.

---

10) Local Development

Prerequisites

• Python 3.11+
• Node.js 20+
• Terraform 1.7+
• AWS CLI v2

Setup

python -m venv .venv
.\.venv\Scripts\python -m pip install -r requirements-dev.txt

Verify AWS SDK and credentials

./scripts/verify-aws-sdk.ps1

If needed:

aws configure sso

Run quality checks

.\.venv\Scripts\python -m ruff check .
.\.venv\Scripts\python -m pytest -q

Run frontend locally

cd apps/admin-dashboard
npm install
npm run dev

---

11) Deployment Guide

A) Preflight

./scripts/preflight-check.ps1

B) Configure environment values

Update infra/terraform/envs/<env>.tfvars with real values:

• webhook_secret
• bedrock_kb_id
• bedrock_model_arn
• cpgrams_endpoint (optional)
• admin_jwt_issuer + admin_jwt_audience (mandatory when jwt_mandatory_mode = true)
• alarm_email
• DR/OTEL/SIEM toggles as required

C) Deploy infrastructure + frontend in one command

./scripts/deploy-all.ps1 -Environment dev

D) Or deploy in stages

./scripts/deploy-dev.ps1
./scripts/deploy-frontend.ps1

---

12) Environment Strategy

• dev - rapid iteration, permissive defaults.
• staging - production-like verification, JWT mandatory recommended.
• prod - strict controls, DR enabled, SIEM export enabled, JWT mandatory.

---

13) Operational Smoke Tests

After deployment:

• POST /pwa/intake returns 202.
• POST /webhooks/whatsapp returns 202 for valid payload/signature.
• GET /queue?status=OPEN returns 200.
• GET /analytics/hotspots returns 200.
• Dashboard URL from Terraform output is reachable over HTTPS.

---

14) Configuration Reference (Key Terraform Variables)

• jwt_mandatory_mode - enforce JWT config presence.
• admin_jwt_issuer, admin_jwt_audience - admin auth config.
• dr_enabled, dr_region - disaster recovery scaffolding.
• otel_enabled, otel_layer_arn, otel_collector_endpoint - telemetry hooks.
• siem_export_enabled - SIEM export pipeline toggle.
• api_throttle_rate_limit, api_throttle_burst_limit - API throttling.

---

15) Troubleshooting

• Terraform validation fails with JWT check:
  • set admin_jwt_issuer and admin_jwt_audience, or disable jwt_mandatory_mode.
• Webhook requests rejected:
  • verify webhook_secret and signature header mapping.
• No dashboard data:
  • verify VITE_API_BASE, API route health, and auth mode.
• No SIEM export data:
  • ensure siem_export_enabled=true and Firehose/log subscription resources are created.
• AWS SDK/auth failures:
  • run aws sts get-caller-identity and refresh SSO credentials.

---

16) Project Principles

1. Serverless-first architecture.
2. Human-in-the-loop for high-risk outcomes.
3. Grounded AI responses with citations.
4. Security and compliance by design.
5. Observable and auditable operations.
6. Configuration-driven hardening for enterprise deployments.

---

17) GitHub Publishing Checklist

Before pushing to GitHub:

1. Initialize git repository and set remote.
2. Confirm .gitignore is in place (prevents .venv, node_modules, dist, Terraform state, and local env files).
3. Ensure no real secrets are present in tracked files:
   • infra/terraform/envs/*.tfvars
   • .env*
   • scripts with inline tokens
4. Run local checks:
   • .\.venv\Scripts\python -m ruff check .
   • .\.venv\Scripts\python -m pytest -q
   • npm --prefix apps/admin-dashboard run build
5. Push and verify GitHub Actions:
   • CI (.github/workflows/ci.yml)
   • Security (.github/workflows/security.yml)
   • Secret scan (.github/workflows/secret-scan.yml)

Repository governance files included:

• SECURITY.md
• CONTRIBUTING.md
• .gitattributes
• .gitignore