Configure Gunicorn for high-performance (1000-2000 req/sec)

- Added Gunicorn configuration files for all services (4 workers, 2 threads)
- Created WSGI entry points for all services
- Updated Dockerfiles to use Gunicorn instead of Flask dev server
- Added GUNICORN_WORKERS and GUNICORN_THREADS environment variables
- Added performance documentation (PERFORMANCE_CONFIG.md, GUNICORN_HIERARCHY.md)
- Each service now handles 8 concurrent requests (4 workers × 2 threads)
- Target capacity: 1000-2000 requests/second per instance

Author: Kamal Sai Devarapalli

GUNICORN_HIERARCHY.md

@@ -0,0 +1,348 @@
+# Gunicorn Architecture Hierarchy
+## Understanding Workers, Threads, and Connections
+
+## Visual Hierarchy
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                     GUNICORN MASTER PROCESS                      │
+│  (Manages everything, listens on port, handles signals)          │
+└─────────────────────┬───────────────────────────────────────────┘
+                      │
+                      │ Forks/Creates
+                      │
+        ┌─────────────┴─────────────┐
+        │                           │
+   ┌────▼─────┐              ┌─────▼──────┐
+   │ WORKER 1 │              │  WORKER 2  │  ← Separate Processes
+   │ (Process)│              │  (Process) │     Each has own Python
+   └────┬─────┘              └─────┬──────┘     interpreter & GIL
+        │                          │
+        │ Creates Threads          │ Creates Threads
+        │                          │
+   ┌────┴────┐                ┌────┴─────┐
+   │ THREAD 1│                │ THREAD 1 │
+   │ THREAD 2│                │ THREAD 2 │  ← Threads within process
+   └────┬────┘                └────┬─────┘
+        │                          │
+        │ Handles Requests         │ Handles Requests
+        │                          │
+   ┌────┴──────────────┐     ┌────┴──────────────┐
+   │  CONNECTION POOL  │     │  CONNECTION POOL  │
+   │  (Database/Redis) │     │  (Database/Redis) │
+   └───────────────────┘     └───────────────────┘
+```
+
+## Detailed Breakdown
+
+### Level 1: Gunicorn Master Process
+
+**What it does:**
+- Listens on the network port (e.g., 9092)
+- Accepts incoming HTTP connections
+- Distributes connections to workers
+- Manages worker lifecycle (start, restart, kill)
+- Handles signals (SIGTERM, SIGHUP, etc.)
+
+**Properties:**
+- Single process
+- Doesn't handle request processing directly
+- Coordinates all workers
+
+### Level 2: Worker Processes
+
+**What they are:**
+- Separate OS processes (forked from master)
+- Each has its own Python interpreter
+- Each has its own Global Interpreter Lock (GIL)
+- Each has its own memory space
+
+**Configuration:**
+```python
+workers = 4  # Creates 4 worker processes
+```
+
+**Why separate processes:**
+- **True parallelism**: Multiple processes can run on multiple CPU cores
+- **Isolation**: If one worker crashes, others continue
+- **GIL bypass**: Each process has its own GIL, so they can truly run in parallel
+
+**Example with 4 workers:**
+```
+Master Process
+    ├── Worker Process #1 (PID 1001) - Python interpreter #1
+    ├── Worker Process #2 (PID 1002) - Python interpreter #2
+    ├── Worker Process #3 (PID 1003) - Python interpreter #3
+    └── Worker Process #4 (PID 1004) - Python interpreter #4
+```
+
+### Level 3: Threads (within each worker)
+
+**What they are:**
+- Lightweight execution units within a process
+- Share the same memory space as their parent worker
+- Share the same GIL (Global Interpreter Lock)
+
+**Configuration:**
+```python
+threads = 2  # 2 threads per worker
+```
+
+**Why threads:**
+- **I/O concurrency**: While one thread waits for DB/network, another can process
+- **Efficiency**: Threads are lighter than processes
+- **Limited by GIL**: In CPU-bound tasks, only one thread runs at a time
+
+**Example within Worker #1:**
+```
+Worker Process #1
+    ├── Thread #1 (handles request A)
+    └── Thread #2 (handles request B)
+    # Both threads share Worker #1's memory and GIL
+```
+
+**GIL Impact:**
+- Python's GIL allows only one thread to execute Python bytecode at a time
+- However, threads can still be useful for I/O-bound operations (DB queries, network calls)
+- While Thread #1 waits for database response, Thread #2 can handle another request
+
+### Level 4: Connection Pools (per worker)
+
+**What they are:**
+- Pool of reusable connections to external resources
+- Database connections (PostgreSQL)
+- Redis connections
+- HTTP client connections
+
+**Configuration:**
+```python
+POOL_SIZE = 5          # 5 connections in pool
+MAX_OVERFLOW = 5       # 5 additional when pool exhausted
+```
+
+**Why connection pools:**
+- Creating connections is expensive (network overhead, authentication)
+- Reusing connections is much faster
+- Limits number of connections to prevent resource exhaustion
+
+**Example within Worker #1:**
+```
+Worker Process #1
+    ├── Thread #1
+    │   └── Uses Connection Pool #1
+    │       ├── DB Connection 1
+    │       ├── DB Connection 2
+    │       ├── DB Connection 3
+    │       └── Redis Connection
+    └── Thread #2
+        └── Uses Connection Pool #1 (same pool, different connections)
+            ├── DB Connection 4
+            ├── DB Connection 5
+            └── Redis Connection (shared)
+```
+
+## Complete Example: 4 Workers × 2 Threads
+
+```
+Gunicorn Master (Port 9092)
+│
+├── Worker 1 (Process PID 1001)
+│   ├── Thread 1 → Can handle 1 request
+│   │   └── Connection Pool 1
+│   │       ├── DB Conn 1-5
+│   │       └── Redis Conn
+│   └── Thread 2 → Can handle 1 request
+│       └── Connection Pool 1 (shared)
+│           ├── DB Conn 1-5 (reused)
+│           └── Redis Conn (shared)
+│
+├── Worker 2 (Process PID 1002)
+│   ├── Thread 1 → Can handle 1 request
+│   │   └── Connection Pool 2
+│   │       ├── DB Conn 1-5 (separate from Worker 1)
+│   │       └── Redis Conn
+│   └── Thread 2 → Can handle 1 request
+│       └── Connection Pool 2 (shared)
+│
+├── Worker 3 (Process PID 1003)
+│   ├── Thread 1 → Can handle 1 request
+│   └── Thread 2 → Can handle 1 request
+│
+└── Worker 4 (Process PID 1004)
+    ├── Thread 1 → Can handle 1 request
+    └── Thread 2 → Can handle 1 request
+
+Total Concurrent Requests: 4 workers × 2 threads = 8 requests simultaneously
+```
+
+## Request Flow Example
+
+```
+1. HTTP Request arrives at Port 9092
+   ↓
+2. Gunicorn Master accepts connection
+   ↓
+3. Master distributes to Worker 2 (round-robin or least busy)
+   ↓
+4. Worker 2 assigns to Thread 1 (available thread)
+   ↓
+5. Thread 1 processes request:
+   - Gets DB connection from Connection Pool 2
+   - Executes database query
+   - Gets Redis connection from pool
+   - Caches result
+   - Returns response
+   ↓
+6. Thread 1 releases connections back to pool
+   ↓
+7. Thread 1 ready for next request
+```
+
+## Connection Pool Hierarchy
+
+### Database Connection Pool (per worker process)
+
+```
+Worker Process #1
+    └── SQLAlchemy Connection Pool
+        ├── Pool Size: 5 connections
+        ├── Max Overflow: 5 connections
+        └── Total Max: 10 connections
+
+Worker Process #2
+    └── SQLAlchemy Connection Pool (separate)
+        ├── Pool Size: 5 connections
+        ├── Max Overflow: 5 connections
+        └── Total Max: 10 connections
+
+Total across all workers: 4 workers × 10 connections = 40 max DB connections
+```
+
+**Important:**
+- Each worker has its own connection pool
+- Threads within a worker share the pool
+- Connections are reused across requests
+- When pool exhausted, new connections created (up to MAX_OVERFLOW)
+
+### Redis Connection Pool (shared or per worker)
+
+```
+Option 1: Per Worker (current setup)
+Worker 1 → Redis Connection Pool 1
+Worker 2 → Redis Connection Pool 2
+Worker 3 → Redis Connection Pool 3
+Worker 4 → Redis Connection Pool 4
+
+Option 2: Shared (if using connection pooling library)
+All Workers → Single Redis Connection Pool (thread-safe)
+```
+
+## Mathematical Relationship
+
+### Capacity Calculation
+
+```
+Concurrent Requests = Workers × Threads
+
+Example:
+- Workers = 4
+- Threads = 2
+- Concurrent Requests = 4 × 2 = 8
+
+Throughput Calculation:
+Throughput (req/sec) = Concurrent Requests × (1000ms / Average Response Time)
+
+Example:
+- 8 concurrent requests
+- Average response time = 100ms
+- Throughput = 8 × (1000 / 100) = 80 req/sec
+
+But with pipelining and request queuing:
+Actual throughput can be higher (1000-2000 req/sec)
+```
+
+### Connection Pool Sizing
+
+```
+Recommended Pool Size per Worker = (Workers × Threads) / Workers
+
+For 4 workers, 2 threads:
+Pool Size = (4 × 2) / 4 = 2 connections per worker minimum
+
+Better: Pool Size = Threads × 2-3
+Pool Size = 2 × 3 = 6 connections per worker
+
+Total DB Connections = Workers × Pool Size
+Total DB Connections = 4 × 6 = 24 connections (reasonable)
+```
+
+## Key Differences
+
+| Aspect | Workers | Threads | Connections |
+|--------|---------|---------|-------------|
+| **Level** | Process-level | Thread-level | Resource-level |
+| **Isolation** | Separate memory | Shared memory | External resource |
+| **Parallelism** | True (multi-core) | Limited (GIL) | N/A |
+| **Communication** | IPC/message passing | Shared variables | Network protocol |
+| **Failure Impact** | Isolated (others continue) | Affects process | Recoverable |
+| **Memory** | Separate heap | Shared heap | External |
+| **Creation Cost** | High (fork) | Low (lightweight) | Medium (network) |
+
+## Best Practices
+
+### 1. Worker Count
+- **Formula**: (2 × CPU cores) + 1
+- **For I/O-bound**: More workers (4-8)
+- **For CPU-bound**: Fewer workers (2-4)
+- **Your setup**: 4 workers (good for mixed workload)
+
+### 2. Thread Count
+- **For I/O-bound**: 2-4 threads per worker
+- **For CPU-bound**: 1 thread per worker (threads don't help with GIL)
+- **Your setup**: 2 threads (good for database/network I/O)
+
+### 3. Connection Pool Size
+- **Per worker**: Threads × 2-3
+- **Example**: 2 threads × 3 = 6 connections per worker
+- **Total**: Workers × Pool Size
+- **Database limit**: Don't exceed database max_connections
+
+## Real-World Example
+
+**Your Current Configuration:**
+```
+Gunicorn Master
+├── Worker 1 (handles requests 1, 9, 17, ...)
+│   ├── Thread 1 (handles requests 1, 5, 9, ...)
+│   └── Thread 2 (handles requests 2, 6, 10, ...)
+├── Worker 2 (handles requests 2, 10, 18, ...)
+│   ├── Thread 1 (handles requests 3, 7, 11, ...)
+│   └── Thread 2 (handles requests 4, 8, 12, ...)
+├── Worker 3
+└── Worker 4
+
+At any moment:
+- 8 requests can be processed simultaneously
+- Each worker has its own connection pool
+- Threads share connection pool within worker
+- Connections are reused across requests
+```
+
+## Summary
+
+**Hierarchy Order:**
+1. **Gunicorn Master** - Coordinates everything
+2. **Workers** (Processes) - Handle request processing (4 workers)
+3. **Threads** (within workers) - Enable concurrency (2 threads each)
+4. **Connection Pools** (per worker) - Reuse expensive connections
+
+**Key Takeaway:**
+- **Workers** = True parallelism (different CPU cores)
+- **Threads** = I/O concurrency (waiting for DB/network)
+- **Connections** = Resource efficiency (reuse expensive connections)
+
+**Your Setup:**
+- 4 workers × 2 threads = 8 concurrent requests
+- Each worker maintains its own DB/Redis connection pool
+- Can handle 1000-2000 req/sec with proper caching and optimization
+