Distributed Log Aggregation System - Initial demo release Date: 2025-11-16
- Functional demo of ingestion → Kafka → consumer → PostgreSQL → search
- Batched ingestion, Kafka durability, prepared DB statements, trigram index for fast search
- Docker Compose setup for local development
run.shfor quick local demo
docker compose up -d- Initialize DB schema (see README)
chmod +x run.sh && ./run.sh
- No TLS/auth for Kafka/Postgres (dev-only)
- Simple caching in search service
- Single-node Kafka (bitnami) for convenience
- Add Kubernetes manifest + Helm chart
- Replace Postgres search with Elasticsearch for scale
- Add metrics + Grafana dashboard
- Add CI tests & GitHub Actions