Skip to content

README.md

Latest commit

 

History

History
47 lines (33 loc) · 1.57 KB

README.md

File metadata and controls

47 lines (33 loc) · 1.57 KB

Kafka Stream Transformer

Real-time Kafka message streaming with live WebSocket dashboard and schema-based transformations.

A Python microservice that connects to any Kafka cluster, consumes messages, applies user-defined schema transformations, and displays results through a live browser dashboard via WebSocket.

Features

  • Real-time streaming — Consumes messages from Kafka topics with live WebSocket updates to the browser
  • Schema-based transformations — Define JSON schemas to validate and transform messages on the fly
  • Dynamic configuration — Set broker addresses, topic names, and consumer group IDs through the web UI
  • Built-in test producer — Automated message generator for development and demos
  • Fully containerised — Docker Compose setup for the entire stack (Kafka + Zookeeper + app)

Architecture

Kafka Cluster --> Python Consumer --> Schema Validation --> WebSocket --> Browser Dashboard
                  (kafka-python)      (JSONSchema)          (FastAPI)     (Live updates)

Tech Stack

Component Technology
Backend FastAPI, Uvicorn
Messaging Kafka-Python
Validation Pydantic, JSONSchema
Frontend HTML/CSS/JavaScript (WebSocket client)
Infra Docker, Docker Compose

Quickstart

git clone https://github.com/brettleehari/kafka-stream-transformer.git
cd kafka-stream-transformer
docker-compose up -d
# Open http://localhost:8000

Author

Hariprasad Sudharshan - GitHub

License

MIT