📖 Available in: Español | English
| Team |
|---|
| Federico Genaro |
| Santiago Sevitz |
| Maximo Utrera |
Architecture Documents:
- Submission I: Design
- Submission II: Scalability
- Submission III: Multiclient
- Submission IV: Fault Tolerance
This project presents the design of a fault-tolerant distributed system for analyzing sales data from a coffee shop chain in Malaysia. It was developed as a Practical Work for the Distributed Systems I (FIUBA) course and prioritizes scalability, robustness, and distributed pipeline processing.
The system enables processing large volumes of transactions sent by a client, applying complex transformations and aggregations, and returning consolidated reports while maintaining consistency even in the face of partial or catastrophic system failures.
- Process large transaction datasets in a distributed manner.
- Support multiple analytical requirements (filtering, aggregation, ranking, joins).
- Guarantee fault tolerance, duplicate handling, and idempotence.
- Horizontal scaling independently for each node type.
The system primarily works with:
- Transactions (multiple files)
- Transaction Items (multiple files)
As reference data (for joins):
- Menu Items
- Stores
- Users
Other datasets (Vouchers, Payment Methods) are not used as they are not necessary for the expected results.
-
Client: sends data and receives reports.
-
Gateway: entry point to the system and unique interface with the client.
-
Controller: coordinates execution and controls transmission end.
-
Workers:
- Filter
- GroupBy
- Reducer
- Aggregator
- Joiner
-
Middleware: RabbitMQ as Message-Oriented Middleware.
Internal communication is conducted exclusively through message queues, enabling decoupling and parallelism.
The system uses the Worker-per-filter model, where each pipeline stage is isolated:
- Filter: applies temporal and business criteria.
- GroupBy: groups records by relevant keys.
- Reducer: executes sums or counts.
- Aggregator: consolidates batches and applies rankings (Top N).
- Joiner: links processed data with reference datasets.
Each functional requirement defines a specific pipeline combining these stages.
- Client requests a task from the Gateway.
- Controller initializes the client control sequence.
- Client sends reference datasets.
- Client sends transaction batches.
- Workers process data in a chained manner.
- Aggregator consolidates results.
- Joiner generates the final report.
- Gateway returns the report to the client.
-
Health Check (Egg of Life): monitoring via UDP heartbeats and automatic node restart.
-
Deferred ACKs: message re-queuing upon processing failures.
-
Disk Persistence:
- Joiner: reference datasets.
- Aggregator: processed batches.
- Controller: client state.
-
Duplicate Handling: sequence numbers for packets per client.
-
Idempotent Messages: especially for transmission end.
- Gateway (Load balanced randomly)
- Controller (Load balanced deterministically by client using a hash function)
- Aggregator
The Bully algorithm was implemented to form a monitoring network among Egg of life nodes, ensuring the system does not depend on a single node for automatic restarts. Instead, they restart each other in case of failure. With the premise of low maintainability, this solution ensures that there will never be a lack of such nodes, thereby allowing recovery from an extreme system failure case (e.g., all workers, gateways, controllers, and egg of life nodes except one).
- RabbitMQ with persistent queues.
- Workers deployed as Docker containers.
- Horizontal scaling through replicas.
- Orchestration with Docker Compose (in this case, but can be deployed on Kubernetes).
- Languages: Go, Python
- Messaging: RabbitMQ
- Serialization: Protobuf
- Containers: Docker
- Dataset: Kaggle – Coffee Shop Transactions