🐝 LogHive

A lightweight microservice written in Rust that collects JSON logs from multiple services, buffers them, and forwards them in batches to Grafana Loki.
It includes health checks, Prometheus metrics, and optional Bearer authentication.

✨ Features

• Log ingestion via HTTP (POST /logs)
• Buffering and batching with configurable size and timeout
• Forwarding to Grafana Loki with retry and exponential backoff
• Optional Bearer authentication
• Prometheus metrics (GET /metrics)
• Health check (GET /health)
• Configuration via TOML file passed as a command‑line argument

📦 Installation

Requirements

• Rust 1.81+
• Cargo
• A running Loki instance (e.g. docker run -p 3100:3100 grafana/loki)

Build

cargo build --release

🚀 Run

Create a configuration file config.toml:

[server]
host = "0.0.0.0"
port = 2157

[loki]
url = "http://localhost:3100"
endpoint = "/loki/api/v1/push"

[pipeline]
queue_capacity = 1000
batch_size = 100
batch_timeout_ms = 1000
max_retries = 3
base_retry_ms = 200

[logging]
level = "info"
json = false

Start the service by passing the config path:

./LogHive config.toml 

🔎 Endpoints

• GET /health: returns OK, is useful to check if the service is online
• GET /metrics: Prometheus metrics
• POST /logs: accepts single log or array of logs in JSON

Example log submission

curl -X POST http://localhost:2157/logs \
  -H "Content-Type: application/json" \
  -d '{
    "timestamp":"2025-11-14T13:00:00Z",
    "service":"auth-service",
    "level":"INFO",
    "message":"Test log"
  }'

If Bearer authentication is enabled:

-H "Authorization: Bearer supersecret-token"

📊 Metrics

• logs_received_total: logs received
• logs_accepted_total: logs accepted
• logs_rejected_total: logs rejected
• batches_sent_total: batches sent to Loki
• send_errors_total: errors sending to Loki
• queue_size: current queue size

🐳 Docker

Build the image:

docker build -t LogHive .

Run:

docker run -p 2157:2157 LogHive config.toml

🧩 Contributing

We welcome contributing. See CONTRIBUTING.md file for details.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project, you agree to abide by its terms.

⚖️ License

This project is licensed under the GPL-3.0 License. See the LICENSE file for details.

⚔️ Contact

• For any inquiries or support, please contact iannacconegiovanni444@gmail.com .
• Visit my site for more informations about me and my work https://giovanni-iannaccone.github.io