Backend Monitoring Template

A production-ready backend monitoring template with Prometheus metrics, Loki logging, Grafana dashboards, and Docker configuration. This template provides a solid foundation for building observable and maintainable backend services with comprehensive monitoring capabilities.

🚀 Features

• Custom Prometheus Metrics: Built-in metrics for request counts, response times, and more
• Structured Logging with Loki: Centralized logging system with query capabilities
• Grafana Dashboards: Pre-configured dashboards for visualizing metrics and logs
• Docker Ready: Full Docker and docker-compose configuration for easy deployment
• TypeScript Support: Written in TypeScript for better type safety and developer experience
• Express.js Backend: Built on the popular Express.js framework
• Production Ready: Includes best practices for monitoring and observability

📋 Prerequisites

• Node.js (v14 or higher)
• Docker and Docker Compose
• TypeScript knowledge (basic)
• Git (for version control)

🛠️ Installation

Using Docker (Recommended)

1. Clone the repository:

git clone <your-repo-url>
cd backend_monitoring_template

2. Start the services using Docker Compose:

docker-compose up -d

This will start:

• Your backend application (port 3000)
• Prometheus (port 9090) - Time series database for metrics
• Loki (port 3100) - Log aggregation system
• Grafana (port 3200) - Visualization and analytics platform

Manual Installation

1. Clone the repository
2. Install dependencies:

npm install

3. Start the development server:

npm start

Note: If running manually, you'll need to set up Prometheus, Loki, and Grafana separately.

📊 Monitoring Stack

Endpoints

• Application: http://localhost:3000
• Metrics: http://localhost:3000/metrics
• Health: http://localhost:3000/health
• Prometheus: http://localhost:9090
• Loki: http://localhost:3100
• Grafana: http://localhost:3200 (default credentials: admin/admin)

Grafana Setup

Grafana is configured with:

1. Data Sources:

   • Prometheus: Pre-configured for metrics visualization
   • Loki: Set up for log querying and visualization

2. Dashboards:

   • Node.js Application Metrics:
     • Request rates and latencies
     • Memory usage and garbage collection metrics
     • CPU utilization
   • Logging Dashboard:
     • Log volume trends
     • Error rate monitoring
     • Log level distribution

3. Alerting:

   • Pre-configured alerts for:
     • High error rates
     • Elevated response times
     • Memory usage thresholds
     • CPU usage spikes

🔍 Custom Metrics

The template includes several custom Prometheus metrics:

• http_request_duration_seconds: HTTP request latency histogram

  • Labels: method, path, status_code
  • Helps track API performance and SLOs

• http_requests_total: Total number of HTTP requests

  • Labels: method, path, status_code
  • Useful for traffic analysis and error rate calculation

• nodejs_heap_size_total_bytes: Node.js heap size metrics

  • Helps monitor memory usage and potential leaks

• Custom business metrics can be added in src/utils/prom.client.ts

📝 Logging

Logs are handled by Winston and shipped to Loki. The logging configuration includes:

• Request Logging:

  • HTTP method and path
  • Response time and status code
  • User agent and IP address
  • Request ID for tracing

• Error Logging:

  • Full stack traces
  • Error context and metadata
  • Severity levels

• Structured JSON Logging:

  • Consistent format for easy querying
  • Timestamp and service name included
  • Custom metadata fields

Configure logging in src/utils/loki.config.ts

🐳 Docker Configuration

The template includes:

• Multi-stage Dockerfile:

  • Build stage for TypeScript compilation
  • Production stage with minimal footprint
  • Security best practices implemented

• Docker Compose Setup:

  • Service dependencies properly configured
  • Volume mounts for data persistence
  • Network isolation for security
  • Health checks for reliability

🔧 Configuration

Environment variables can be set in docker-compose.yml or .env file:

• PORT: Application port (default: 3000)
• NODE_ENV: Environment (development/production)
• LOG_LEVEL: Winston log level
• METRICS_PATH: Custom path for Prometheus metrics
• Add additional variables as needed

🤝 Contributing

1. Fork the repository
2. Create your feature branch
3. Commit your changes
4. Push to the branch
5. Create a new Pull Request

📜 License

This project is licensed under the ISC License - a permissive open source license that lets you do anything with the code as long as you include the original copyright and license notice. Similar to MIT but with simpler language.

🆘 Support

For support:

• Open an issue in the repository
• Check existing issues for solutions
• Review documentation in the wiki
• Contact me on LinkedIn or Twitter

🙏 Acknowledgments

• Express.js team
• Prometheus community
• Grafana Labs for Loki and Grafana

---

Made with ❤️ for the monitoring community