LOCAL GPU Health Monitor

[DataBoySu]

Discussion

A lightweight nvidia-smi wrapper and local GPU monitoring utility.

Table of Contents

• Overview
• Features
• Interfaces
• Benchmarking & Simulation
• Installation & Quick Start
• Roadmap & Contribution
• License

---

Overview

This project began as a personal tool for local GPU monitoring and experiment management. Basically, the right balance between the raw data output of nvidia-smi and heavyweight monitoring solutions.

The design goals are:

• Minimal resource footprint and simple deployment.
• Useful admin features (e.g., VRAM enforcement and watchlists).
• Extensible benchmarking and testing utilities for reproducible local experiments.

Features

• Real-time Metrics: GPU utilization, VRAM usage, temperature, power draw; system metrics such as CPU, memory, and disk I/O.
• Per-Process Tracking: Track VRAM per process and maintain watchlists for focused monitoring.
• Admin Enforcement: Configurable VRAM caps with optional auto-termination logic to protect shared systems.
• Flexible Deployments: CLI-only, background service, or full Web Dashboard (FastAPI + vanilla JS).
• Lightweight Dashboard: Clean UI with live charts and historical buffering (local only by default).

Interfaces

• CLI: Quick checks, scripted workflows, and headless environments.
• Service / Daemon: Background monitoring suitable for developer machines or single-host clusters.
• Web Dashboard: Interactive browser-based UI with REST endpoints for integration.

Benchmarking & Simulation

• GEMM Workloads: Configurable matrix multiplications to exercise compute and memory.
• Particle Simulation: Visual simulation mode for stress tests and TFLOPS estimations.

Roadmap & Contribution

Planned improvements and areas where contributions are welcome:

• Multi-GPU Support — robust handling for multiple card setups and topologies.
• Containerization — official Docker images and Compose files for easy deployment.
• Remote Access — SSH tunneling and secure remote monitoring.
• Cross-Platform — Linux (Ubuntu/Debian) and macOS (including Apple Silicon) support.
• Hardware Agnostic — AMD (ROCm) and Intel (Arc) GPU support.

If you'd like to contribute, see CONTRIBUTING.md for setup instructions, coding guidelines, and the contribution process.

License

This project is MIT licensed. See LICENSE for details.

---

If you'd like any specific sections expanded (developer API, config reference, examples), tell me which area to elaborate on.