Skip to content

Commit 16d1126

Browse files
authored
Fix AMD GPU power measurement with rocm-smi (#4)
## Problem The rocm-smi energy meter was returning 0.0 for all power readings on AMD GPUs. ## Root Cause 1. Used `-p` flag which shows 'Performance Level', not power consumption 2. Regex patterns didn't match the actual output format from modern rocm-smi ## Fix - Changed flag from `-p` to `--showpower` - Added regex pattern for `Current Socket Graphics Package Power (W): X.XXX` format ## Testing Verified on AMD Radeon Graphics (RYZEN AI MAX+ 395): ``` Before: energy_wh_raw: 0.0 After: energy_wh_raw: 0.0022 (2 second test) ``` Also includes copilot-instructions.md for AI coding agents.
2 parents da544fc + 3fec8fe commit 16d1126

File tree

2 files changed

+105
-4
lines changed

2 files changed

+105
-4
lines changed

.github/copilot-instructions.md

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# Copilot Instructions for Energy Leaderboard Runner
2+
3+
## Project Overview
4+
5+
This project measures real-world energy consumption of local LLMs. It has two main components:
6+
1. **Python CLI** (`src/`) - Runs benchmarks and collects energy metrics from hardware sensors
7+
2. **React Web App** (`energy-leaderboard-web/`) - Displays crowdsourced benchmark results
8+
9+
## Architecture & Key Patterns
10+
11+
### Energy Meter System (Plugin Architecture)
12+
All energy meters inherit from `EnergyMeter` base class in [src/energy_meter/base.py](src/energy_meter/base.py):
13+
- `is_available()` → Platform detection
14+
- `start()` / `stop()` → Measurement lifecycle returning `energy_wh_raw`, `duration_s`, `sampling_ms`
15+
16+
Platform detection priority in [src/energy_meter/integrator.py](src/energy_meter/integrator.py):
17+
- macOS: `powermetrics` (requires sudo)
18+
- Linux: NVML (NVIDIA) → ROCm (AMD) → RAPL (CPU)
19+
20+
### LLM Integrations (Plugin Architecture)
21+
Implement `LlmRunner` from [src/llm_integrations/base.py](src/llm_integrations/base.py):
22+
- `check_connection()` → Validate endpoint
23+
- `generate()` → Returns `(text, tokens_prompt, tokens_completion, response_time_s)`
24+
25+
Supported: Ollama (`ollama_client.py`), OpenAI-compatible (`openai_client.py`)
26+
27+
### Configuration
28+
Environment-based via [src/config.py](src/config.py):
29+
- `OLLAMA_HOST` (default: `http://localhost:11434`)
30+
- `CO2_INTENSITY_G_KWH` (default: 350.0 - EU average)
31+
- `SAMPLING_INTERVAL_MS` (default: 100)
32+
33+
## Commands & Workflows
34+
35+
```bash
36+
# Install dependencies
37+
pip install -r requirements.txt
38+
39+
# Run single benchmark (requires Ollama running)
40+
python src/main.py run-test --model llama3:latest --test-set easy
41+
42+
# Run all test sets (easy, medium, hard, mixed) - preferred for contributions
43+
python run_all_tests.py --model llama3:latest
44+
45+
# OpenAI-compatible provider
46+
python run_all_tests.py --model gpt-4 --provider openai --base-url https://api.example.com
47+
48+
# Docker (Linux with GPU)
49+
docker build -t energy-leaderboard-runner .
50+
docker run --rm --gpus all -v $(pwd)/results:/app/results \
51+
-e OLLAMA_HOST=http://172.17.0.1:11434 energy-leaderboard-runner \
52+
run_all_tests.py --model llama3:latest
53+
```
54+
55+
## Test Sets
56+
57+
Located in [src/data/testsets/](src/data/testsets/). Structure:
58+
```json
59+
{
60+
"id": "ts1",
61+
"name": "...",
62+
"goal": "...",
63+
"questions": [{ "id": "...", "prompt": "...", "difficulty": "easy" }]
64+
}
65+
```
66+
Reference testset by name without prefix: `--test-set easy` (resolves to `testset_easy.json`)
67+
68+
## Output Schema
69+
70+
Results validated against [src/data/metrics_schema.json](src/data/metrics_schema.json). Key metrics:
71+
- `energy_wh_raw` / `energy_wh_net` - Energy consumption
72+
- `wh_per_1k_tokens` - Efficiency metric
73+
- `g_co2` - Calculated from `CO2_INTENSITY_G_KWH`
74+
75+
Output files: `results/output_{model}_{testset}_{date}.json`
76+
77+
## Web Frontend (`energy-leaderboard-web/`)
78+
79+
React + Vite + Tailwind. Data lives in `public/data/*.json`.
80+
81+
```bash
82+
cd energy-leaderboard-web
83+
npm install && npm run dev # Development
84+
npm run build # Production build
85+
```
86+
87+
## Contributing Benchmarks
88+
89+
1. Run `python run_all_tests.py --model <model>`
90+
2. Copy `results/output_*.json``energy-leaderboard-web/public/data/`
91+
3. Submit PR with new JSON files
92+
93+
## Code Conventions
94+
95+
- CLI uses Typer with Rich for console output
96+
- Dual import pattern in main.py supports both module and direct execution
97+
- Abstract base classes define interfaces; implementations are in same directory
98+
- All file paths use `pathlib.Path`

src/energy_meter/rocm_smi_meter.py

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -73,9 +73,9 @@ def _poll_power(self) -> None:
7373
try:
7474
# Run rocm-smi to get power usage
7575
# -d: device index
76-
# -p: show power consumption
76+
# --showpower: show current power consumption
7777
result = subprocess.run(
78-
["rocm-smi", "-d", str(self.device_index), "-p"],
78+
["rocm-smi", "-d", str(self.device_index), "--showpower"],
7979
capture_output=True,
8080
text=True,
8181
timeout=1.0,
@@ -102,9 +102,12 @@ def _parse_power(self, output: str) -> Optional[float]:
102102
Returns:
103103
Power in watts, or None if parsing fails.
104104
"""
105-
# Look for patterns like "Average Graphics Package Power: 123.45 W"
106-
# or "Power: 123 W"
105+
# Look for patterns like:
106+
# "Current Socket Graphics Package Power (W): 4.052"
107+
# "Average Graphics Package Power: 123.45 W"
108+
# "Power: 123 W"
107109
patterns = [
110+
r"Current Socket Graphics Package Power \(W\):\s+([\d.]+)",
108111
r"Average Graphics Package Power:\s+([\d.]+)\s*W",
109112
r"Power:\s+([\d.]+)\s*W",
110113
r"GPU Power:\s+([\d.]+)\s*W",

0 commit comments

Comments
 (0)