Merge pull request #83 from miciav/feature/loki-grafana-main

Feature/loki grafana main

Author: miciav

QWEN.md

@@ -0,0 +1,183 @@
+# Linux Benchmark Library - QWEN Context
+
+## Project Overview
+
+The Linux Benchmark Library (LBB) is a robust and configurable Python library for benchmarking Linux computational node performance. It provides a layered architecture for orchestrating repeatable workloads, collecting rich metrics, and producing clean outputs.
+
+### Key Features
+- **Layered Architecture**: Runner, controller, app, UI, and analytics components
+- **Workload Plugins**: Extensible via entry points and user plugin directory
+- **Remote Orchestration**: Uses Ansible with run journaling
+- **Organized Outputs**: Results, reports, and exports per run and host
+- **Multiple Execution Modes**: Local, remote, Docker, and Multipass execution
+
+### Project Structure
+```
+linux-benchmark-lib/
+├── lb_runner/        # Runner (collectors, local execution helpers)
+├── lb_controller/    # Orchestration and journaling
+├── lb_app/           # Stable API for CLI/UI integrations
+├── lb_ui/            # CLI/TUI implementation
+├── lb_analytics/     # Reporting and analytics
+├── lb_plugins/       # Workload plugins and registry
+├── lb_provisioner/   # Docker/Multipass helpers
+├── lb_common/        # Shared API helpers
+└── tests/            # Unit and integration tests
+```
+
+## Architecture Components
+
+### lb_runner
+- Core benchmark execution engine
+- Local metric collection system
+- Plugin execution framework
+- System information gathering
+
+### lb_controller
+- Remote orchestration engine
+- Ansible integration
+- Run journaling and state management
+- Lifecycle management and interrupt handling
+
+### lb_plugins
+- Workload plugin system with built-in plugins (stress_ng, fio, dd, hpl, stream, etc.)
+- Plugin registry and discovery system
+- Configuration models for each workload type
+
+### lb_ui
+- Command-line interface (CLI) and text user interface (TUI)
+- Typer-based command structure
+- Configuration management
+
+### lb_common
+- Shared utilities and configuration helpers
+- Logging and observability components
+- Environment variable parsing
+
+## Building and Running
+
+### Installation
+```bash
+# Create virtual environment
+uv venv
+
+# Install in different modes
+uv pip install -e .                    # runner only
+uv pip install -e ".[ui]"              # CLI/TUI
+uv pip install -e ".[controller]"      # Ansible + analytics
+uv pip install -e ".[ui,controller]"   # full CLI
+uv pip install -e ".[dev]"             # test + lint tools
+uv pip install -e ".[docs]"            # mkdocs
+```
+
+### Switching Dependency Sets
+```bash
+bash scripts/switch_mode.sh base        # Base runner only
+bash scripts/switch_mode.sh controller  # Full CLI with UI
+bash scripts/switch_mode.sh headless    # Controller without UI
+bash scripts/switch_mode.sh dev         # Development mode
+```
+
+### Quick Start (CLI)
+```bash
+# Initialize configuration
+lb config init -i
+
+# Enable a plugin and run
+lb plugin list --enable stress_ng
+lb run --remote --run-id demo-run
+
+# Development Docker run
+LB_ENABLE_TEST_CLI=1 lb run --docker --run-id demo-docker
+```
+
+### Quick Start (Python API)
+```python
+from lb_controller.api import (
+    BenchmarkConfig,
+    BenchmarkController,
+    RemoteExecutionConfig,
+    RemoteHostConfig,
+)
+
+config = BenchmarkConfig(
+    repetitions=2,
+    remote_hosts=[
+        RemoteHostConfig(name="node1", address="192.168.1.10", user="ubuntu")
+    ],
+    remote_execution=RemoteExecutionConfig(enabled=True),
+)
+
+controller = BenchmarkController(config)
+summary = controller.run(["stress_ng"], run_id="demo-run")
+print(summary.per_host_output)
+```
+
+## Key APIs
+
+### Runner API
+- `LocalRunner`: Core local benchmark execution
+- `BenchmarkConfig`: Configuration for benchmark runs
+- `MetricCollectorConfig`: Configuration for metric collection
+- `WorkloadConfig`: Configuration for individual workloads
+
+### Controller API
+- `BenchmarkController`: Remote orchestration controller
+- `RunJournal`: Run state and journaling
+- `RunLifecycle`: Run phase management
+- `StopCoordinator`: Interrupt and stop handling
+
+### Plugin API
+- `WorkloadPlugin`: Base class for workload plugins
+- `BasePluginConfig`: Base configuration for plugins
+- `PluginRegistry`: Plugin discovery and management
+- Various plugin-specific configs (StressNGConfig, FIOConfig, etc.)
+
+## Development Conventions
+
+### Logging Policy
+- Configure logging via `lb_common.api.configure_logging()` in entrypoints
+- `lb_ui` configures logging automatically; `lb_runner` and `lb_controller` do not
+- Keep stdout clean for `LB_EVENT` streaming when integrating custom UIs
+
+### Testing
+- Unit tests marked with specific markers (unit_runner, unit_controller, etc.)
+- Integration tests with different levels (inter_generic, inter_docker, inter_multipass, etc.)
+- Slow tests marked with `slow` and `slowest` markers
+
+### Code Quality
+- Uses mypy for type checking
+- Black for code formatting
+- Pytest for testing
+- Various linting tools (flake8, vulture, etc.)
+
+## Available Workload Plugins
+
+The library includes several built-in workload plugins:
+- **stress_ng**: CPU, memory, I/O stress testing
+- **fio**: Flexible I/O tester
+- **dd**: Basic disk I/O operations
+- **hpl**: High Performance Linpack
+- **stream**: Memory bandwidth test
+- **sysbench**: System performance benchmark
+- **geekbench**: Cross-platform benchmark
+- **unixbench**: Unix system benchmark
+- **yabs**: Yet Another Benchmark Suite
+- **phoronix_test_suite**: Phoronix test framework
+
+## Documentation and Resources
+
+- Documentation site: https://miciav.github.io/linux-benchmark-lib/
+- API reference: https://miciav.github.io/linux-benchmark-lib/api/
+- Workloads & plugins: https://miciav.github.io/linux-benchmark-lib/plugins/
+- Diagrams: https://miciav.github.io/linux-benchmark-lib/diagrams/
+
+## CLI Commands
+
+The main CLI provides several command groups:
+- `lb config`: Configuration management
+- `lb plugin`: Plugin management and listing
+- `lb run`: Running benchmarks
+- `lb provision`: Environment provisioning
+- `lb runs`: Run history and management
+- `lb doctor`: System checks and diagnostics

benchmark_config.dfaas_multipass.json

@@ -2,7 +2,7 @@
   "remote_hosts": [
     {
       "name": "dfaas-target",
-      "address": "192.168.2.2",
+      "address": "192.168.2.4",
       "user": "ubuntu",
       "become": true,
       "vars": {
@@ -17,12 +17,12 @@
   },
   "plugin_settings": {
     "dfaas": {
-      "k6_host": "192.168.2.3",
+      "k6_host": "192.168.2.5",
       "k6_user": "ubuntu",
       "k6_ssh_key": "/home/ubuntu/.ssh/dfaas_k6_key",
       "k6_port": 22,
-      "gateway_url": "http://192.168.2.2:31112",
-      "prometheus_url": "http://192.168.2.2:30411",
+      "gateway_url": "http://{host.address}:31112",
+      "prometheus_url": "http://{host.address}:30411",
       "functions": [
         {
           "name": "env",
@@ -37,8 +37,11 @@
   },
   "workloads": {
     "dfaas": {
-      "plugin": "dfaas",
-      "enabled": true
+      "plugin": "dfaas"
     }
+  },
+  "loki": {
+    "enabled": true,
+    "endpoint": "http://192.168.2.1:3100"
   }
 }

docs/cli.md

@@ -37,6 +37,8 @@ Order used by commands that need a config:
   Run analytics on an existing run.
 - `lb plugin ...`
   Inspect and manage workload plugins.
+- `lb provision loki-grafana install|remove|status [--mode local|docker] [--grafana-url URL] [--grafana-api-key KEY] [--loki-endpoint URL] [--no-configure]`
+  Install/remove Loki + Grafana and configure datasources/dashboards.
 - `lb config ...`
   Create and manage benchmark configuration files.
 - `lb doctor ...`
@@ -46,10 +48,11 @@ Order used by commands that need a config:
 
 ## Plugin management (`lb plugin ...`)
 
-- `lb plugin list [--select] [--enable NAME | --disable NAME] [-c FILE] [--set-default]`
-- `lb plugin select [-c FILE] [--set-default]`
+- `lb plugin list [--select] [--enable NAME | --disable NAME]`
+- `lb plugin select`
 
 Running `lb plugin` with no subcommand is equivalent to `lb plugin list`.
+Plugin enablement is stored in the platform config (`~/.config/lb/platform.json`); workloads live in the run config.
 
 ## Config management (`lb config ...`)
 

docs/configuration.md

@@ -1,6 +1,7 @@
 ## Configuration
 
-All knobs are defined in `BenchmarkConfig` (import from `lb_runner.api`).
+Runtime configuration is split between the run config (`BenchmarkConfig`) and the
+platform config (`PlatformConfig`) from `lb_runner.api`.
 
 ```python
 from pathlib import Path
@@ -26,7 +27,6 @@ config = BenchmarkConfig(
     workloads={
         "stress_ng": WorkloadConfig(
             plugin="stress_ng",
-            enabled=True,
             options={"cpu_workers": 4, "vm_workers": 2, "vm_bytes": "2G"},
         )
     },
@@ -38,19 +38,44 @@ config = BenchmarkConfig.load(Path("benchmark_config.json"))
 
 ### Notes
 
-- `workloads` is the primary map of workload names to configuration.
+- `workloads` is the primary map of workload names to configuration; presence implies execution.
 - `plugin_settings` can hold typed Pydantic configs for plugins; it is optional.
 - `plugin_assets` is populated from the plugin registry and captures setup/teardown playbooks plus extravars.
 - `output_dir`, `report_dir`, and `data_export_dir` control where artifacts are written.
 - `remote_execution.enabled` controls whether the controller uses Ansible to run workloads.
 - `remote_execution.upgrade_pip` toggles the pip upgrade step during global setup.
 - `workloads.<name>.intensity` accepts `low`, `medium`, `high`, or `user_defined`.
 
+### Platform vs Run Config
+
+The configuration model is split into two files:
+
+- **Platform config**: `~/.config/lb/platform.json`
+  - Holds environment-level settings (e.g. Loki endpoint/labels, Grafana URL/API key, output defaults).
+  - Includes only plugin enablement flags: `"plugins": { "dfaas": true, "fio": false }`.
+  - Optional `grafana` block stores connection defaults (`url`, `api_key`, `org_id`) for provisioning.
+  - Does **not** contain workload definitions or plugin configs.
+  - Never drives execution directly.
+
+- **Run config**: passed via `-c/--config` or `benchmark_config.json`
+  - Includes `remote_hosts` and workload definitions.
+  - Only workloads present in the file are considered runnable.
+  - Workloads do not use an `enabled` flag; presence implies execution.
+  - Experiment-specific plugin options live here (e.g. DFaaS rates/functions/iterations).
+
+Behavioral rules:
+- If a workload is present in the run config but disabled in the platform config,
+  it will be skipped with a warning in the run plan.
+- Provisioning choices (multipass/docker/remote) remain CLI-driven and do not
+  live in the platform config.
+- This is a breaking change: legacy run configs with `enabled` or platform-only
+  fields must be updated to the new split.
+
 ### Plugin settings vs workloads
 
 `workloads` drives execution and can include ad-hoc `options`. `plugin_settings` is
 the typed, validated config model for a plugin. The config service will hydrate
-`plugin_settings` and backfill `workloads` when missing.
+`plugin_settings`, but does not auto-create workloads.
 
 Example:
 
@@ -63,8 +88,7 @@ Example:
 },
 "workloads": {
   "fio": {
-    "plugin": "fio",
-    "enabled": true
+    "plugin": "fio"
   }
 }
 ```