docs: restructure and add new user documentation

Reorganize user-facing documentation into a better guide structure:

- Move API reference to docs/user/reference/api.md
- Create new user guide sections: installation, guides, reference
- Replace scattered user-guides with organized documentation covering:
  - Installation (Kubernetes/OpenShift/monitoring stack setup)
  - Usage guides (PowerMonitor, configuration, Grafana, validation)
  - Reference (API docs, troubleshooting, uninstallation)
- Update README.md with new documentation structure and quick starts
- Update tooling paths (.pre-commit-config.yaml, Makefile)
- Remove deprecated grafana-deployment.md and assets (moved to user/guides)

This restructure improves documentation discoverability and provides
clear installation and usage paths for both Kubernetes and OpenShift users.

Signed-off-by: Sunil Thaha <[email protected]>

Author: Sunil Thaha

.pre-commit-config.yaml

@@ -21,7 +21,7 @@ repos:
     hooks:
       - id: trailing-whitespace
       - id: end-of-file-fixer
-        exclude: ^docs/api\.md$
+        exclude: ^docs/user/reference/api\.md$
       - id: check-added-large-files
       - id: check-merge-conflict
 
@@ -35,7 +35,7 @@ repos:
     rev: v0.44.0
     hooks:
       - id: markdownlint
-        exclude: ^docs/api\.md$
+        exclude: ^docs/user/reference/api\.md$
 
   - repo: https://github.com/codespell-project/codespell
     rev: v2.4.1

Makefile

@@ -144,7 +144,7 @@ coverage: test ## Run tests and generate coverage report.
 
 .PHONY: docs
 docs: crdoc manifests ## Generate docs.
-	$(CRDOC) --resources config/crd/bases --output docs/api.md
+	$(CRDOC) --resources config/crd/bases --output docs/user/reference/api.md
 
 ##@ Development env
 CLUSTER_PROVIDER ?= kind

README.md

@@ -9,124 +9,123 @@ Kepler Operator is a Kubernetes operator that automates the deployment and manag
 
 ## 🔍 What is Kepler?
 
-[Kepler](https://github.com/sustainable-computing-io/kepler) (Kubernetes-based Efficient Power Level Exporter) is a Prometheus
-exporter. It uses eBPF to probe CPU performance counters and Linux kernel
-tracepoints.
-
-These data and stats from cgroup and sysfs can then be fed into ML models to
-estimate energy consumption by Pods.
+[Kepler](https://github.com/sustainable-computing-io/kepler) (Kubernetes-based Efficient Power Level Exporter) is a Prometheus exporter
+that measures energy consumption metrics at the container, pod, and node level in
+Kubernetes clusters.
 
 Check out the project on GitHub ➡️ [Kepler](https://github.com/sustainable-computing-io/kepler)
 
 ## 🚀 Getting Started
 
-You'll need a Kubernetes or OpenShift cluster. For local testing, use [KIND](https://sigs.k8s.io/kind). Otherwise, connect to a remote cluster.
-
-**Note:** The operator uses the current kubeconfig context (check with `kubectl cluster-info`).
-
-### 💻 Using Kind Cluster
-
-To quickly set up a local environment with Kind:
-
-```sh
-make cluster-up
-```
+### For Users
 
-### 🛠️ Local Development
+#### Quick Start (Kubernetes with Helm)
 
-To run the operator locally outside the cluster:
+Get Kepler up and running in minutes. For comprehensive installation instructions, prerequisites, configuration options, and troubleshooting, see the **[Kubernetes Installation Guide](docs/user/installation/kubernetes.md)**.
 
 ```sh
-make tools
-make run
-kubectl apply -k config/samples/
-```
-
-### On Vanilla Kubernetes
-
-Deploy the operator and its dependencies:
-
-```sh
-make tools
-kubectl create -f https://github.com/prometheus-operator/prometheus-operator/releases/download/v0.76.0/bundle.yaml
-kubectl create -f https://github.com/cert-manager/cert-manager/releases/download/v1.18.2/cert-manager.yaml
-make deploy
-kubectl apply -k config/samples/
-```
+# 1. Install cert-manager (required for operator webhooks)
+# For the latest version, see: https://cert-manager.io/docs/installation/
+kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.18.2/cert-manager.yaml
 
-### 📦 Using Helm Chart
+# Wait for cert-manager to be ready
+kubectl wait --for=condition=available --timeout=120s deployment -n cert-manager --all
 
-Install the Kepler Operator using Helm:
+# 2. Install monitoring stack (Prometheus + Grafana)
+# Skip if you already have Prometheus installed
+helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
+helm install prometheus prometheus-community/kube-prometheus-stack \
+  --namespace monitoring \
+  --create-namespace \
+  --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false
 
-```sh
-# Install cert-manager (if not already installed)
-kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.18.2/cert-manager.yaml
+# Wait for monitoring stack to be ready
+kubectl wait --for=condition=ready --timeout=180s pod -n monitoring --all
 
-# Install Kepler Operator from OCI registry
+# 3. Install Kepler Operator
 helm install kepler-operator \
   oci://quay.io/sustainable_computing_io/charts/kepler-operator \
   --namespace kepler-operator \
   --create-namespace
-```
 
-For detailed Helm installation options and configuration, see [Kepler Operator Helm Chart](manifests/helm/kepler-operator/README.md).
+# Wait for operator to be ready
+kubectl wait --for=condition=available --timeout=120s deployment -n kepler-operator --all
 
-### 📦 Using Pre-published Image
+# 4. Deploy Kepler
+# Note: PowerMonitor must be named "power-monitor" (enforced by operator)
+kubectl apply -f https://raw.githubusercontent.com/sustainable-computing-io/kepler-operator/main/config/samples/kepler.system_v1alpha1_powermonitor.yaml
 
-You can use the pre-built image from quay.io:
+# Wait for Kepler pods to be running
+kubectl wait --for=condition=ready --timeout=120s pod -n power-monitor --all
 
-```sh
-make deploy OPERATOR_IMG=quay.io/sustainable_computing_io/kepler-operator:latest
-kubectl apply -k config/samples/
+# 5. Verify installation
+kubectl get pods -n power-monitor
 ```
 
-Alternatively, build and use your own image:
+**Next Steps:**
 
-```sh
-make operator-build operator-push IMG_BASE=<your-registry>
-make deploy IMG_BASE=<your-registry>/kepler-operator:<tag>
-kubectl apply -k config/samples/
-```
+To ensure Kepler is working correctly, follow these validation guides:
 
-### On OpenShift
+- **[Validate Prometheus Integration](docs/user/guides/validating-prometheus-integration.md)** - Verify metrics are being collected (recommended)
+- **[Setup Grafana Dashboards](docs/user/guides/grafana-dashboard.md)** - Visualize power consumption metrics
+- [Configuration Options](docs/user/guides/power-monitor.md) - Customize Kepler deployment
 
-Deploy the operator on OpenShift:
+**Need Help?**
 
-```sh
-make tools
-make operator-build operator-push \
-     bundle bundle-build bundle-push \
-     IMG_BASE=<your-registry> VERSION=0.0.0-dev
-./tmp/bin/operator-sdk run bundle <your-registry>/kepler-operator-bundle:0.0.0-dev \
-  --install-mode AllNamespaces --namespace openshift-operators --skip-tls
-```
+- [Kubernetes Installation Guide](docs/user/installation/kubernetes.md) - Detailed prerequisites, configuration options, and installation steps
+- [Validate Prometheus Integration](docs/user/guides/validating-prometheus-integration.md) - If Kepler is running but metrics aren't appearing
+- [Troubleshooting Guide](docs/user/guides/troubleshooting.md) - Common issues and solutions
 
-## 🗑️ Uninstallation
+#### Quick Start (OpenShift)
 
-To list the installed resources before deletion:
+Install from OperatorHub via the OpenShift Web Console. See the [OpenShift Installation Guide](docs/user/installation/openshift.md) for details.
 
-```sh
-./hack/uninstall-operator.sh
-```
+#### User Documentation
+
+For detailed installation, configuration, and usage instructions, see the [User Guides](docs/user/README.md):
+
+- **Installation**:
+  - [Kubernetes Installation (Helm)](docs/user/installation/kubernetes.md)
+  - [Monitoring Stack Installation](docs/user/installation/monitoring-stack-kubernetes.md)
+  - [OpenShift Installation (OperatorHub)](docs/user/installation/openshift.md)
+- **Usage**:
+  - [Creating PowerMonitor Resources](docs/user/guides/power-monitor.md)
+  - [Configuring PowerMonitor](docs/user/guides/power-monitor.md)
+  - [Setting up Grafana Dashboards](docs/user/guides/grafana-dashboard.md)
+  - [Validating Prometheus Integration](docs/user/guides/validating-prometheus-integration.md)
+  - [Upgrading](docs/user/guides/upgrading.md)
+- **Reference**:
+  - [API Reference](docs/user/reference/api.md)
+  - [Troubleshooting](docs/user/guides/troubleshooting.md)
+  - [Uninstallation](docs/user/reference/uninstallation.md)
 
-To completely remove the operator and all related resources:
+### For Developers
+
+#### Quick Start
 
 ```sh
-./hack/uninstall-operator.sh --delete
+# Setup local Kind cluster with prerequisites
+make cluster-up
+
+# Run operator locally
+make run
+
+# In another terminal, create a PowerMonitor
+kubectl apply -k config/samples/
 ```
 
-## 📚 Developer Documentation
+#### Developer Documentation
 
-[Developer documentation](https://github.com/sustainable-computing-io/kepler-operator/tree/main/docs/developer) is available for those who want to contribute to the codebase or understand its internals.
+For contribution guidelines, architecture details, and development workflows, see the [Developer Documentation](docs/developer/README.md)
 
 ## 🤝 Contributing
 
 You can contribute by:
 
-* Reporting [issues](https://github.com/sustainable-computing-io/kepler-operator/issues)
-* Fixing issues by opening [Pull Requests](https://github.com/sustainable-computing-io/kepler-operator/pulls)
-* Improving documentation
-* Sharing your success stories with Kepler
+- Reporting [issues](https://github.com/sustainable-computing-io/kepler-operator/issues)
+- Fixing issues by opening [Pull Requests](https://github.com/sustainable-computing-io/kepler-operator/pulls)
+- Improving documentation
+- Sharing your success stories with Kepler
 
 ## 📝 License
 

docs/developer/README.md

@@ -1,69 +1,95 @@
+# Developer Documentation
 
-# Code Organization
+This documentation is for developers and contributors who want to understand the Kepler Operator codebase, contribute code, or maintain the project.
+
+**Looking to use Kepler Operator?** See the [User Guides](../user/README.md) instead.
+
+## Audience
+
+This documentation is for:
+
+- Contributors adding features or fixing bugs
+- Maintainers managing releases and project infrastructure
+- Developers wanting to understand internal architecture
+- Anyone building on or extending Kepler Operator
+
+## Code Organization
 
 ```sh
 🏠 kepler-operator
 │
-├── automation       👈  ⚙️  CI related scripts
-│   └── presubmit-tests
+├── .github          👈  ⚙️  GitHub Actions workflows and CI/CD
+│   └── workflows
+│
+├── api              👈  📋 Kubernetes API types
+│   └── v1alpha1        CRDs (Kepler, PowerMonitor, etc)
 │
-├── bundle           👈  📦 Autogenerated and OLM related files
-│   ├── manifests
-│   ├── metadata
-│   └── tests
+├── bundle           👈  📦 Autogenerated OLM bundle
+│   ├── manifests
+│   ├── metadata
+│   └── tests
+│
+├── cmd              👈  🚀 Main entrypoint
+│   └── main.go         Operator manager CLI
+│
+├── config           👈  ⚙️  Kustomize configs (source of truth for manifests)
+│   ├── certmanager     Cert-manager for webhooks
+│   ├── crd             CRD definitions
+│   ├── default         Main kustomization overlays
+│   ├── manager         Manager deployment config
+│   ├── manifests       OLM bundle kustomization
+│   ├── network-policy  Network policies
+│   ├── prometheus      ServiceMonitor configs
+│   ├── rbac            RBAC resources
+│   ├── samples      👈 Sample CRs for users
+│   ├── scorecard       Operator scorecard tests
+│   └── webhook         Webhook configurations
 │
 ├── docs
-│   ├── developer    👈 developer / contributor (design) docs
-│   └── user-guides
+│   ├── design       👈  📐 Architecture and design docs
+│   ├── developer    👈  👩‍💻 Developer/contributor guides
+│   ├── logo            Project logos and assets
+│   └── user         👈  📖 User guides and documentation
+│
+├── hack             👈  🛠️  Development scripts and tools
+│   ├── crds            CRD utilities
+│   ├── dashboard       Grafana dashboard assets
+│   ├── helm            Helm chart tooling and validation
+│   ├── monitoring      Monitoring setup scripts
+│   └── reports         Report generation utilities
+│
+├── internal         👈  🔒 Private application code
+│   ├── config          Operator configuration management
+│   └── controller      Reconciler implementations
+│
+├── manifests        👈  📦 Distribution manifests
+│   └── helm            Helm chart for operator deployment
 │
-├── config           👈 configuration is used to generate the bundle
-│   ├── crd
-│   ├── default
-│   ├── manager
-│   ├── manifests
-│   ├── prometheus
-│   ├── rbac
-│   ├── samples      👈 contains samples that users can use to deploy kepler
-│   └── scorecard
-│ 
-├── hack             👈 🛠️  scripts to help with development
-│   ├── crds
-│   └── dashboard
+├── must-gather      👈  🔍 Debugging and diagnostics collection
 │
-├── cmd              👈 source - for the Manager CLI
-│   └── manager
+├── pkg              👈  📚 Public reusable packages
+│   ├── components      Component manifest builders (PowerMonitor, etc)
+│   ├── reconciler      Generic reconciliation framework
+│   ├── utils           Utility functions (k8s, test helpers)
+│   └── version         Version information
 │
-├── pkg
-│   ├── api
-│   │    └── v1alpha1      👈 k8s API types
-│   │
-│   ├── components         👈 source for creating kepler or other components
-│   │   ├── exporter          manifests like daemsonsets, service, etc
-│   │   └── modelserver
-│   │
-│   ├── controllers        👈 controller-runtime components
-│   │
-│   ├── reconciler         👈 single object creator / deletor etc
-│   └── utils
-│       ├── k8s
-│       └── test
-├── tests            👈 🧪 end-to-end tests 🏠
-│   └── e2e
-└── tmp
-    └── bin          👈  all binaries are installed to `tmp/bin` (make tools)
+└── tests            👈  🧪 Test suites
+    ├── e2e             End-to-end tests
+    ├── testdata        Test fixtures and data
+    └── utils           Test utilities and helpers
 ```
 
-# How do I get started ?
+## How do I get started?
 
-* Knowledge of Kubernetes
-  * API
-  * Controller Generator
+- Knowledge of Kubernetes
+  - API
+  - Controller Generator
 
-* Kube Builder Book: <https://book.kubebuilder.io/>
-* Operator SDK Getting Started: <https://sdk.operatorframework.io/docs/building-operators/golang/tutorial/>
-* Kubernetes Programming Book: <https://www.oreilly.com/library/view/programming-kubernetes/9781492047094/>
+- Kube Builder Book: <https://book.kubebuilder.io/>
+- Operator SDK Getting Started: <https://sdk.operatorframework.io/docs/building-operators/golang/tutorial/>
+- Kubernetes Programming Book: <https://www.oreilly.com/library/view/programming-kubernetes/9781492047094/>
 
-# Developer Guides
+## Developer Guides
 
-* [Helm Chart Maintenance](helm-chart-maintenance.md) - How to update and maintain the Helm chart
-* [Pre-commit Hooks](pre-commit-hooks.md) - Setting up and using pre-commit hooks
+- [Helm Chart Maintenance](helm-chart-maintenance.md) - How to update and maintain the Helm chart
+- [Pre-commit Hooks](pre-commit-hooks.md) - Setting up and using pre-commit hooks

docs/developer/pre-commit-hooks.md

@@ -41,7 +41,7 @@ The following hooks are configured in `.pre-commit-config.yaml`:
 ### 1. [pre-commit-hooks](https://github.com/pre-commit/pre-commit-hooks)
 
 - `trailing-whitespace`: Removes trailing whitespace from lines.
-- `end-of-file-fixer`: Ensures files end with a newline. (Excludes `docs/api.md`)
+- `end-of-file-fixer`: Ensures files end with a newline. (Excludes `docs/user/reference/api.md`)
 - `check-added-large-files`: Prevents committing large files.
 - `check-merge-conflict`: Detects merge conflict markers.
 
@@ -51,7 +51,7 @@ The following hooks are configured in `.pre-commit-config.yaml`:
 
 ### 3. [markdownlint](https://github.com/igorshubovych/markdownlint-cli)
 
-- `markdownlint`: Lints Markdown files (excludes `docs/api.md`).
+- `markdownlint`: Lints Markdown files (excludes `docs/user/reference/api.md`).
 
 ### 4. [codespell](https://github.com/codespell-project/codespell)