document event audit logs

Signed-off-by: olalekan odukoya <[email protected]>

Author: olamilekan000

docs/content/setup/production/audit-logging.md

@@ -0,0 +1,140 @@
+---
+description: >
+  Understanding kcp audit event annotations and how to use them for monitoring and compliance.
+---
+
+# Audit Logging
+
+kcp extends Kubernetes audit events with workspace-specific annotations that help identify which workspace (logical cluster) each request belongs to. This is essential for multi-tenant environments where you need to track and audit access across different workspaces.
+
+## Audit Event Annotations
+
+kcp automatically adds the following annotations to all audit events:
+
+### `kcp.io/path`
+
+The canonical workspace path (e.g., `root:consumer:production`). This is the human-readable hierarchical path that uniquely identifies the workspace.
+
+### `tenancy.kcp.io/workspace`
+
+The workspace cluster name (logical cluster identifier). This is the internal cluster name used by KCP.
+
+### `kcp.io/cluster`
+
+Alias for the workspace cluster name. Contains the same value as `tenancy.kcp.io/workspace`.
+
+## Example Audit Event
+
+```json
+{
+  "kind": "Event",
+  "apiVersion": "audit.k8s.io/v1",
+  "level": "Request",
+  "auditID": "5684337a-48d6-4491-aed2-a0bca6fcda2b",
+  "stage": "RequestReceived",
+  "requestURI": "/api/v1/namespaces/default/configmaps?limit=500",
+  "verb": "list",
+  "user": {
+    "username": "kcp-admin",
+    "uid": "e6741a4d-fc7c-44c5-b5ec-9357b44b7e0b",
+    "groups": [
+      "system:kcp:admin",
+      "system:authenticated"
+    ]
+  },
+  "sourceIPs": [
+    "127.0.0.1"
+  ],
+  "userAgent": "kubectl/v1.30.1 (darwin/arm64) kubernetes/6911225",
+  "objectRef": {
+    "resource": "configmaps",
+    "namespace": "default",
+    "apiVersion": "v1"
+  },
+  "requestReceivedTimestamp": "2025-12-09T16:03:44.214758Z",
+  "stageTimestamp": "2025-12-09T16:03:44.214758Z",
+  "annotations": {
+    "kcp.io/cluster": "16iai06e7ob717ht",
+    "kcp.io/path": "root:consumer:production",
+    "tenancy.kcp.io/workspace": "16iai06e7ob717ht"
+  }
+}
+```
+
+The example audit event above would be generated by running the following command:
+
+```bash
+kubectl --server=https://127.0.0.1:6443/clusters/root:consumer:production get configmap -n default
+```
+
+This command lists configmaps in the `default` namespace within the `root:consumer:production` workspace, which triggers the audit event with the workspace annotations shown in the example.
+
+
+## Enabling Audit Logging
+
+To enable audit logging in kcp, you need to provide an audit policy file and configure where audit logs should be written. Audit logging is disabled by default.
+
+### Basic Setup
+
+Enable audit logging by starting kcp with the following flags:
+
+```bash
+kcp start \
+  --audit-policy-file=/path/to/audit-policy.yaml \
+  --audit-log-path=/path/to/kcp.audit \
+  --audit-log-format=json
+```
+
+### Required Flags
+
+- **`--audit-policy-file`**: Path to the audit policy YAML file that defines which events should be logged
+- **`--audit-log-path`**: Path where audit logs will be written
+- **`--audit-log-format`**: Output format (`json` or `legacy`). JSON format is recommended for easier parsing
+
+### Example Audit Policy
+
+Create an audit policy file (e.g., `audit-policy.yaml`) to define what events to log:
+
+```yaml
+apiVersion: audit.k8s.io/v1
+kind: Policy
+rules:
+  - level: Request
+    verbs: ["*"]
+```
+
+This example policy logs all requests at the `Request` level. For production environments, you may want to be more selective about what gets logged to manage log volume.
+
+### Complete Example
+
+Here's a complete example of starting kcp with audit logging enabled:
+
+```bash
+# Create audit policy file
+cat > audit-policy.yaml <<EOF
+apiVersion: audit.k8s.io/v1
+kind: Policy
+rules:
+  - level: Request
+    verbs: ["*"]
+    resources:
+      - group: "*"
+        resources: ["*"]
+EOF
+
+# Start kcp with audit logging
+kcp start \
+  --audit-policy-file=./audit-policy.yaml \
+  --audit-log-path=./kcp.audit \
+  --audit-log-format=json
+```
+
+## Configuration
+
+Audit logging in kcp uses the standard Kubernetes audit policy configuration. For detailed information on configuring audit policies, including different log levels and filtering options, see the [Kubernetes audit documentation](https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/).
+
+## Notes
+
+- The `kcp.io/path` annotation is only populated when the LogicalCluster informer is available. In cache-server deployments, this annotation may be empty.
+- All annotations are added to both `RequestReceived` and `ResponseComplete` audit event stages.
+- The workspace annotations are automatically added to all audit events, regardless of the audit policy configuration.

docs/content/setup/production/index.md

@@ -131,6 +131,14 @@ pods are scheduled properly.
 
 All kcp data is stored in etcd, there is no need to perform a dedicated kcp backup.
 
+### Observability
+
+#### Audit Logging
+
+kcp extends Kubernetes audit events with workspace-specific annotations that help identify which workspace (logical cluster) each request belongs to. This is essential for multi-tenant environments where you need to track and audit access across different workspaces.
+
+See [Audit Logging](audit-logging.md) for details on the annotations (`kcp.io/path`, `tenancy.kcp.io/workspace`, `kcp.io/cluster`) and how to use them.
+
 ---
 
 ## Production Deployment Options