Merge pull request #558 from alokdangre/kubeflow-plugin-foundation

kubeflow: add base plugin structure and foundation

Author: illume

kubeflow/.gitignore

@@ -0,0 +1,29 @@
+# Dependencies
+node_modules/
+npm-debug.log
+yarn-debug.log
+yarn-error.log
+.pnpm-debug.log
+
+# Build outputs
+dist/
+build/
+lib/
+coverage/
+
+# Environment and config
+.env
+.env.local
+.env.*.local
+
+# IDE and editor files
+.idea/
+.vscode/
+*.swp
+*.swo
+.DS_Store
+
+# Cache and temporary files
+.npm
+.eslintcache
+.tsbuildinfo

kubeflow/README.md

@@ -0,0 +1,103 @@
+
+# Kubeflow Plugin for Headlamp
+
+## Purpose
+The Kubeflow plugin for Headlamp provides an operator-focused UI for managing and observing Kubeflow resources running in your Kubernetes clusters. It seamlessly detects available Kubeflow dependencies by natively checking the cluster APIs for Custom Resource Definitions (CRDs).
+
+Instead of duplicating the Kubeflow Central Dashboard, this plugin augments it by providing deep, Kubernetes-centric observability. It helps operators inspect the underlying cluster health, pods, exact status conditions, and detailed configurations of complex Machine Learning workloads.
+
+**Key Feature:** The plugin auto-detects installed features dynamically. You don't need the entire Kubeflow platform to use this plugin! If you install only Katib or only Notebooks via Helm/Kustomize, the plugin will recognize what is available and unlock precisely those specific UI sections in the Headlamp sidebar.
+
+## Supported Components and CRDs
+The plugin currently supports the following modular Kubeflow AI families:
+* **Notebooks** (`kubeflow.org/v1`): `Notebook`, `Profile`, `PodDefault`
+* **Pipelines** (`pipelines.kubeflow.org/v2beta1`): `Pipeline`, `PipelineVersion`
+* **Katib / AutoML** (`kubeflow.org/v1beta1`): `Experiment`, `Trial`, `Suggestion`
+* **Training Operators** (`trainer.kubeflow.org/v1alpha1`): `TrainJob`, `TrainingRuntime`, `ClusterTrainingRuntime`
+* **Spark Operators** (`sparkoperator.k8s.io/v1beta2`): `SparkApplication`, `ScheduledSparkApplication`
+
+## Installation of Kubeflow Components
+Kubeflow is a very large platform, and it is common to install only the systems your team requires. Our plugin is fully compatible with modular installations. 
+
+For full installation workflows across different environments, review the [upstream Kubeflow Manifests Repository](https://github.com/kubeflow/manifests). 
+
+Below are examples of how operators can apply components piece-by-piece via `kustomize` (assuming you have cloned the upstream repo):
+
+**Notebooks Support:**
+```bash
+kustomize build applications/jupyter/notebook-controller/upstream/crd/bases | kubectl apply -f -
+kustomize build applications/jupyter/notebook-controller/upstream/overlays/kubeflow | kubectl apply -f -
+```
+
+**Katib (AutoML) Support:**
+```bash
+# From the root of the cloned kubeflow/manifests repository:
+kustomize build applications/katib/upstream/installs/katib-with-kubeflow | kubectl apply -f -
+```
+
+**Training Operators:**
+```bash
+kustomize build applications/trainer/upstream/base | kubectl apply -f -
+```
+
+**Kubeflow Pipelines (Cloud-Native Mode):**
+```bash
+kustomize build applications/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user-k8s-native | kubectl apply -f -
+```
+
+*Note: Headlamp will always dynamically scan and gracefully handle missing CRDs based on whichever installation method you choose, including experimental Helm charts.*
+
+## Developer Testing (Lightweight Sandbox)
+For UI plugin contributors, a full Kubeflow controller installation is not required and would drain considerable local CPU resources. A lightweight "CRD-only" setup using Headlamp's native probing design is highly recommended.
+
+1. Create a lightweight `kind` cluster:
+   ```bash
+   kind create cluster --name headlamp-kubeflow
+   kubectl config use-context kind-headlamp-kubeflow
+   ```
+
+2. Apply **only the CRDs** directly from upstream. This guarantees near-zero cluster CPU usage while completely activating all native UI tabs within the Headlamp plugin:
+
+   **Notebooks & Multi-Tenancy:**
+   ```bash
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/jupyter/notebook-controller/upstream/crd/bases/kubeflow.org_notebooks.yaml
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/profiles/upstream/crd/bases/kubeflow.org_profiles.yaml
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/admission-webhook/upstream/base/crd.yaml
+   ```
+
+   **Katib:**
+   ```bash
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/katib/upstream/components/crd/experiment.yaml
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/katib/upstream/components/crd/trial.yaml
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/katib/upstream/components/crd/suggestion.yaml
+   ```
+
+   **Pipelines:**
+   ```bash
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/pipeline/upstream/base/crds/pipelines.kubeflow.org_pipelines.yaml
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/pipeline/upstream/base/crds/pipelines.kubeflow.org_pipelineversions.yaml
+   ```
+
+   **Training Operators:**
+   ```bash
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/trainer/upstream/base/crds/trainer.kubeflow.org_trainjobs.yaml
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/trainer/upstream/base/crds/trainer.kubeflow.org_trainingruntimes.yaml
+   kubectl apply -f https://raw.githubusercontent.com/kubeflow/manifests/master/applications/trainer/upstream/base/crds/trainer.kubeflow.org_clustertrainingruntimes.yaml
+   ```
+
+## Running the Plugin Locally
+
+To test modifications dynamically:
+
+1. Start the Plugin Watcher under your `plugins/kubeflow` directory:
+   ```bash
+   npm install
+   npm run start
+   ```
+   *Note: This will watch for changes and compile the plugin into your local Headlamp plugins folder (e.g., `~/.config/Headlamp/plugins/` on Linux).*
+
+2. Launch Headlamp:
+   - **Using the Desktop App:** Simply open your installed Headlamp application.
+   - **From Source:** Clone the [Headlamp repository](https://github.com/headlamp-k8s/headlamp) in a separate directory and start the application following its developer guide (e.g., `make run-backend` and `npm start` in the `frontend` folder).
+
+As long as the plugin watcher is running, any changes saved in the `src/` directory will automatically hot-reload inside Headlamp!

kubeflow/artifacthub-pkg.yml

@@ -0,0 +1,8 @@
+version: 0.1.0
+name: headlamp_kubeflow
+displayName: Kubeflow
+createdAt: "2024-03-17T00:00:00Z"
+description: A UI for viewing and managing Kubeflow resources in Headlamp.
+annotations:
+  headlamp/plugin/version-compat: ">=0.22"
+  headlamp/plugin/distro-compat: "in-cluster,web,docker-desktop,desktop"