How to edit config for on-prem k8s deployment

[sfjhu]

Hi all! I have an on-prem Kubernetes cluster at work that I am attempting to install Nebari into, but the docs for pre-existing Kubernetes cluster only covers AWS installation. I am looking for a step-by-step guide for how to take the bootstrapped nebari-config.yaml and edit it for a cluster that I have a .kubeconfig for.

For example, following the directions in the above referenced docs, I end up with this nebari-config.yaml:

provider: existing
namespace: dev
nebari_version: 2025.10.1
project_name: nebari-new
ci_cd:
  type: none
terraform_state:
  type: remote
security:
  keycloak:
    initial_root_password: 33pd3kzhd2o4yw3m1s3lryg63dqlf9nc
  authentication:
    type: password
theme:
  jupyterhub:
    hub_title: Nebari - nebari-new
    welcome: Welcome! Learn about Nebari's features and configurations in <a href="https://www.nebari.dev/docs/welcome">the
      documentation</a>. If you have any questions or feedback, reach the team on
      <a href="https://www.nebari.dev/docs/community#getting-support">Nebari's support
      forums</a>.
    hub_subtitle: Your open source data science platform, hosted
existing:
  kube_context:
  node_selectors:
    general:
      key: kubernetes.io/os
      value: linux
    user:
      key: kubernetes.io/os
      value: linux
    worker:
      key: kubernetes.io/os
      value: linux

Where in there do I plug in my existing cluster's .kubeconfig values in order to connect the Nebari CLI with the cluster and start the install process?

Values for reference:

apiVersion: v1
clusters:
- cluster:
    insecure-skip-tls-verify: true
    server: https://***link to our cluster***
  name: vcluster
contexts:
- context:
    cluster: vcluster
    user: vcluster
  name: vcluster
current-context: vcluster
kind: Config
preferences: {}
users:
- name: vcluster
  user:
    token: ***token-value***

[viniciusdc]

Hey @sfjhu, thanks for opening the discussion. You are almost there, since you already have a cluster running. I would just double-check that you have the correct context already set up

kubectl config current-context

then, in your config, make sure you pass that name onto

existing:
  kube_context: <my-context> # vcluster
  node_selectors:
    general:
      key: kubernetes.io/os
      value: linux
    user:
      key: kubernetes.io/os
      value: linux
    worker:
      key: kubernetes.io/os
      value: linux

For starters, Nebari depends on three nodes to set up its services. Most of the tools are deployed into "general", while the ephemeral services like dask workers and argo jobs are scheduled into "workers", while "users" are reserved for jhub user instances/profiles.

The reason I mentioned this is that, on on-prem deployments, the assignment of those node groups is based on the labels present there. By default, it targets any node with kubernetes.io/os /os: linux, but you may need to change this later to organize your service roles better.

Before you do that on a production-ready cluster, I would recommend you try out kind or minikube and run a local cluster on your machine to familiarize yourself with the deployment process.

[viniciusdc]

if you can provide a bit more context about how your cluster has been deployed, I will be able to help with any caviats. One thing to note, never interrupt the deployment process middle execution, while it's not the end of the world, it will lock your tf-state files, and that will be very problematic for your first dive with Nebari.