docs: add quickstart guide for kcp with kind and helm#3774
docs: add quickstart guide for kcp with kind and helm#3774majiayu000 wants to merge 9 commits intokcp-dev:mainfrom
Conversation
Add comprehensive documentation for deploying kcp on a kind cluster using Helm. The guide covers: - Setting up a kind cluster with port mapping - Installing cert-manager for TLS management - Deploying kcp via the official Helm chart - Creating multiple team workspaces - Generating client certificates for teams - Configuring RBAC for workspace access Fixes kcp-dev#3723 Signed-off-by: majiayu000 <1835304752@qq.com>
kcp-ci-bot
added
dco-signoff: yes
|
Hi @majiayu000. Thanks for your PR. I'm waiting for a kcp-dev member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
mjudeikis
left a comment
mjudeikis
left a comment
There was a problem hiding this comment.
good start but still missing some parts.
In addition it would be great to have this quite in 2 parts:
- helm (as is now)
- kcp-operator.
We dont need todo part2 now, but once we get this working and merged, we can create follow-up issue to update this document :)
| kcp requires cert-manager for TLS certificate management: | ||
|
|
||
| ```bash | ||
| kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml |
There was a problem hiding this comment.
1.19.2 is latest. any reason we use old one?
| Install kcp: | ||
|
|
||
| ```bash | ||
| helm upgrade --install kcp kcp/kcp \ |
There was a problem hiding this comment.
we have only few values. Can we merge values with helm command and use --set flag?
| kcpFrontProxy: | ||
| service: | ||
| type: NodePort | ||
| nodePort: 8443 |
There was a problem hiding this comment.
Are you sure this work?
~/go/src/github.com/kcp-dev/kcp @826066e1* ❯ helm upgrade --install kcp kcp/kcp \ 08:07:30
--namespace kcp \
--create-namespace \
--values kcp-values.yaml \
--wait
Release "kcp" does not exist. Installing it now.
Error: 1 error occurred:
* Service "kcp-front-proxy" is invalid: spec.ports[0].nodePort: Invalid value: 8443: provided port is not in the valid range. The range of valid ports is 30000-32767
Should nodeports be higher up? Or its might be "mac" thing. What os did you tested this on?
| Create four workspaces for different teams: | ||
|
|
||
| ```bash | ||
| kubectl ws create team-alpha --enter |
There was a problem hiding this comment.
This should fail as your kcp deployment does not have host-alias set:
328","apf_pl":"exempt","apf_fs":"exempt","apf_iseats":1,"apf_fseats":0,"apf_additionalLatency":"0s","apf_execution_time":"31.679064ms","resp":200}
{"ts":1767594287729.2998,"caller":"httplog/httplog.go:134","msg":"HTTP","v":3,"verb":"GET","URI":"/livez","latency":"2.296612ms","userAgent":"kube-probe/1.33","audit-ID":"d3763a4f-1dae-4751-8f72-7999ef2bd896","srcIP":"10.244.0.1:46340","apf_pl":"catch-all","apf_fs":"catch-all","apf_iseats":1,"apf_fseats":0,"apf_additionalLatency":"0s","apf_execution_time":"1.932382ms","resp":200}
{"ts":1767594287733.6992,"caller":"httplog/httplog.go:134","msg":"HTTP","v":3,"verb":"GET","URI":"/readyz","latency":"1.820817ms","userAgent":"kube-probe/1.33","audit-ID":"19ff54c8-c6e3-4f20-91a4-542bf9c084d6","srcIP":"10.244.0.1:46342","apf_pl":"catch-all","apf_fs":"catch-all","apf_iseats":1,"apf_fseats":0,"apf_additionalLatency":"0s","apf_execution_time":"1.566277ms","resp":200}
{"ts":1767594288377.4663,"logger":"UnhandledError","caller":"workspace/workspace_controller.go:229","msg":"Unhandled Error","err":"\"kcp-workspace\" controller failed to sync \"root|team-beta\", err: Get \"https://kcp.dev.local:443/clusters/22mycglge1n5hnl0/apis/core.kcp.io/v1alpha1/logicalclusters/cluster\": dial tcp: lookup kcp.dev.local on 10.96.0.10:53: server misbehaving"}
{"ts":1767594289389.4412,"caller":"httplog/httplog.go:134","msg":"HTTP","v":3,"verb":"PUT","URI":"/clusters/system:admin/apis/coordination.k8s.io/v1/namespaces/kube-system/lease
Basically, the pod inside does not know how to reach recursively kcp.dev.local. WE have helm values for this but it need some scripting to detect right IP of the service to use here.
kcp-ci-bot
added
release-note
|
In addition - please instruct your AI bot to follow our PR template :) it prevents all the CI jobs from running if not confogured right |
|
@mjudeikis Thanks for your review! I will fix those comments and improve my bot ! |
kcp-ci-bot
added
dco-signoff: no
Signed-off-by: lif <1835304752@qq.com>
majiayu000
force-pushed
the
fix-3723-feature-quick-start-kind-docum-0101-2313
branch
from
2753592 to
55202c4
Compare
kcp-ci-bot
added
dco-signoff: yes
|
@mjudeikis Thanks for the detailed review! I updated the doc to use cert-manager v1.19.2, switched Helm values to --set flags, fixed NodePort to 30443, and added a hostAliases upgrade step for in-cluster kcp.dev.local resolution. I also noted that the guide currently covers Helm only (operator in follow-up). Appreciate another look when you have a chance. |
mjudeikis
left a comment
mjudeikis
left a comment
There was a problem hiding this comment.
While I really appreciate the effort, this PR looks like it was not tested by human. There are issues in the PR which prevents it from working (see comments).
While you use tools to help, I would really appreciate it if you do a dry run after and make sure it works. And provide logs of the working system in the comments. Else reviewers need to do this, and this is very time-consuming. Especially when you have n other AI-assited PRs to review.
kcp-ci-bot
added
dco-signoff: no
|
Thanks for the review! I've updated the quickstart guide with the following fixes:
Ready for re-review. |
…g, url) Signed-off-by: majiayu000 <1835304752@qq.com>
kcp-ci-bot
added
dco-signoff: yes
|
@majiayu000 any output from the last changes? |
- Add kcpFrontProxy.hostAliases to helm upgrade for full DNS resolution - Add workspace isolation verification in Step 10 - Remove stale kcp-values.yaml reference from cleanup - Add Linux sed syntax note for cross-platform cleanup - Make verify script idempotent (skip existing workspaces) - Show actual kubectl output in verify script instead of suppressing - Add workspace isolation check to verify script Signed-off-by: lif <1835304752@qq.com>
|
@mjudeikis Updated based on your feedback. Changes in this commit: Fixes:
Verification improvements:
Helm chart verification (from
Will run a full end-to-end test and post the output shortly. |
kcp-ci-bot
added
dco-signoff: no
|
Added an explicit write-access check in Step 10 and the verify script: each team now creates (idempotently) a per-team namespace and verifies it. This exercises RBAC beyond just listing pods/namespaces. |
|
You added the script but not the result of script running and showing it works. |
Signed-off-by: majiayu000 <1835304752@qq.com>
Signed-off-by: majiayu000 <1835304752@qq.com>
majiayu000
force-pushed
the
fix-3723-feature-quick-start-kind-docum-0101-2313
branch
from
224379b to
769030b
Compare
kcp-ci-bot
added
dco-signoff: yes
Signed-off-by: majiayu000 <1835304752@qq.com>
|
Ran the full verification script on kind (darwin/arm64). Output below: Also fixed two issues found while running the guide:
|
| EOF | ||
| ``` | ||
|
|
||
| ### Team Gamma |
There was a problem hiding this comment.
Let's reduce the example to alpha, beta. All the rest are the same and just copy-sate
| EOF | ||
| ``` | ||
|
|
||
| ### Team Gamma Access |
| @@ -0,0 +1,200 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
name the file based on docs page it verified:
verify-docs-quickstart-kind.sh
|
@majiayu000 any chance you can turn on your AI agent? :D |
…ript Signed-off-by: majiayu000 <1835304752@qq.com>
|
Sorry, I am currently on vacation and do not have access to a computer capable of running Docker for testing. However, I have simplified the guide to 2 teams and renamed the verification script as suggested, verifying the changes via static analysis. |
mjudeikis-bot
left a comment
mjudeikis-bot
left a comment
There was a problem hiding this comment.
Tested end-to-end locally on Linux with kind v0.30.0, kubectl-kcp/kubectl-ws v0.30.0, helm v3.19.0, cert-manager v1.19.2. Full flow verified:
✅ kind cluster creation with NodePort mapping
✅ cert-manager install & readiness
✅ kcp helm install (initial + hostAliases upgrade)
✅ Admin kubeconfig setup
✅ Team workspace creation (team-alpha, team-beta)
✅ Team cert issuance via cert-manager
✅ RBAC binding per workspace
✅ Team kubeconfig generation
✅ Team access verified — each team can list namespaces and create resources
✅ Workspace isolation verified — team-alpha correctly denied access to team-beta workspace
Two notes below, neither blocking.
| All pods should reach the `Running` state. | ||
|
|
||
| ## Step 5: Configure Admin Access | ||
|
|
There was a problem hiding this comment.
Heads-up: kubectl ws create is deprecated in v0.30.0. When run against a v0.30.0 kcp instance, users will see:
Command "create" is deprecated, install the 'kubectl create-workspace' plugin instead, compare https://docs.kcp.io/kcp/latest/setup/kubectl-plugin/.
The command still works (workspace is created successfully), so it's not breaking. But it would be worth either:
- updating the doc to use
kubectl create workspace <name>(requires thekubectl-create-workspaceplugin), or - adding a note that this deprecation warning is expected and harmless for now.
Same applies to the verification script's workspace create calls.
| KCP_PORT=${KCP_PORT:-8443} | ||
| KIND_KUBECONFIG=${KIND_KUBECONFIG:-"${HOME}/.kube/config"} | ||
| KIND_CONTEXT=${KIND_CONTEXT:-"kind-kcp"} | ||
| KCP_VERIFY_LOG=${KCP_VERIFY_LOG:-""} |
There was a problem hiding this comment.
Minor: cert-manager >= v1.18.0 changed the default spec.privateKey.rotationPolicy from Never to Always. Since the guide installs v1.19.2, all Certificate objects in this script will emit a warning:
Warning: spec.privateKey.rotationPolicy: In cert-manager >= v1.18.0, the default value changed from `Never` to `Always`.
Not harmful — the certificate is still issued correctly. But users may be confused if they see it. Could suppress it by explicitly adding rotationPolicy: Always to the Certificate specs, or just leave it since it's a warning not an error.
Local Test Run — Full OutputTested on Linux, kind v0.30.0, helm v3.19.0, cert-manager v1.19.2, kubectl-kcp/kubectl-ws v0.30.0. Step 1–4: Cluster + cert-manager + kcp helm installStep 5: Admin accessSteps 6–10: Workspaces, certs, RBAC, isolationConclusionLGTM — guide is accurate and works end-to-end. The NodePort mapping, hostAliases upgrade step, cert-manager integration, and workspace isolation all behave exactly as documented. Two observations (neither blocking):
Neither of these affects the correctness of the guide. Good to merge. |
|
/lgtm |
|
LGTM label has been added. DetailsGit tree hash: 44ba096fbc15e42af7df2f5bc7ebe97db755506e |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: mjudeikis The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
kcp-ci-bot
added
the
approved
|
@majiayu000: The following test failed, say
DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
4 participants
Summary
What Type of PR Is This?
/kind documentation
Related Issue(s)
Fixes #3723
Release Notes