Skip to content

Commit 4673b9e

Browse files
committed
feat(kind): skip unprovisioned vTeam tenants in fleet; self-contained vTeam catalog lab
- kind-up now provisions only the demo fleet (tenant-a, tenant-b): the fleet loop applies an overlay only when examples/overlays/<ns>/ exists and fails loudly if any tenant apply fails (no silent partial provisioning). Reduced OPENSHELL_TENANTS default accordingly. - vTeam catalog lab is self-contained against a clean cluster: applying a catalog creates the Project and the control plane provisions the backing namespace + gateway from that record. Rewrote QUICKSTART/vteam-lab/READMEs accordingly, including correct provider-secret setup (vertex/github token key; jira url/email/token).
1 parent 922dbc4 commit 4673b9e

6 files changed

Lines changed: 130 additions & 82 deletions

File tree

Makefile

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,10 @@ GOOGLE_APPLICATION_CREDENTIALS ?= $(or $(shell echo $$GOOGLE_APPLICATION_CREDENT
114114
VERTEX_CRED ?= $(GOOGLE_APPLICATION_CREDENTIALS)
115115

116116
# OpenShell Gateway Configuration (OPENSHELL_USE_GATEWAY=true by default)
117-
# Provisions tenant namespaces with an OpenShell gateway each.
117+
# Provisions the demo fleet tenant namespaces with an OpenShell gateway each.
118+
# Only tenants that have an examples/overlays/<ns>/ overlay are fully applied.
119+
# The vTeam catalog lab (examples/vteam-catalog/) is intentionally NOT provisioned
120+
# here — it is self-contained and applied by the user against a clean cluster.
118121
# Override with OPENSHELL_TENANTS="ns1 ns2" to change the set of tenant namespaces.
119122
# Skip acpctl apply for specific tenants: SKIP_TENANT_SETUP="tenant-c"
120123
OPENSHELL_USE_GATEWAY ?= true
@@ -1002,26 +1005,32 @@ kind-up: preflight-cluster build-cli ## Start kind cluster and deploy the platfo
10021005
sleep 2; \
10031006
TOKEN=$$(kubectl get secret test-user-token -n $(NAMESPACE) -o jsonpath='{.data.token}' 2>/dev/null | base64 -d 2>/dev/null); \
10041007
$$ACPCTL login --url "http://localhost:$${PF_PORT}" --token "$$TOKEN" >/dev/null 2>&1; \
1008+
FLEET_FAILED=""; \
10051009
for ns in $(OPENSHELL_TENANTS); do \
10061010
if echo " $(SKIP_TENANT_SETUP) " | grep -q " $$ns "; then \
10071011
echo " $$ns: skipped (SKIP_TENANT_SETUP)"; \
10081012
continue; \
10091013
fi; \
1014+
[ -f "examples/overlays/$$ns/kustomization.yaml" ] || continue; \
10101015
if [ -f "$(VERTEX_CRED)" ]; then \
10111016
kubectl create secret generic vertex-sa-key \
10121017
--namespace="$$ns" \
10131018
"--from-literal=token=$$(cat '$(VERTEX_CRED)')" \
10141019
--dry-run=client -o yaml | kubectl apply -f - 2>/dev/null; \
10151020
fi; \
1016-
if [ -d "examples/overlays/$$ns" ]; then \
1017-
VERTEX_SA_KEY=$$(cat "$(VERTEX_CRED)" 2>/dev/null || echo "") \
1018-
$$ACPCTL apply -k "examples/overlays/$$ns/" --project "$$ns" && \
1021+
if VERTEX_SA_KEY=$$(cat "$(VERTEX_CRED)" 2>/dev/null || echo "") \
1022+
$$ACPCTL apply -k "examples/overlays/$$ns/" --project "$$ns"; then \
10191023
echo " $$ns: applied"; \
10201024
else \
1021-
echo " $$ns: no overlay directory — skipping fleet"; \
1025+
echo "$(COLOR_RED)$(COLOR_RESET) $$ns: apply failed"; \
1026+
FLEET_FAILED="$$FLEET_FAILED $$ns"; \
10221027
fi; \
10231028
done; \
10241029
kill $$PF_PID 2>/dev/null || true; \
1030+
if [ -n "$$FLEET_FAILED" ]; then \
1031+
echo "$(COLOR_RED)$(COLOR_RESET) Fleet provisioning failed for:$$FLEET_FAILED"; \
1032+
exit 1; \
1033+
fi; \
10251034
fi
10261035
@# Vertex AI setup if requested (non-gateway)
10271036
@if [ "$(OPENSHELL_USE_GATEWAY)" != "true" ]; then \

docs/src/content/docs/guides/vteam-lab.md

Lines changed: 22 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,12 @@ ACP ships two catalog examples:
2424
The manifests live in
2525
[examples/vteam-catalog](https://github.com/openshift-online/agent-control-plane/tree/main/examples/vteam-catalog).
2626

27-
For local Kind clusters, `make kind-up` includes `vteam-product-swarm` and
28-
`codebase-maintainers` in the default `OPENSHELL_TENANTS` list, so those
29-
namespaces are ready for the lab.
27+
The lab is self-contained and runs against a clean cluster: applying a catalog
28+
creates the `Project` record, and the control plane provisions the backing
29+
namespace and gateway from it (see [Gateway and namespace
30+
behavior](#gateway-and-namespace-behavior)). On a local Kind cluster, `make
31+
kind-up` only provisions the demo fleet (`tenant-a`, `tenant-b`); it does not
32+
pre-create the vTeam tenants — the apply step below does.
3033

3134
## Apply a catalog team
3235

@@ -71,8 +74,10 @@ server_dns_names:
7174
- openshell-gateway.vteam-product-swarm.svc.cluster.local
7275
```
7376
74-
The control plane resolves the project namespace as `vteam-product-swarm` and
75-
creates the gateway Kubernetes resources there.
77+
When the `Project` record is applied, the control plane creates the namespace
78+
`vteam-product-swarm` (no `kubectl create namespace` needed) and then, on a
79+
subsequent reconcile pass, deploys the gateway Kubernetes resources there. Both
80+
are eventually consistent — they appear a few seconds after the apply.
7681

7782
## Verify
7883

@@ -90,14 +95,20 @@ acpctl agent list --project codebase-maintainers
9095
acpctl provider list --project codebase-maintainers
9196
```
9297

93-
On a local Kind cluster, also check the project namespaces:
98+
On a local Kind cluster, also check the project namespaces. These are created by
99+
the control plane reconciler after the apply, so wait for them rather than
100+
expecting them immediately:
94101

95102
```bash
96-
kubectl get namespace vteam-product-swarm
97-
kubectl get statefulset openshell-gateway -n vteam-product-swarm
98-
99-
kubectl get namespace codebase-maintainers
100-
kubectl get statefulset openshell-gateway -n codebase-maintainers
103+
kubectl wait --for=jsonpath='{.status.phase}'=Active \
104+
namespace/vteam-product-swarm --timeout=60s
105+
kubectl rollout status statefulset/openshell-gateway \
106+
-n vteam-product-swarm --timeout=120s
107+
108+
kubectl wait --for=jsonpath='{.status.phase}'=Active \
109+
namespace/codebase-maintainers --timeout=60s
110+
kubectl rollout status statefulset/openshell-gateway \
111+
-n codebase-maintainers --timeout=120s
101112
```
102113

103114
For a hand-run local reload flow, use the

examples/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -331,7 +331,7 @@ vteam-catalog/
331331
└── codebase-maintainers/ # Internal codebase maintenance team
332332
```
333333

334-
The `vteam-product-swarm` and `codebase-maintainers` namespaces are provisioned automatically during `make kind-up` (included in the default `OPENSHELL_TENANTS`). See the [vTeam Catalog README](vteam-catalog/README.md) for architecture details and the [QUICKSTART](vteam-catalog/QUICKSTART.md) for a step-by-step walkthrough.
334+
The vTeam lab is self-contained: applying a catalog creates the `Project`, and the control plane provisions its namespace and gateway from that record — so `make kind-up` does not pre-create these tenants (only the `tenant-a`/`tenant-b` demo fleet). See the [vTeam Catalog README](vteam-catalog/README.md) for architecture details and the [QUICKSTART](vteam-catalog/QUICKSTART.md) for a step-by-step walkthrough.
335335

336336
### Applying
337337

examples/vteam-catalog/QUICKSTART.md

Lines changed: 79 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
# vTeam Catalog Manual Reload Quickstart
22

3-
Use this when you want to recreate the current vTeam Catalog lab environment
4-
from the manifests by hand.
3+
Use this to stand up the vTeam Catalog lab environment from the manifests by hand.
4+
The lab is **self-contained**: it runs against a clean/empty cluster and creates
5+
everything it needs (project, namespace, gateway, agents, providers) itself via
6+
`acpctl apply`. `make kind-up` only brings up the cluster and platform — it does
7+
**not** pre-create the `vteam-product-swarm` tenant.
58

69
## 0. Start From The Feature Worktree
710

@@ -34,6 +37,10 @@ Then create the cluster:
3437
make kind-up OPENSHELL_USE_GATEWAY=true
3538
```
3639

40+
This provisions the cluster, the platform, and the OpenShell gateway
41+
infrastructure. It does **not** create the `vteam-product-swarm` project or
42+
namespace — you create those in step 4 by applying the catalog.
43+
3744
After it finishes, check the assigned ports:
3845

3946
```bash
@@ -85,7 +92,6 @@ In the first terminal, log in with the Makefile helper:
8592

8693
```bash
8794
make kind-acpctl-login
88-
export AMBIENT_PROJECT=vteam-product-swarm
8995
```
9096

9197
If you need to do the same steps manually, derive the backend port from
@@ -109,53 +115,19 @@ TOKEN=$(kubectl get secret test-user-token -n ambient-code \
109115
--token "$TOKEN"
110116
```
111117

112-
Quick check:
118+
Quick check that login works. The vTeam project does **not** exist yet — you
119+
create it in the next step:
113120

114121
```bash
115-
export AMBIENT_PROJECT=vteam-product-swarm
116122
"$ACPCTL" get projects
117123
```
118124

119-
## 4. Optional Runtime Secrets
120-
121-
Applying the catalog does not require provider secrets. Starting real sessions
122-
needs the provider backing secrets in namespace `vteam-product-swarm`:
123-
124-
- `vertex-sa-key`
125-
- `github-creds`
126-
- `jira`
127-
128-
If you want Vertex credentials from the repo workflow, run:
129-
130-
```bash
131-
make kind-setup-vertex
132-
```
133-
134-
For the other providers, add your own credentials as `kind: Credential`
135-
resources and apply them with `acpctl`. For example, to give the GitHub-backed
136-
agents (Amber, Parker, …) a token, create a file like
137-
[`examples/overlays/tenant-a/credential-github.yaml`](../overlays/tenant-a/credential-github.yaml):
125+
## 4. Apply The vTeam Catalog Manifests
138126

139-
```yaml
140-
kind: Credential
141-
name: github-cred
142-
provider: github
143-
token: $GITHUB_PAT # a GitHub Personal Access Token
144-
```
145-
146-
```bash
147-
export AMBIENT_PROJECT=vteam-product-swarm
148-
"$ACPCTL" apply -f credential-github.yaml --project vteam-product-swarm
149-
```
150-
151-
The control plane materializes the `github-creds` secret in the project
152-
namespace from this record. See the
153-
[Credentials concept guide](https://openshift-online.github.io/agent-control-plane/concepts/credentials/)
154-
for how credentials, role bindings, and runtime wiring fit together. Without a
155-
credential for a provider an agent declares, its sessions fail to start with
156-
`reading secret <provider>-creds ... not found`.
157-
158-
## 5. Apply The Current vTeam Catalog Manifests
127+
This is the core lab step. Applying the catalog creates the `vteam-product-swarm`
128+
**project** record; the control plane then provisions the backing Kubernetes
129+
namespace and the OpenShell gateway from that record — no `kubectl create
130+
namespace` needed.
159131

160132
```bash
161133
export AMBIENT_PROJECT=vteam-product-swarm
@@ -164,31 +136,85 @@ export AMBIENT_PROJECT=vteam-product-swarm
164136
--project vteam-product-swarm
165137
```
166138

167-
Verify ACP records:
139+
Verify the ACP records (available immediately after apply):
168140

169141
```bash
170-
export AMBIENT_PROJECT=vteam-product-swarm
171142
"$ACPCTL" get project vteam-product-swarm
172143
"$ACPCTL" agent list --project vteam-product-swarm
173144
"$ACPCTL" provider list --project vteam-product-swarm
174145
```
175146

176-
Verify Kubernetes-side objects:
147+
Verify the Kubernetes-side objects. The namespace and gateway are created by the
148+
control plane reconciler, so they appear a few seconds after the apply — wait for
149+
them rather than expecting them immediately:
177150

178151
```bash
179-
kubectl get namespace vteam-product-swarm
152+
kubectl wait --for=jsonpath='{.status.phase}'=Active \
153+
namespace/vteam-product-swarm --timeout=60s
180154
kubectl get all,configmap,secret,pvc,serviceaccount,role,rolebinding \
181155
-n vteam-product-swarm
156+
# The gateway StatefulSet is deployed on the next reconcile pass:
157+
kubectl rollout status statefulset/openshell-gateway \
158+
-n vteam-product-swarm --timeout=120s
182159
```
183160

161+
## 5. Optional Runtime Secrets
162+
163+
Applying the catalog does not require provider secrets. But starting real
164+
sessions does: at gateway setup the control plane reads each provider's backing
165+
Kubernetes Secret directly from the project namespace, so those Secrets must
166+
exist there. The catalog's providers declare these secret names:
167+
168+
- `vertex-sa-key`
169+
- `github-creds`
170+
- `jira`
171+
172+
Do **not** rely on `make kind-setup-vertex` — that target is scoped to the demo
173+
fleet tenants in `OPENSHELL_TENANTS`, not the catalog project.
174+
175+
Vertex and GitHub each use a secret with a single `token` key, created directly
176+
in the `vteam-product-swarm` namespace (created in step 4):
177+
178+
```bash
179+
# Vertex — token is the full contents of a GCP service-account JSON key
180+
kubectl create secret generic vertex-sa-key \
181+
--namespace vteam-product-swarm \
182+
--from-literal=token="$(cat /path/to/gcp-sa-key.json)"
183+
184+
# GitHub — token is a Personal Access Token
185+
kubectl create secret generic github-creds \
186+
--namespace vteam-product-swarm \
187+
--from-literal=token="$GITHUB_PAT"
188+
```
189+
190+
Jira (used only by Parker) needs more than a token — the `jira` provider passes
191+
its Secret keys straight through as environment variables, so the Secret must
192+
carry the base URL, account, and API token that the Atlassian MCP expects
193+
(`JIRA_URL`, `JIRA_USERNAME`, `JIRA_API_TOKEN`), matching the tenant example in
194+
[examples/README.md](../README.md):
195+
196+
```bash
197+
kubectl create secret generic jira \
198+
--namespace vteam-product-swarm \
199+
--from-literal=JIRA_URL=https://your-org.atlassian.net \
200+
--from-literal=JIRA_USERNAME="you@example.com" \
201+
--from-literal=JIRA_API_TOKEN="$(cat ~/jira-token.txt)"
202+
```
203+
204+
Only set up the providers whose agents you actually run. See the
205+
[Credentials concept guide](https://openshift-online.github.io/agent-control-plane/concepts/credentials/)
206+
for how provider secrets, credentials, and runtime wiring fit together. Without
207+
the backing secret/credential for a provider an agent declares, its sessions fail
208+
to start with `reading secret <provider>-creds ... not found`.
209+
184210
## 6. Troubleshooting
185211

186212
Common causes when ACP commands do not show the vTeam records:
187213

188214
- `make kind-port-forward` is not running.
189215
- `acpctl` is logged into the wrong backend port.
190216
- The lab worktree cluster is not running.
191-
- The vTeam manifests have not been applied yet.
217+
- The vTeam manifests have not been applied yet (step 4).
192218

193219
Useful reset commands:
194220

@@ -203,12 +229,11 @@ current backend port from `make kind-status`.
203229

204230
## 7. Optional: Start A Work Packet Session
205231

206-
Starting real sessions needs provider secrets and an OpenShell gateway for the
207-
`vteam-product-swarm` namespace. The default `OPENSHELL_TENANTS` includes
208-
`vteam-product-swarm`, so the namespace and gateway are provisioned
209-
automatically during `make kind-up`.
232+
Starting real sessions needs the provider secrets from step 5 and the OpenShell
233+
gateway. Both come from the catalog apply (step 4) plus your credentials — the
234+
gateway StatefulSet is deployed by the control plane once the namespace exists.
210235

211-
After those runtime prerequisites are available, start Stella with the demo work
236+
Once those runtime prerequisites are available, start Stella with the demo work
212237
packet:
213238

214239
```bash

examples/vteam-catalog/README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -70,9 +70,9 @@ The provider declarations reference these Kubernetes Secret names:
7070
- `github-creds`
7171
- `jira`
7272

73-
Create those secrets in the project namespace before starting sessions that need
74-
the providers. Applying the catalog creates the ACP records; missing provider
75-
secrets become runtime issues when an agent session starts.
73+
Applying the catalog creates the project and its namespace; create these secrets
74+
in that namespace before starting sessions that need the providers. Missing
75+
provider secrets become runtime issues when an agent session starts.
7676

7777
## Codebase Maintainers
7878

@@ -110,6 +110,6 @@ The provider declarations reference these Kubernetes Secret names:
110110
- `github-creds`
111111
- `runtime-kubeconfig`
112112

113-
Create those secrets in the project namespace before starting sessions that need
114-
the providers. Applying the catalog creates the ACP records; missing provider
115-
secrets become runtime issues when an agent session starts.
113+
Applying the catalog creates the project and its namespace; create these secrets
114+
in that namespace before starting sessions that need the providers. Missing
115+
provider secrets become runtime issues when an agent session starts.

scripts/setup-kind-openshell.sh

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#!/usr/bin/env bash
2-
# Install OpenShell gateway prerequisites into a Kind cluster (dual-tenant mode).
2+
# Install OpenShell gateway prerequisites into a Kind cluster (multi-tenant mode).
33
# Called by `make kind-up OPENSHELL_USE_GATEWAY=true`.
44
#
55
# Provisions for each tenant in OPENSHELL_TENANTS (default: tenant-a tenant-b):
@@ -19,7 +19,7 @@ set -euo pipefail
1919
NAMESPACE="${NAMESPACE:-ambient-code}"
2020
AGENT_SANDBOX_VERSION="${AGENT_SANDBOX_VERSION:-v0.5.1}"
2121
# Space-separated list of tenant namespaces to provision
22-
IFS=' ' read -ra TENANTS <<< "${OPENSHELL_TENANTS:-tenant-a tenant-b vteam-product-swarm codebase-maintainers}"
22+
IFS=' ' read -ra TENANTS <<< "${OPENSHELL_TENANTS:-tenant-a tenant-b}"
2323

2424
echo "Setting up OpenShell gateway prerequisites (tenants: ${TENANTS[*]})..."
2525

@@ -192,8 +192,11 @@ else
192192
fi
193193
echo " Note: ambient-ui gateway mode is baked in at build time via --build-arg OPENSHELL_USE_GATEWAY=true"
194194

195-
# Vertex credentials and tenant overlays (examples/overlays/<tenant>/) are
196-
# applied by `make kind-up` after this script finishes — see the
197-
# setup-vertex-provider.sh calls in the Makefile.
195+
# After this script finishes, `make kind-up` applies the fleet overlay for any
196+
# tenant that has one at examples/overlays/<tenant>/ (currently tenant-a, tenant-b).
197+
# The vTeam catalog lab (examples/vteam-catalog/) is NOT provisioned here — it is
198+
# self-contained: the user's `acpctl apply -k examples/vteam-catalog/...` creates the
199+
# project, and the control plane reconciler provisions the namespace + gateway from
200+
# that record. See examples/vteam-catalog/QUICKSTART.md.
198201

199202
echo "OpenShell gateway setup complete (${TENANTS[*]})."

0 commit comments

Comments
 (0)