Skip to content

Commit 84f506a

Browse files
jewertowjewertow
authored
samples: add a demo for SPIRE and trust domain federation (#58462)
* samples: add a demo for SPIRE and trust domain federation Signed-off-by: Jacek Ewertowski <[email protected]> * Fix linter errors Signed-off-by: Jacek Ewertowski <[email protected]> * Remove DestinationRule with all trust domains Signed-off-by: Jacek Ewertowski <[email protected]> --------- Signed-off-by: Jacek Ewertowski <[email protected]>
1 parent a65889d commit 84f506a

File tree

6 files changed

+305
-0
lines changed

6 files changed

+305
-0
lines changed

‎samples/security/spire-trust-domain-federation/README.md‎

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
# Integrating multi-cluster mesh with federated SPIRE servers
2+
3+
This sample deploys a setup of [Federated SPIRE Architecture](https://spiffe.io/docs/latest/architecture/federation/readme/)
4+
as an example of integrating multi-cluster mesh with SPIRE and different trust domains.
5+
6+
## Before you begin
7+
8+
### Cluster
9+
10+
```shell
11+
samples/kind-lb/setupkind.sh --cluster-name cluster-east --ip-space 254
12+
samples/kind-lb/setupkind.sh --cluster-name cluster-west --ip-space 255
13+
```
14+
15+
### Environment variables
16+
17+
```shell
18+
export CTX_CLUSTER_EAST=$(kubectl config get-contexts -o name | grep kind-cluster-east)
19+
export CTX_CLUSTER_WEST=$(kubectl config get-contexts -o name | grep kind-cluster-west)
20+
```
21+
22+
## Demo
23+
24+
### Deploy SPIRE
25+
26+
1. Install Helm charts:
27+
28+
```shell
29+
helm repo add spiffe-hardened https://spiffe.github.io/helm-charts-hardened
30+
```
31+
32+
1. Install SPIRE components (CRDs, CSI driver, server and agent):
33+
34+
```shell
35+
# CRDs
36+
helm --kube-context="${CTX_CLUSTER_EAST}" upgrade --install spire-crds spiffe-hardened/spire-crds --version 0.5.0
37+
helm --kube-context="${CTX_CLUSTER_WEST}" upgrade --install spire-crds spiffe-hardened/spire-crds --version 0.5.0
38+
# CSI driver, server and agent
39+
helm --kube-context="${CTX_CLUSTER_EAST}" upgrade --install spire spiffe-hardened/spire -n spire --create-namespace \
40+
-f samples/security/spire-trust-domain-federation/spire-base.yaml \
41+
-f samples/security/spire-trust-domain-federation/spire-east.yaml \
42+
--version 0.24.0 --wait
43+
helm --kube-context="${CTX_CLUSTER_WEST}" upgrade --install spire spiffe-hardened/spire -n spire --create-namespace \
44+
-f samples/security/spire-trust-domain-federation/spire-base.yaml \
45+
-f samples/security/spire-trust-domain-federation/spire-west.yaml \
46+
--version 0.24.0 --wait
47+
```
48+
49+
1. Federate bundles:
50+
51+
```shell
52+
spire_bundle_endpoint_west=$(kubectl --context="${CTX_CLUSTER_WEST}" get svc spire-server -n spire -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
53+
west_bundle=$(kubectl --context="${CTX_CLUSTER_WEST}" exec -c spire-server -n spire --stdin spire-server-0 -- spire-server bundle show -format spiffe)
54+
indented_west_bundle=$(echo "$west_bundle" | jq -r '.' | sed 's/^/ /')
55+
(cat samples/security/spire-trust-domain-federation/cluster-federated-trust-domain.yaml; echo -e " trustDomainBundle: |-\n$indented_west_bundle") |\
56+
sed "s/\${CLUSTER}/west/g" |\
57+
sed "s/\${BUNDLE_ENDPOINT}/$spire_bundle_endpoint_west/g" |\
58+
kubectl --context="${CTX_CLUSTER_EAST}" apply -f -
59+
```
60+
61+
```shell
62+
spire_bundle_endpoint_east=$(kubectl --context="${CTX_CLUSTER_EAST}" get svc spire-server -n spire -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
63+
east_bundle=$(kubectl --context="${CTX_CLUSTER_EAST}" exec -c spire-server -n spire --stdin spire-server-0 -- spire-server bundle show -format spiffe)
64+
indented_east_bundle=$(echo "$east_bundle" | jq -r '.' | sed 's/^/ /')
65+
(cat samples/security/spire-trust-domain-federation/cluster-federated-trust-domain.yaml; echo -e " trustDomainBundle: |-\n$indented_east_bundle") |\
66+
sed "s/\${CLUSTER}/east/g" |\
67+
sed "s/\${BUNDLE_ENDPOINT}/$spire_bundle_endpoint_east/g" |\
68+
kubectl --context="${CTX_CLUSTER_WEST}" apply -f -
69+
```
70+
71+
### Deploy Istio
72+
73+
1. Install control planes:
74+
75+
```shell
76+
sed -e "s/\${LOCAL_CLUSTER}/east/g" \
77+
-e "s/\${REMOTE_CLUSTER}/west/g" \
78+
-e "s/\${REMOTE_BUNDLE_ENDPOINT}/$spire_bundle_endpoint_west/g" \
79+
samples/security/spire-trust-domain-federation/istio.tmpl | istioctl --context="${CTX_CLUSTER_EAST}" install -y -f -
80+
sed -e "s/\${LOCAL_CLUSTER}/west/g" \
81+
-e "s/\${REMOTE_CLUSTER}/east/g" \
82+
-e "s/\${REMOTE_BUNDLE_ENDPOINT}/$spire_bundle_endpoint_east/g" \
83+
samples/security/spire-trust-domain-federation/istio.tmpl | istioctl --context="${CTX_CLUSTER_WEST}" install -y -f -
84+
```
85+
86+
1. Install an E/W gateway and expose services:
87+
88+
```shell
89+
samples/multicluster/gen-eastwest-gateway.sh --network west-network | istioctl --context="${CTX_CLUSTER_WEST}" install -y -f -
90+
kubectl --context="${CTX_CLUSTER_WEST}" patch deploy istio-eastwestgateway -n istio-system --type='merge' \
91+
-p '{"spec": {"template": {"metadata": {"annotations": {"inject.istio.io/templates": "gateway,spire-gateway"}}}}}'
92+
kubectl --context="${CTX_CLUSTER_WEST}" apply -n istio-system -f samples/multicluster/expose-services.yaml
93+
```
94+
95+
1. Enable service discovery in remote cluster:
96+
97+
```shell
98+
WEST_API_SERVER_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' cluster-west-control-plane)
99+
istioctl --context="${CTX_CLUSTER_WEST}" create-remote-secret --name=west --server="https://${WEST_API_SERVER_IP}:6443" | kubectl --context="${CTX_CLUSTER_EAST}" apply -f -
100+
```
101+
102+
### Verify the installation
103+
104+
1. To confirm that Istiod is now able to communicate with the Kubernetes control plane of the remote cluster.
105+
106+
```shell
107+
istioctl --context="${CTX_CLUSTER_EAST}" remote-clusters
108+
```
109+
110+
```text
111+
NAME SECRET STATUS ISTIOD
112+
east synced istiod-8887cd9cd-zv2zf
113+
west istio-system/istio-remote-secret-west synced istiod-8887cd9cd-zv2zf
114+
```
115+
116+
```shell
117+
istioctl --context="${CTX_CLUSTER_WEST}" pc secret deploy/istio-eastwestgateway -n istio-system
118+
```
119+
120+
```text
121+
RESOURCE NAME TYPE STATUS VALID CERT SERIAL NUMBER NOT AFTER NOT BEFORE TRUST DOMAIN
122+
default Cert Chain ACTIVE true 508823656af07c0056bae33c3c9dd26f 2025-09-05T21:43:55Z 2025-09-05T17:43:45Z west.local
123+
ROOTCA CA ACTIVE true bbe128cc1365e276cd5e114a482aefee 2025-09-06T17:33:34Z 2025-09-05T17:33:24Z east.local
124+
ROOTCA CA ACTIVE true 594e1e8df46d0e3f95ed3d4571abb011 2025-09-06T17:35:05Z 2025-09-05T17:34:55Z west.local
125+
```
126+
127+
1. Deploy the `HelloWorld` service:
128+
129+
```shell
130+
kubectl --context="${CTX_CLUSTER_EAST}" label namespace default istio-injection=enabled
131+
kubectl --context="${CTX_CLUSTER_WEST}" label namespace default istio-injection=enabled
132+
```
133+
134+
```shell
135+
kubectl apply --context="${CTX_CLUSTER_EAST}" \
136+
-f samples/helloworld/helloworld.yaml \
137+
-l service=helloworld
138+
kubectl apply --context="${CTX_CLUSTER_WEST}" \
139+
-f samples/helloworld/helloworld.yaml \
140+
-l service=helloworld
141+
```
142+
143+
1. Deploy V1 in the `east` cluster:
144+
145+
```shell
146+
kubectl apply --context="${CTX_CLUSTER_EAST}" \
147+
-f samples/helloworld/helloworld.yaml \
148+
-l version=v1
149+
kubectl --context="${CTX_CLUSTER_EAST}" patch deploy helloworld-v1 --type='merge' \
150+
-p '{"spec": {"template": {"metadata": {"annotations": {"inject.istio.io/templates": "sidecar,spire"}}}}}'
151+
```
152+
153+
1. Deploy V2 in the `west` cluster:
154+
155+
```shell
156+
kubectl apply --context="${CTX_CLUSTER_WEST}" \
157+
-f samples/helloworld/helloworld.yaml \
158+
-l version=v2
159+
kubectl --context="${CTX_CLUSTER_WEST}" patch deploy helloworld-v2 --type='merge' \
160+
-p '{"spec": {"template": {"metadata": {"annotations": {"inject.istio.io/templates": "sidecar,spire"}}}}}'
161+
```
162+
163+
1. Deploy `curl` in the `east` cluster:
164+
165+
```shell
166+
kubectl --context="${CTX_CLUSTER_EAST}" apply -f samples/curl/curl.yaml
167+
kubectl --context="${CTX_CLUSTER_EAST}" patch deploy curl --type='merge' \
168+
-p '{"spec": {"template": {"metadata": {"annotations": {"inject.istio.io/templates": "sidecar,spire"}}}}}'
169+
```
170+
171+
1. Send a few requests and verify that you receive responses from `v1` and `v2`:
172+
173+
```shell
174+
kubectl --context="${CTX_CLUSTER_EAST}" exec deploy/curl -c curl -- curl -v helloworld:5000/hello
175+
```
176+
177+
1. Verify logs - you should see that responses come from `v1` and `v2`:
178+
179+
```text
180+
Hello version: v1, instance: helloworld-v1-f957bfb54-fmqbn
181+
Hello version: v2, instance: helloworld-v2-5f86f7755b-6rhj4
182+
...
183+
```

‎samples/security/spire-trust-domain-federation/cluster-federated-trust-domain.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
apiVersion: spire.spiffe.io/v1alpha1
2+
kind: ClusterFederatedTrustDomain
3+
metadata:
4+
name: ${CLUSTER}
5+
spec:
6+
className: spire-spire
7+
trustDomain: ${CLUSTER}.local
8+
bundleEndpointURL: https://${BUNDLE_ENDPOINT}:8443
9+
bundleEndpointProfile:
10+
type: https_spiffe
11+
endpointSPIFFEID: spiffe://${CLUSTER}.local/spire/server

‎samples/security/spire-trust-domain-federation/istio.tmpl‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
apiVersion: install.istio.io/v1alpha1
2+
kind: IstioOperator
3+
spec:
4+
profile: minimal
5+
meshConfig:
6+
accessLogFile: /dev/stdout
7+
trustDomain: ${LOCAL_CLUSTER}.local
8+
caCertificates:
9+
- spiffeBundleUrl: https://spire-server.spire.svc.cluster.local:8443
10+
trustDomains:
11+
- ${LOCAL_CLUSTER}.local
12+
- spiffeBundleUrl: https://${REMOTE_BUNDLE_ENDPOINT}:8443
13+
trustDomains:
14+
- ${REMOTE_CLUSTER}.local
15+
values:
16+
global:
17+
meshID: ${LOCAL_CLUSTER}-mesh
18+
multiCluster:
19+
clusterName: ${LOCAL_CLUSTER}
20+
network: ${LOCAL_CLUSTER}-network
21+
sidecarInjectorWebhook:
22+
templates:
23+
spire: |
24+
spec:
25+
initContainers:
26+
- name: istio-proxy
27+
volumeMounts:
28+
- name: workload-socket
29+
mountPath: /var/run/secrets/workload-spiffe-uds
30+
volumes:
31+
- name: workload-socket
32+
csi:
33+
driver: "csi.spiffe.io"
34+
readOnly: true
35+
spire-gateway: |
36+
spec:
37+
containers:
38+
- name: istio-proxy
39+
volumeMounts:
40+
- name: workload-socket
41+
mountPath: /var/run/secrets/workload-spiffe-uds
42+
volumes:
43+
- name: workload-socket
44+
csi:
45+
driver: "csi.spiffe.io"
46+
readOnly: true

‎samples/security/spire-trust-domain-federation/spire-base.yaml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
global:
2+
spire:
3+
caSubject:
4+
country: US
5+
organization: istio.io
6+
7+
spire-server:
8+
service:
9+
type: LoadBalancer
10+
federation:
11+
enabled: true
12+
controllerManager:
13+
enabled: true
14+
identities:
15+
clusterSPIFFEIDs:
16+
default:
17+
enabled: false
18+
sidecars:
19+
spiffeIDTemplate: "spiffe://{{ .TrustDomain }}/ns/{{ .PodMeta.Namespace }}/sa/{{ .PodSpec.ServiceAccountName }}"
20+
podSelector:
21+
matchExpressions:
22+
- key: service.istio.io/canonical-name
23+
operator: Exists
24+
oidc-discovery-provider:
25+
enabled: false
26+
test-keys:
27+
enabled: false
28+
29+
spire-agent:
30+
sds:
31+
enabled: true
32+
defaultSVIDName: default
33+
defaultBundleName: "null"
34+
defaultAllBundlesName: ROOTCA
35+
36+
spiffe-oidc-discovery-provider:
37+
enabled: false

‎samples/security/spire-trust-domain-federation/spire-east.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
global:
2+
spire:
3+
clusterName: east
4+
trustDomain: east.local
5+
caSubject:
6+
commonName: east.local
7+
8+
spire-server:
9+
controllerManager:
10+
identities:
11+
clusterSPIFFEIDs:
12+
sidecars:
13+
federatesWith:
14+
- west.local

‎samples/security/spire-trust-domain-federation/spire-west.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
global:
2+
spire:
3+
clusterName: west
4+
trustDomain: west.local
5+
caSubject:
6+
commonName: west.local
7+
8+
spire-server:
9+
controllerManager:
10+
identities:
11+
clusterSPIFFEIDs:
12+
sidecars:
13+
federatesWith:
14+
- east.local

0 commit comments

Comments
 (0)