Skip to content

Commit 1849f78

Browse files
luksaluksa
authored
Add documentation for running multiple meshes on a single cluster (openshift-service-mesh#506)
* Add documentation for running multiple meshes on a single cluster Signed-off-by: Marko Lukša <[email protected]> * Fix apiVersion Signed-off-by: Marko Lukša <[email protected]> --------- Signed-off-by: Marko Lukša <[email protected]>
1 parent 9579529 commit 1849f78

File tree

1 file changed

+252
-0
lines changed

1 file changed

+252
-0
lines changed

‎docs/README.md‎

Lines changed: 252 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,14 @@
2020
- [Example using the InPlace strategy](#example-using-the-inplace-strategy)
2121
- [RevisionBased](#revisionbased)
2222
- [Example using the RevisionBased strategy](#example-using-the-revisionbased-strategy)
23+
- [Multiple meshes on a single cluster](#multiple-meshes-on-a-single-cluster)
24+
- [Prerequisites](#prerequisites)
25+
- [Installation Steps](#installation-steps)
26+
- [Deploying the control planes](#deploying-the-control-planes)
27+
- [Deploying the applications](#deploying-the-applications)
28+
- [Validation](#validation)
29+
- [Checking application to control plane mapping](#checking-application-to-control-plane-mapping)
30+
- [Checking application connectivity](#checking-application-connectivity)
2331
- [Multi-cluster](#multi-cluster)
2432
- [Prerequisites](#prerequisites)
2533
- [Common Setup](#common-setup)
@@ -468,6 +476,250 @@ Steps:
468476
```
469477
The old `IstioRevision` resource and the old control plane will be deleted when the grace period specified in the `Istio` resource field `spec.updateStrategy.inactiveRevisionDeletionGracePeriodSeconds` expires.
470478

479+
## Multiple meshes on a single cluster
480+
481+
The Sail Operator supports running multiple meshes on a single cluster and associating each workload with a specific mesh.
482+
Each mesh is managed by a separate control plane.
483+
484+
Applications are installed in multiple namespaces, and each namespace is associated with one of the control planes through its labels.
485+
The `istio.io/rev` label determines which control plane injects the sidecar proxy into the application pods.
486+
Additional namespace labels determine whether the control plane discovers and manages the resources in the namespace.
487+
A control plane will discover and manage only those namespaces that match the discovery selectors configured on the control plane.
488+
Additionally, discovery selectors determine which control plane creates the `istio-ca-root-cert` ConfigMap in which namespace.
489+
490+
Currently, discovery selectors in multiple control planes must be configured so that they don't overlap (i.e. the discovery selectors of two control planes don't match the same namespace).
491+
Each control plane must be deployed in a separate Kubernetes namespace.
492+
493+
This guide explains how to set up two meshes: `mesh1` and `mesh2` in namespaces `istio-system1` and `istio-system2`, respectively, and three application namespaces: `app1`, `app2a`, and `app2b`.
494+
Mesh 1 will manage namespace `app1`, and Mesh 2 will manage namespaces `app2a` and `app2b`.
495+
Because each mesh will use its own root certificate authority and configured to use a peer authentication policy with the `STRICT` mTLS mode, the communication between the two meshes will not be allowed.
496+
497+
### Prerequisites
498+
499+
- Install [istioctl](common/install-istioctl-tool.md).
500+
- Kubernetes 1.23 cluster.
501+
- kubeconfig file with a context for the Kubernetes cluster.
502+
- Install the Sail Operator and the Sail CRDs to the cluster.
503+
504+
### Installation Steps
505+
506+
#### Deploying the control planes
507+
508+
1. Create the system namespace `istio-system1` and deploy the `mesh1` control plane in it.
509+
```sh
510+
$ kubectl create namespace istio-system1
511+
$ kubectl label ns istio-system1 mesh=mesh1
512+
$ kubectl apply -f - <<EOF
513+
apiVersion: sailoperator.io/v1alpha1
514+
kind: Istio
515+
metadata:
516+
name: mesh1
517+
spec:
518+
namespace: istio-system1
519+
version: v1.24.0
520+
values:
521+
meshConfig:
522+
discoverySelectors:
523+
- matchLabels:
524+
mesh: mesh1
525+
EOF
526+
```
527+
528+
2. Create the system namespace `istio-system2` and deploy the `mesh2` control plane in it.
529+
```sh
530+
$ kubectl create namespace istio-system2
531+
$ kubectl label ns istio-system2 mesh=mesh2
532+
$ kubectl apply -f - <<EOF
533+
apiVersion: sailoperator.io/v1alpha1
534+
kind: Istio
535+
metadata:
536+
name: mesh2
537+
spec:
538+
namespace: istio-system2
539+
version: v1.24.0
540+
values:
541+
meshConfig:
542+
discoverySelectors:
543+
- matchLabels:
544+
mesh: mesh2
545+
EOF
546+
```
547+
548+
3. Create a peer authentication policy that only allows mTLS communication within each mesh.
549+
```sh
550+
$ kubectl apply -f - <<EOF
551+
apiVersion: security.istio.io/v1
552+
kind: PeerAuthentication
553+
metadata:
554+
name: default
555+
namespace: istio-system1
556+
spec:
557+
mtls:
558+
mode: STRICT
559+
EOF
560+
561+
$ kubectl apply -f - <<EOF
562+
apiVersion: security.istio.io/v1
563+
kind: PeerAuthentication
564+
metadata:
565+
name: default
566+
namespace: istio-system2
567+
spec:
568+
mtls:
569+
mode: STRICT
570+
EOF
571+
```
572+
573+
#### Verifying the control planes
574+
575+
1. Check the labels on the control plane namespaces:
576+
```sh
577+
$ kubectl get ns -l mesh -L mesh
578+
NAME STATUS AGE MESH
579+
istio-system1 Active 106s mesh1
580+
istio-system2 Active 105s mesh2
581+
```
582+
583+
2. Check the control planes are `Healthy`:
584+
```sh
585+
$ kubectl get istios
586+
NAME REVISIONS READY IN USE ACTIVE REVISION STATUS VERSION AGE
587+
mesh1 1 1 0 mesh1 Healthy v1.24.0 84s
588+
mesh2 1 1 0 mesh2 Healthy v1.24.0 77s
589+
```
590+
591+
3. Confirm that the validation and mutation webhook configurations exist for both meshes:
592+
```sh
593+
$ kubectl get validatingwebhookconfigurations
594+
NAME WEBHOOKS AGE
595+
istio-validator-mesh1-istio-system1 1 2m45s
596+
istio-validator-mesh2-istio-system2 1 2m38s
597+
598+
$ kubectl get mutatingwebhookconfigurations
599+
NAME WEBHOOKS AGE
600+
istio-sidecar-injector-mesh1-istio-system1 2 5m55s
601+
istio-sidecar-injector-mesh2-istio-system2 2 5m48s
602+
```
603+
604+
#### Deploying the applications
605+
606+
1. Create three application namespaces:
607+
```sh
608+
$ kubectl create ns app1
609+
$ kubectl create ns app2a
610+
$ kubectl create ns app2b
611+
```
612+
613+
2. Label each namespace to enable discovery by the corresponding control plane:
614+
```sh
615+
$ kubectl label ns app1 mesh=mesh1
616+
$ kubectl label ns app2a mesh=mesh2
617+
$ kubectl label ns app2b mesh=mesh2
618+
```
619+
620+
3. Label each namespace to enable injection by the corresponding control plane:
621+
```sh
622+
$ kubectl label ns app1 istio.io/rev=mesh1
623+
$ kubectl label ns app2a istio.io/rev=mesh2
624+
$ kubectl label ns app2b istio.io/rev=mesh2
625+
```
626+
627+
4. Deploy the `curl` and `httpbin` sample applications in each namespace:
628+
```sh
629+
$ kubectl -n app1 apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml
630+
$ kubectl -n app1 apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/httpbin/httpbin.yaml
631+
632+
$ kubectl -n app2a apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml
633+
$ kubectl -n app2a apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/httpbin/httpbin.yaml
634+
635+
$ kubectl -n app2b apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml
636+
$ kubectl -n app2b apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/httpbin/httpbin.yaml
637+
```
638+
639+
5. Confirm that a sidecar has been injected into each of the application pods. The value `2/2` should be displayed in the `READY` column for each pod, as in the following example:
640+
```sh
641+
$ kubectl get pods -n app1
642+
NAME READY STATUS RESTARTS AGE
643+
curl-5b549b49b8-mg7nl 2/2 Running 0 102s
644+
httpbin-7b549f7859-h6hnk 2/2 Running 0 89s
645+
646+
$ kubectl get pods -n app2a
647+
NAME READY STATUS RESTARTS AGE
648+
curl-5b549b49b8-2hlvm 2/2 Running 0 2m3s
649+
httpbin-7b549f7859-bgblg 2/2 Running 0 110s
650+
651+
$ kubectl get pods -n app2b
652+
NAME READY STATUS RESTARTS AGE
653+
curl-5b549b49b8-xnzzk 2/2 Running 0 2m9s
654+
httpbin-7b549f7859-7k5gf 2/2 Running 0 118s
655+
```
656+
657+
### Validation
658+
659+
#### Checking application to control plane mapping
660+
661+
Use the `istioctl ps` command to confirm that the application pods are connected to the correct control plane.
662+
663+
The `curl` and `httpbin` pods in namespace `app1` should be connected to the control plane in namespace `istio-system1`, as shown in the following example (note the `.app1` suffix in the `NAME` column):
664+
665+
```sh
666+
$ istioctl ps -i istio-system1
667+
NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION
668+
curl-5b549b49b8-mg7nl.app1 Kubernetes SYNCED (4m40s) SYNCED (4m40s) SYNCED (4m31s) SYNCED (4m40s) IGNORED istiod-mesh1-5df45b97dd-tf2wl 1.24.0
669+
httpbin-7b549f7859-h6hnk.app1 Kubernetes SYNCED (4m31s) SYNCED (4m31s) SYNCED (4m31s) SYNCED (4m31s) IGNORED istiod-mesh1-5df45b97dd-tf2wl 1.24.0
670+
```
671+
672+
The pods in namespaces `app2a` and `app2b` should be connected to the control plane in namespace `istio-system2`:
673+
674+
```sh
675+
$ istioctl ps -i istio-system2
676+
NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION
677+
curl-5b549b49b8-2hlvm.app2a Kubernetes SYNCED (4m37s) SYNCED (4m37s) SYNCED (4m31s) SYNCED (4m37s) IGNORED istiod-mesh2-59f6b874fb-mzxqw 1.24.0
678+
curl-5b549b49b8-xnzzk.app2b Kubernetes SYNCED (4m37s) SYNCED (4m37s) SYNCED (4m31s) SYNCED (4m37s) IGNORED istiod-mesh2-59f6b874fb-mzxqw 1.24.0
679+
httpbin-7b549f7859-7k5gf.app2b Kubernetes SYNCED (4m31s) SYNCED (4m31s) SYNCED (4m31s) SYNCED (4m31s) IGNORED istiod-mesh2-59f6b874fb-mzxqw 1.24.0
680+
httpbin-7b549f7859-bgblg.app2a Kubernetes SYNCED (4m32s) SYNCED (4m32s) SYNCED (4m31s) SYNCED (4m32s) IGNORED istiod-mesh2-59f6b874fb-mzxqw 1.24.0
681+
```
682+
683+
#### Checking application connectivity
684+
685+
As both meshes are configured to use the `STRICT` mTLS peer authentication mode, the applications in namespace `app1` should not be able to communicate with the applications in namespaces `app2a` and `app2b`, and vice versa.
686+
To test whether the `curl` pod in namespace `app2a` can connect to the `httpbin` service in namespace `app1`, run the following commands:
687+
688+
```sh
689+
$ kubectl -n app2a exec deploy/curl -c curl -- curl -sIL http://httpbin.app1:8000
690+
HTTP/1.1 503 Service Unavailable
691+
content-length: 95
692+
content-type: text/plain
693+
date: Fri, 29 Nov 2024 08:58:28 GMT
694+
server: envoy
695+
```
696+
697+
As expected, the response indicates that the connection was not successful.
698+
In contrast, the same pod should be able to connect to the `httpbin` service in namespace `app2b`, because they are part of the same mesh:
699+
700+
```sh
701+
$ kubectl -n app2a exec deploy/curl -c curl -- curl -sIL http://httpbin.app2b:8000
702+
HTTP/1.1 200 OK
703+
access-control-allow-credentials: true
704+
access-control-allow-origin: *
705+
content-security-policy: default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' camo.githubusercontent.com
706+
content-type: text/html; charset=utf-8
707+
date: Fri, 29 Nov 2024 08:57:52 GMT
708+
x-envoy-upstream-service-time: 0
709+
server: envoy
710+
transfer-encoding: chunked
711+
```
712+
713+
### Cleanup
714+
715+
To clean up the resources created in this guide, delete the `Istio` resources and the namespaces:
716+
717+
```sh
718+
$ kubectl delete istio mesh1 mesh2
719+
$ kubectl delete ns istio-system1 istio-system2 app1 app2a app2b
720+
```
721+
722+
471723
## Multi-cluster
472724

473725
You can use the Sail Operator and the Sail CRDs to manage a multi-cluster Istio deployment. The following instructions are adapted from the [Istio multi-cluster documentation](https://istio.io/latest/docs/setup/install/multicluster/) to demonstrate how you can setup the various deployment models with Sail. Please familiarize yourself with the different [deployment models](https://istio.io/latest/docs/ops/deployment/deployment-models/) before starting.

0 commit comments

Comments
 (0)