Skip to content

Commit af221a1

Browse files
jaelliokeithmattix
authored andcommitted
Install guide for ambient multi network
Signed-off-by: Jackie Elliott <[email protected]>
1 parent 169fc7b commit af221a1

File tree

10 files changed

+1510
-0
lines changed

10 files changed

+1510
-0
lines changed
Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
---
2+
title: Install Multicluster
3+
description: Install an Istio mesh in ambient mode across multiple Kubernetes clusters.
4+
weight: 40
5+
aliases:
6+
- /docs/ambient/kubernetes/multicluster-install/
7+
- /docs/ambient/kubernetes/multicluster/
8+
- /docs/ambient/kubernetes/install/multicluster/
9+
- /docs/ambient/install/multicluster/gateways/
10+
- /docs/ambient/install/multicluster/shared/
11+
keywords: [kubernetes,multicluster]
12+
simple_list: true
13+
content_above: true
14+
test: table-of-contents
15+
owner: istio/wg-environments-maintainers
16+
---
17+
18+
{{< boilerplate alpha >}}
19+
20+
Follow this guide to install an Istio {{< gloss "ambient" >}}ambient service mesh{{< /gloss >}}
21+
that spans multiple {{< gloss "cluster" >}}clusters{{< /gloss >}}.
22+
23+
## Current Status and Limitations
24+
25+
{{< warning >}}
26+
**Ambient multicluster is currently in alpha status** and has significant limitations.
27+
This feature is under active development and should not be used in production environments.
28+
{{< /warning >}}
29+
30+
Before proceeding with ambient multicluster installation, it's critical to understand
31+
the current state and limitations of this feature:
32+
33+
### Supported Configurations
34+
35+
Currently, ambient multicluster **only supports**:
36+
- **Multi-network topologies** with multiple primary clusters
37+
- **Double HBONE encapsulation** for cross-cluster traffic
38+
- **Universal waypoint deployments** across all clusters with identical names
39+
40+
### Critical Limitations
41+
42+
#### Network Topology Restrictions
43+
- **Multi-cluster single-network configurations are untested, broken, and pre-experimental**
44+
- Do not attempt to deploy ambient across clusters that share the same network
45+
- Only multi-network configurations are supported
46+
47+
#### Control Plane Limitations
48+
- **Primary remote configuration is not currently supported**
49+
- You can only have multiple primary clusters
50+
- Configurations with one or more remote clusters will not work correctly
51+
52+
#### Waypoint Requirements
53+
- **Universal waypoint deployments are assumed across clusters**
54+
- All clusters must have identically named waypoint deployments
55+
- Waypoint configurations must be synchronized manually across clusters
56+
- Traffic routing relies on consistent waypoint naming conventions
57+
58+
#### Service Visibility and Scoping
59+
- **Service scope configurations are not read from across clusters**
60+
- Only the local cluster's service scope configuration is used as the source of truth
61+
- Remote cluster service scopes are ignored, which can lead to unexpected traffic behavior
62+
- Cross-cluster service discovery may not respect intended service boundaries
63+
64+
#### Gateway Limitations
65+
- **Ambient east-west gateways only support double HBONE**
66+
- Cannot expose `istiod` across networks using ambient east-west gateways
67+
68+
69+
{{< tip >}}
70+
As ambient multicluster matures, many of these limitations will be addressed.
71+
Check the [Istio release notes](https://istio.io/latest/news/) for updates on
72+
ambient multicluster capabilities.
73+
{{< /tip >}}
Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
---
2+
title: Before you begin
3+
description: Initial steps before installing Istio on multiple clusters.
4+
weight: 1
5+
keywords: [kubernetes,multicluster]
6+
test: n/a
7+
owner: istio/wg-environments-maintainers
8+
---
9+
10+
{{< boilerplate alpha >}}
11+
12+
Before you begin a multicluster installation, review the
13+
[deployment models guide](/docs/ops/deployment/deployment-models)
14+
which describes the foundational concepts used throughout this guide.
15+
16+
In addition, review the requirements and perform the initial steps below.
17+
18+
## Requirements
19+
20+
### Cluster
21+
22+
This guide requires that you have two Kubernetes clusters with any of the
23+
[supported Kubernetes versions:](/docs/releases/supported-releases#support-status-of-istio-releases) {{< supported_kubernetes_versions >}}.
24+
25+
### API Server Access
26+
27+
The API Server in each cluster must be accessible to the other clusters in the
28+
mesh. Many cloud providers make API Servers publicly accessible via network
29+
load balancers (NLB). The ambient east-west gateway cannot be used to expose
30+
the API server as it only supports double HBONE traffic. A non-ambient
31+
[east-west](https://en.wikipedia.org/wiki/East-west_traffic) gateway could be
32+
used to enable access to the API Server.
33+
34+
## Environment Variables
35+
36+
This guide will refer to two clusters: `cluster1` and `cluster2`. The following
37+
environment variables will be used throughout to simplify the instructions:
38+
39+
Variable | Description
40+
-------- | -----------
41+
`CTX_CLUSTER1` | The context name in the default [Kubernetes configuration file](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/) used for accessing the `cluster1` cluster.
42+
`CTX_CLUSTER2` | The context name in the default [Kubernetes configuration file](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/) used for accessing the `cluster2` cluster.
43+
44+
Set the two variables before proceeding:
45+
46+
{{< text syntax=bash snip_id=none >}}
47+
$ export CTX_CLUSTER1=<your cluster1 context>
48+
$ export CTX_CLUSTER2=<your cluster2 context>
49+
{{< /text >}}
50+
51+
## Configure Trust
52+
53+
A multicluster service mesh deployment requires that you establish trust
54+
between all clusters in the mesh. Depending on the requirements for your
55+
system, there may be multiple options available for establishing trust.
56+
See [certificate management](/docs/tasks/security/cert-management/) for
57+
detailed descriptions and instructions for all available options.
58+
Depending on which option you choose, the installation instructions for
59+
Istio may change slightly.
60+
61+
This guide will assume that you use a common root to generate intermediate
62+
certificates for each primary cluster.
63+
Follow the [instructions](/docs/tasks/security/cert-management/plugin-ca-cert/)
64+
to generate and push a CA certificate secret to both the `cluster1` and `cluster2`
65+
clusters.
66+
67+
{{< tip >}}
68+
If you currently have a single cluster with a self-signed CA (as described
69+
in [Getting Started](/docs/setup/getting-started/)), you need to
70+
change the CA using one of the methods described in
71+
[certificate management](/docs/tasks/security/cert-management/). Changing the
72+
CA typically requires reinstalling Istio. The installation instructions
73+
below may have to be altered based on your choice of CA.
74+
{{< /tip >}}
75+
76+
## Next steps
77+
78+
You're now ready to install an Istio ambient mesh across multiple clusters. The
79+
particular steps will depend on your requirements for network and
80+
control plane topology.
81+
82+
Choose the installation that best fits your needs:
83+
84+
- [Install Multi-Primary on Different Networks](/docs/ambient/install/multicluster/multi-primary_multi-network)
85+
86+
{{< tip >}}
87+
If you plan on installing Istio multi-cluster using Helm, follow the
88+
[Helm prerequisites](/docs/setup/install/helm/#prerequisites) in the Helm install guide first.
89+
{{< /tip >}}
Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
#!/usr/bin/env bash
2+
# shellcheck disable=SC1090,SC2034,SC2154
3+
4+
# Copyright Istio Authors
5+
#
6+
# Licensed under the Apache License, Version 2.0 (the "License");
7+
# you may not use this file except in compliance with the License.
8+
# You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
18+
# Initialize KUBECONFIG_FILES and KUBE_CONTEXTS
19+
_set_kube_vars
20+
21+
source content/en/docs/ambient/install/multicluster/verify/snips.sh
22+
23+
# set_single_network_vars initializes all variables for a single network config.
24+
function set_single_network_vars
25+
{
26+
export KUBECONFIG_CLUSTER1="${KUBECONFIG_FILES[0]}"
27+
export KUBECONFIG_CLUSTER2="${KUBECONFIG_FILES[1]}"
28+
export CTX_CLUSTER1="${KUBE_CONTEXTS[0]}"
29+
export CTX_CLUSTER2="${KUBE_CONTEXTS[1]}"
30+
}
31+
32+
# set_multi_network_vars initializes all variables for a multi-network config.
33+
function set_multi_network_vars
34+
{
35+
export KUBECONFIG_CLUSTER1="${KUBECONFIG_FILES[0]}"
36+
export KUBECONFIG_CLUSTER2="${KUBECONFIG_FILES[2]}"
37+
export CTX_CLUSTER1="${KUBE_CONTEXTS[0]}"
38+
export CTX_CLUSTER2="${KUBE_CONTEXTS[2]}"
39+
}
40+
41+
# configure_trust creates a hierarchy of
42+
function configure_trust
43+
{
44+
# Keeps the certs under a separate directory.
45+
mkdir -p certs
46+
pushd certs || exit
47+
48+
# Create the root cert.
49+
make -f ../tools/certs/Makefile.selfsigned.mk root-ca
50+
51+
# Create and deploy intermediate certs for cluster1 and cluster2.
52+
make -f ../tools/certs/Makefile.selfsigned.mk cluster1-cacerts
53+
make -f ../tools/certs/Makefile.selfsigned.mk cluster2-cacerts
54+
55+
# Create the istio-system namespace in each cluster so that we can create the secrets.
56+
kubectl --context="$CTX_CLUSTER1" create namespace istio-system
57+
kubectl --context="$CTX_CLUSTER2" create namespace istio-system
58+
59+
# Deploy secret to each cluster
60+
kubectl --context="$CTX_CLUSTER1" create secret generic cacerts -n istio-system \
61+
--from-file=cluster1/ca-cert.pem \
62+
--from-file=cluster1/ca-key.pem \
63+
--from-file=cluster1/root-cert.pem \
64+
--from-file=cluster1/cert-chain.pem
65+
kubectl --context="$CTX_CLUSTER2" create secret generic cacerts -n istio-system \
66+
--from-file=cluster2/ca-cert.pem \
67+
--from-file=cluster2/ca-key.pem \
68+
--from-file=cluster2/root-cert.pem \
69+
--from-file=cluster2/cert-chain.pem
70+
71+
popd || exit # Return to the previous directory.
72+
}
73+
74+
# cleanup_istioctl removes all resources created by the tests with istioctl.
75+
function cleanup_istioctl
76+
{
77+
# Remove temp files.
78+
rm -f cluster1.yaml cluster2.yaml certs
79+
80+
# Cleanup both clusters concurrently
81+
cleanup_cluster1_istioctl &
82+
cleanup_cluster2_istioctl &
83+
wait
84+
}
85+
86+
# cleanup_cluster1_istioctl removes the istio-system and sample namespaces on CLUSTER1 with istioctl.
87+
function cleanup_cluster1_istioctl
88+
{
89+
echo y | istioctl uninstall --revision=default --context="${CTX_CLUSTER1}"
90+
kubectl delete ns istio-system sample --context="${CTX_CLUSTER1}" --ignore-not-found
91+
}
92+
93+
# cleanup_cluster2_istioctl removes the istio-system and sample namespaces on CLUSTER2 with istioctl.
94+
function cleanup_cluster2_istioctl
95+
{
96+
echo y | istioctl uninstall --revision=default --context="${CTX_CLUSTER2}"
97+
kubectl delete ns istio-system sample --context="${CTX_CLUSTER2}" --ignore-not-found
98+
}
99+
100+
# verify_load_balancing verifies that traffic is load balanced properly
101+
# between CLUSTER1 and CLUSTER2.
102+
function verify_load_balancing
103+
{
104+
# Verify istiod is synced
105+
echo "Verifying istiod is synced to remote cluster."
106+
_verify_like snip_verify_multicluster_1 "$snip_verify_multicluster_1_out"
107+
108+
# Deploy the HelloWorld service.
109+
snip_deploy_the_helloworld_service_1
110+
snip_deploy_the_helloworld_service_2
111+
snip_deploy_the_helloworld_service_3
112+
113+
# Deploy HelloWorld v1 and v2
114+
snip_deploy_helloworld_v1_1
115+
snip_deploy_helloworld_v2_1
116+
117+
# Deploy curl
118+
snip_deploy_curl_1
119+
120+
# Wait for all the deployments.
121+
_wait_for_deployment sample helloworld-v1 "${CTX_CLUSTER1}"
122+
_wait_for_deployment sample curl "${CTX_CLUSTER1}"
123+
_wait_for_deployment sample helloworld-v2 "${CTX_CLUSTER2}"
124+
_wait_for_deployment sample curl "${CTX_CLUSTER2}"
125+
126+
# Verify everything is deployed as expected.
127+
VERIFY_TIMEOUT=0 # Don't retry.
128+
echo "Verifying helloworld v1 deployment"
129+
_verify_like snip_deploy_helloworld_v1_2 "$snip_deploy_helloworld_v1_2_out"
130+
echo "Verifying helloworld v2 deployment"
131+
_verify_like snip_deploy_helloworld_v2_2 "$snip_deploy_helloworld_v2_2_out"
132+
echo "Verifying curl deployment in ${CTX_CLUSTER1}"
133+
_verify_like snip_deploy_curl_2 "$snip_deploy_curl_2_out"
134+
echo "Verifying curl deployment in ${CTX_CLUSTER2}"
135+
_verify_like snip_deploy_curl_3 "$snip_deploy_curl_3_out"
136+
unset VERIFY_TIMEOUT # Restore default
137+
138+
local EXPECTED_RESPONSE_FROM_CLUSTER1="Hello version: v1, instance:"
139+
local EXPECTED_RESPONSE_FROM_CLUSTER2="Hello version: v2, instance:"
140+
141+
# Verify we hit both clusters from CLUSTER1
142+
echo "Verifying load balancing from ${CTX_CLUSTER1}"
143+
_verify_contains snip_verifying_crosscluster_traffic_1 "$EXPECTED_RESPONSE_FROM_CLUSTER1"
144+
_verify_contains snip_verifying_crosscluster_traffic_1 "$EXPECTED_RESPONSE_FROM_CLUSTER2"
145+
146+
# Verify we hit both clusters from CLUSTER2
147+
echo "Verifying load balancing from ${CTX_CLUSTER2}"
148+
_verify_contains snip_verifying_crosscluster_traffic_3 "$EXPECTED_RESPONSE_FROM_CLUSTER1"
149+
_verify_contains snip_verifying_crosscluster_traffic_3 "$EXPECTED_RESPONSE_FROM_CLUSTER2"
150+
}
151+
152+
# For Helm multi-cluster installation steps
153+
154+
function create_istio_system_ns
155+
{
156+
snip_create_istio_system_namespace_cluster_1
157+
snip_create_istio_system_namespace_cluster_2
158+
}
159+
160+
function setup_helm_repo
161+
{
162+
snip_setup_helm_repo_cluster_1
163+
snip_setup_helm_repo_cluster_2
164+
}
165+
166+
snip_create_istio_system_namespace_cluster_1() {
167+
kubectl create namespace istio-system --context "${CTX_CLUSTER1}"
168+
}
169+
170+
snip_create_istio_system_namespace_cluster_2() {
171+
kubectl create namespace istio-system --context "${CTX_CLUSTER2}"
172+
}
173+
174+
snip_setup_helm_repo_cluster_1() {
175+
helm repo add istio https://istio-release.storage.googleapis.com/charts --kube-context "${CTX_CLUSTER1}"
176+
helm repo update --kube-context "${CTX_CLUSTER1}"
177+
}
178+
179+
snip_setup_helm_repo_cluster_2() {
180+
helm repo add istio https://istio-release.storage.googleapis.com/charts --kube-context "${CTX_CLUSTER2}"
181+
helm repo update --kube-context "${CTX_CLUSTER2}"
182+
}
183+
184+
snip_delete_sample_ns_cluster_1() {
185+
kubectl delete namespace sample --context "${CTX_CLUSTER1}"
186+
}
187+
188+
snip_delete_sample_ns_cluster_2() {
189+
kubectl delete namespace sample --context "${CTX_CLUSTER2}"
190+
}

content/en/docs/ambient/install/multicluster/multi-primary_multi-network/arch.svg

Lines changed: 1 addition & 0 deletions
Loading

0 commit comments

Comments
 (0)