Skip to content

Commit f0c6fa9

Browse files
author
Kristiaan Hoogendoorn
committed
sidecar containers
Some documentation on how someone could run the Istio proxy as a container on their preferred VM platform. - A users can iterate on the existing VM documentation (and is required to understand it.) - The use is responsible for automation around it.
1 parent 4dac6f0 commit f0c6fa9

File tree

3 files changed

+228
-1
lines changed

3 files changed

+228
-1
lines changed

‎.spelling

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -456,6 +456,7 @@ DNS
456456
DNS_AUTO_ALLOCATE
457457
docker-compose's
458458
docker.io
459+
Podman
459460
doc_type
460461
dogfood
461462
Dolfi

‎content/en/docs/setup/install/virtual-machine/index.md renamed to ‎content/en/docs/setup/install/virtual-machine/_index.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,9 @@ keywords:
77
- virtual-machine
88
- gateways
99
- vms
10+
skip_list: true
11+
test: true
1012
owner: istio/wg-environments-maintainers
11-
test: yes
1213
---
1314

1415
Follow this guide to deploy Istio and connect a virtual machine to it.
@@ -449,6 +450,7 @@ For more information about virtual machines:
449450

450451
* [Debugging Virtual Machines](/docs/ops/diagnostic-tools/virtual-machines/) to troubleshoot issues with virtual machines.
451452
* [Bookinfo with a Virtual Machine](/docs/examples/virtual-machines/) to set up an example deployment of virtual machines.
453+
* [Running Containerized proxies](/docs/setup/install/virtual-machine/containerized/) to deploy containerized sidecars on virtual machines.
452454

453455
## Uninstall
454456

‎content/en/docs/setup/install/virtual-machine/containerized/index.md

Lines changed: 224 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,224 @@
1+
---
2+
title: Running Istio-proxy containers
3+
description: Running containerized istio-proxy on Virtual Machines.
4+
weight: 70
5+
keywords:
6+
- kubernetes
7+
- virtual-machine
8+
- gateways
9+
- vms
10+
- docker
11+
- containers
12+
owner: istio/wg-environments-maintainers
13+
test: yes
14+
15+
---
16+
Follow this guide to run the Istio-proxy as a container instead of the Istio virtual machine integration runtime allowing more flexibility of the underlying platform.
17+
18+
## Prerequisites
19+
20+
1. Follow the steps described on [Virtual Machine Installation](/docs/setup/install/virtual-machine/) until [configure-the-virtual-machine](/docs/setup/install/virtual-machine/#configure-the-virtual-machine).
21+
1. Have your [guide environment](/docs/setup/install/virtual-machine/#prepare-the-guide-environment) prepared.
22+
1. Learn about [Virtual Machine Architecture](/docs/ops/deployment/vm-architecture/) to gain an understanding of the high level architecture of Istio's virtual machine integration.
23+
24+
## Host OS requirements
25+
26+
1. Have a container runtime installed like [Docker](https://docs.docker.com/engine/install/) (used in this guide) or [Podman](https://podman.io/docs/installation).
27+
1. Ability to run a container with `--network=host`- allowing it to interact with the hosts Iptables.
28+
1. Ability to run a container with the capability: `NET_ADMIN` - allowing it to update Iptables.
29+
1. Reserve UID `1337` for the user: `istio-proxy`.
30+
31+
## Overview
32+
33+
Installing the Istio-proxy package comes with a start-[script]({{< github_blob >}}tools/packaging/common/istio-start.sh) to bootstrap some final variables
34+
and runs [istio-iptables](/docs/reference/commands/pilot-agent/#pilot-agent-istio-iptables) and [istio-clean-iptables](/docs/reference/commands/pilot-agent/#pilot-agent-istio-clean-iptables)
35+
to correctly configure `iptables` before starting the [istio-proxy](docs/reference/commands/pilot-agent/#pilot-agent-proxy) itself.
36+
This guide will cover this extra configurations to run the Istio-proxy as a sidecar-container.
37+
38+
## Extra configuration
39+
40+
Like mentioned above, extra configuration to `cluster.env` and `mesh.yaml` is required. This is an addition on the already generated configuration via [Virtual Machine Installation](/docs/setup/install/virtual-machine/#create-files-to-transfer-to-the-virtual-machine):
41+
42+
1. setup some extra environment variables:
43+
44+
{{< text bash >}}
45+
$ INSTANCE_IP="<the primary IP of the VM>"
46+
$ POD_NAME="<hostname of the VM (not FQDN)>"
47+
$ SERVICECLUSTER="${VM_APP}.${VM_NAMESPACE}"
48+
{{< /text >}}
49+
50+
1. Update `cluster.env`:
51+
52+
{{< text bash >}}
53+
$ cat <<EOF >> ${WORK_DIR}/cluster.env
54+
INSTANCE_IP=${INSTANCE_IP}
55+
ISTIO_CLUSTER_CONFIG=./var/lib/istio/envoy/cluster.env
56+
POD_NAME=${POD_NAME}
57+
OUTPUT_CERTS=./etc/certs
58+
PROV_CERT=./etc/certs
59+
EOF
60+
{{< /text >}}
61+
62+
1. Update `mesh.yaml`:
63+
64+
{{< text bash >}}
65+
$ cat <<EOF >> ${WORK_DIR}/mesh.yaml
66+
serviceCluster: ${SERVICECLUSTER}
67+
EOF
68+
{{< /text >}}
69+
70+
## Prepare the machine
71+
72+
Run the following commands on the virtual machine:
73+
74+
1. Securely transfer the files from `"${WORK_DIR}"`
75+
to the virtual machine. How you choose to securely transfer those files should be done with consideration for
76+
your information security policies. For convenience in this guide, transfer all of the required files to `"${HOME}"` in the virtual machine.
77+
78+
1. Install the root certificate at `/etc/certs`:
79+
80+
{{< text bash >}}
81+
$ sudo mkdir -p /etc/certs
82+
$ sudo cp "${HOME}"/root-cert.pem /etc/certs/root-cert.pem
83+
{{< /text >}}
84+
85+
1. Install the token at `/var/run/secrets/tokens`:
86+
87+
{{< text bash >}}
88+
$ sudo mkdir -p /var/run/secrets/tokens
89+
$ sudo cp "${HOME}"/istio-token /var/run/secrets/tokens/istio-token
90+
{{< /text >}}
91+
92+
1. Install `cluster.env` within the directory `/var/lib/istio/envoy/`:
93+
94+
{{< text bash >}}
95+
$ sudo mkdir -p /var/lib/istio/envoy
96+
$ sudo cp "${HOME}"/cluster.env /var/lib/istio/envoy/cluster.env
97+
{{< /text >}}
98+
99+
1. Install the [Mesh Config](/docs/reference/config/istio.mesh.v1alpha1/#MeshConfig) to `/etc/istio/config/mesh`:
100+
101+
{{< text bash >}}
102+
$ sudo mkdir -p /etc/istio/config
103+
$ sudo cp "${HOME}"/mesh.yaml /etc/istio/config/mesh
104+
{{< /text >}}
105+
106+
1. Add the istiod host to `/etc/hosts`:
107+
108+
{{< text bash >}}
109+
$ sudo sh -c 'cat $(eval echo ~$SUDO_USER)/hosts >> /etc/hosts'
110+
{{< /text >}}
111+
112+
1. Transfer ownership of the files in `/etc/certs/` and `/var/lib/istio/envoy/` to the Istio proxy:
113+
114+
{{< text bash >}}
115+
$ sudo mkdir -p /etc/istio/proxy
116+
$ sudo chown -R istio-proxy /var/lib/istio /etc/certs /etc/istio/proxy /etc/istio/config /var/run/secrets /etc/certs/root-cert.pem
117+
{{< /text >}}
118+
119+
1. Docker does not exclude quotes when supplying a `--env-file`, remove them ourselves:
120+
121+
{{< tabset category-name="quotes" >}}
122+
123+
{{< tab name="macOS" category-value="mac" >}}
124+
125+
{{< text bash >}}
126+
$ sed -i '' "s/'//g" ${WORK_DIR}/cluster.env
127+
{{< /text >}}
128+
129+
{{< /tab >}}
130+
131+
{{< tab name="Linux" category-value="linux" >}}
132+
133+
{{< text bash >}}
134+
$ sed -i "s/'//g" ${WORK_DIR}/cluster.env
135+
{{< /text >}}
136+
137+
{{< /tab >}}
138+
139+
{{< /tabset >}}
140+
141+
## Start the sidecar container on the virtual machine
142+
143+
1. Configure iptables to redirect all traffic from the VM to the Istio sidecar:
144+
145+
{{< text bash >}}
146+
$ docker run \
147+
--rm \
148+
--cap-add=NET_ADMIN \
149+
--entrypoint="" \
150+
--network host \
151+
--env-file /var/lib/istio/envoy/cluster.env \
152+
--name istio-init \
153+
istio/proxyv2:{{< istio_full_version >}} \
154+
/bin/bash -c "update-alternatives --set iptables /usr/sbin/iptables-nft && \
155+
update-alternatives --set ip6tables /usr/sbin/ip6tables-nft && \
156+
/usr/local/bin/pilot-agent istio-iptables"
157+
{{< /text >}}
158+
159+
{{< warning >}}
160+
The proxyv2 image is configured to use iptables-legacy, hence we need to do some trickery with `update-alternatives` to consult the correct iptables endpoint.
161+
{{< /warning >}}
162+
163+
1. Start the Istio-proxy container:
164+
165+
{{< text bash >}}
166+
$ docker run \
167+
-u '1337:1337' \
168+
--net host \
169+
-d \
170+
--env-file /var/lib/istio/envoy/cluster.env \
171+
-v /var/run/secrets/:/var/run/secrets \
172+
-v /var/lib/istio/config:/var/lib/istio/config \
173+
-v /var/lib/istio/proxy:/var/lib/istio/proxy \
174+
-v /etc/certs:/etc/certs \
175+
-v /etc/istio:/etc/istio \
176+
--name istio-proxy \
177+
istio/proxyv2:{{< istio_full_version >}} \
178+
proxy --concurrency 2 --log_as_json
179+
{{< /text >}}
180+
181+
## Verify Istio-proxy container
182+
183+
1. Check the docker logs for any potential errors:
184+
185+
{{< text bash >}}
186+
$ docker logs istio-proxy
187+
{{< /text >}}
188+
189+
1. Verify if the side car is registered:
190+
191+
{{< text bash >}}
192+
$ istioctl proxy-status | grep ${SERVICENAME}
193+
NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION
194+
${VM_NAME}.${VM_NAMESPACE} Kubernetes SYNCED (4m30s) SYNCED (4m30s) SYNCED (4m30s) SYNCED (4m30s) IGNORED istiod-5644d594-q4qr5 {{< istio_full_version >}}
195+
{{< /text >}}
196+
197+
## Stopping the Istio-proxy container
198+
199+
1. Stop the sidecar container:
200+
201+
{{< text bash >}}
202+
$ docker stop istio-proxy; docker rm istio-proxy
203+
{{< /text >}}
204+
205+
1. Cleanup iptables to stop redirecting traffic to the Istio-proxy (that no longer exists/runs):
206+
207+
{{< text bash >}}
208+
$ docker run \
209+
--rm \
210+
--cap-add=NET_ADMIN \
211+
--entrypoint="" \
212+
--network host \
213+
--env-file /var/lib/istio/envoy/cluster.env \
214+
--name istio-init \
215+
istio/proxyv2:{{< istio_full_version >}} \
216+
/bin/bash -c "update-alternatives --set iptables /usr/sbin/iptables-nft && \
217+
update-alternatives --set ip6tables /usr/sbin/ip6tables-nft && \
218+
/usr/local/bin/pilot-agent istio-clean-iptables"
219+
{{< /text >}}
220+
221+
{{< idea >}}
222+
A more sophisticated way of running containerized Istio-proxy would be to reuse the startup script or setup systemd unit files to take care of start and stopping your Istio-proxy.
223+
Making sure it always has a correct configured environment.
224+
{{< /idea >}}

0 commit comments

Comments
 (0)