|
1 | 1 | ---
|
2 |
| -title: Create virtual nodes using the portal in Azure Kubernetes Services (AKS) |
| 2 | +title: Create virtual nodes in Azure Kubernetes Service (AKS) using the Azure portal |
3 | 3 | description: Learn how to use the Azure portal to create an Azure Kubernetes Services (AKS) cluster that uses virtual nodes to run pods.
|
4 | 4 | ms.topic: conceptual
|
5 |
| -ms.date: 03/15/2021 |
| 5 | +ms.date: 05/09/2023 |
6 | 6 | ms.custom: references_regions
|
7 | 7 | ---
|
8 | 8 |
|
9 | 9 | # Create and configure an Azure Kubernetes Services (AKS) cluster to use virtual nodes in the Azure portal
|
10 | 10 |
|
11 |
| -This article shows you how to use the Azure portal to create and configure the virtual network resources and an AKS cluster with virtual nodes enabled. |
| 11 | +Virtual nodes enable network communication between pods that run in Azure Container Instances (ACI) and Azure Kubernetes Service (AKS) clusters. To provide this communication, a virtual network subnet is created and delegated permissions are assigned. Virtual nodes only work with AKS clusters created using *advanced* networking (Azure CNI). AKS clusters are created with *basic* networking (kubenet) by default. |
| 12 | + |
| 13 | +This article shows you how to create a virtual network and subnets, and then deploy an AKS cluster that uses advanced networking using the Azure portal. |
12 | 14 |
|
13 | 15 | > [!NOTE]
|
14 |
| -> [This article](virtual-nodes.md) gives you an overview of the region availability and limitations using virtual nodes. |
| 16 | +> For an overview of virtual node region availability and limitations, see [Use virtual nodes in AKS](virtual-nodes.md). |
15 | 17 |
|
16 | 18 | ## Before you begin
|
17 | 19 |
|
18 |
| -Virtual nodes enable network communication between pods that run in Azure Container Instances (ACI) and the AKS cluster. To provide this communication, a virtual network subnet is created and delegated permissions are assigned. Virtual nodes only work with AKS clusters created using *advanced* networking (Azure CNI). By default, AKS clusters are created with *basic* networking (kubenet). This article shows you how to create a virtual network and subnets, then deploy an AKS cluster that uses advanced networking. |
19 |
| - |
20 |
| -If you have not previously used ACI, register the service provider with your subscription. You can check the status of the ACI provider registration using the [az provider list][az-provider-list] command, as shown in the following example: |
| 20 | +You need the ACI service provider registered on your subscription. |
21 | 21 |
|
22 |
| -```azurecli-interactive |
23 |
| -az provider list --query "[?contains(namespace,'Microsoft.ContainerInstance')]" -o table |
24 |
| -``` |
| 22 | +* Check the status of the ACI provider registration using the [`az provider list`][az-provider-list] command. |
25 | 23 |
|
26 |
| -The *Microsoft.ContainerInstance* provider should report as *Registered*, as shown in the following example output: |
| 24 | + ```azurecli-interactive |
| 25 | + az provider list --query "[?contains(namespace,'Microsoft.ContainerInstance')]" -o table |
| 26 | + ``` |
27 | 27 |
|
28 |
| -```output |
29 |
| -Namespace RegistrationState RegistrationPolicy |
30 |
| ---------------------------- ------------------- -------------------- |
31 |
| -Microsoft.ContainerInstance Registered RegistrationRequired |
32 |
| -``` |
| 28 | + The following example output shows the *Microsoft.ContainerInstance* provider is *Registered*: |
33 | 29 |
|
34 |
| -If the provider shows as *NotRegistered*, register the provider using the [az provider register][az-provider-register] as shown in the following example: |
| 30 | + ```output |
| 31 | + Namespace RegistrationState RegistrationPolicy |
| 32 | + --------------------------- ------------------- -------------------- |
| 33 | + Microsoft.ContainerInstance Registered RegistrationRequired |
| 34 | + ``` |
35 | 35 |
|
36 |
| -```azurecli-interactive |
37 |
| -az provider register --namespace Microsoft.ContainerInstance |
38 |
| -``` |
| 36 | +* If the provider is *NotRegistered*, register it using the [`az provider register`][az-provider-register] command. |
39 | 37 |
|
40 |
| -## Sign in to Azure |
41 |
| - |
42 |
| -Sign in to the Azure portal at https://portal.azure.com. |
| 38 | + ```azurecli-interactive |
| 39 | + az provider register --namespace Microsoft.ContainerInstance |
| 40 | + ``` |
43 | 41 |
|
44 | 42 | ## Create an AKS cluster
|
45 | 43 |
|
46 |
| -In the top left-hand corner of the Azure portal, select **Create a resource** > **Kubernetes Service**. |
47 |
| - |
48 |
| -On the **Basics** page, configure the following options: |
49 |
| - |
50 |
| -- *PROJECT DETAILS*: Select an Azure subscription, then select or create an Azure resource group, such as *myResourceGroup*. Enter a **Kubernetes cluster name**, such as *myAKSCluster*. |
51 |
| -- *CLUSTER DETAILS*: Select a region and Kubernetes version for the AKS cluster. |
52 |
| -- *PRIMARY NODE POOL*: Select a VM size for the AKS nodes. The VM size **cannot** be changed once an AKS cluster has been deployed. |
53 |
| - - Select the number of nodes to deploy into the cluster. For this article, set **Node count** to *1*. Node count **can** be adjusted after the cluster has been deployed. |
54 |
| - |
55 |
| -Click **Next: Node Pools**. |
56 |
| - |
57 |
| -On the **Node Pools** page, select *Enable virtual nodes*. |
| 44 | +1. Navigate to the Azure portal home page. |
| 45 | +2. Select **Create a resource** > **Containers**. |
| 46 | +3. On the **Azure Kubernetes Service (AKS)** resource, select **Create**. |
| 47 | +4. On the **Basics** page, configure the following options: |
| 48 | + * *Project details*: Select an Azure subscription, then select or create an Azure resource group, such as *myResourceGroup*. |
| 49 | + * *Cluster details*: Enter a **Kubernetes cluster name**, such as *myAKSCluster*. Select a region and Kubernetes version for the AKS cluster. |
| 50 | +5. Select **Next: Node pools** and check **Enable virtual nodes*. |
| 51 | + :::image type="content" source="media/virtual-nodes-portal/enable-virtual-nodes.png" alt-text="Screenshot that shows creating a cluster with virtual nodes enabled on the Azure portal. The option 'Enable virtual nodes' is highlighted."::: |
| 52 | +6. Select **Review + create**. |
| 53 | +7. After the validation completes, select **Create**. |
58 | 54 |
|
59 |
| -:::image type="content" source="media/virtual-nodes-portal/enable-virtual-nodes.png" alt-text="In a browser, shows creating a cluster with virtual nodes enabled on the Azure portal. The option 'Enable virtual nodes' is highlighted."::: |
| 55 | +By default, this process creates a managed cluster identity, which is used for cluster communication and integration with other Azure services. For more information, see [Use managed identities](use-managed-identity.md). You can also use a service principal as your cluster identity. |
60 | 56 |
|
61 |
| -By default, a cluster identity is created. This cluster identity is used for cluster communication and integration with other Azure services. By default, this cluster identity is a managed identity. For more information, see [Use managed identities](use-managed-identity.md). You can also use a service principal as your cluster identity. |
62 |
| - |
63 |
| -The cluster is also configured for advanced networking. The virtual nodes are configured to use their own Azure virtual network subnet. This subnet has delegated permissions to connect Azure resources between the AKS cluster. If you don't already have delegated subnet, the Azure portal creates and configures the Azure virtual network and subnet for use with the virtual nodes. |
64 |
| - |
65 |
| -Select **Review + create**. After the validation is complete, select **Create**. |
66 |
| - |
67 |
| -It takes a few minutes to create the AKS cluster and to be ready for use. |
| 57 | +This process configures the cluster for advanced networking and the virtual nodes to use their own Azure virtual network subnet. The subnet has delegated permissions to connect Azure resources between the AKS cluster. If you don't already have a delegated subnet, the Azure portal creates and configures an Azure virtual network and subnet with the virtual nodes. |
68 | 58 |
|
69 | 59 | ## Connect to the cluster
|
70 | 60 |
|
71 |
| -The Azure Cloud Shell is a free interactive shell that you can use to run the steps in this article. It has common Azure tools preinstalled and configured to use with your account. To manage a Kubernetes cluster, use [kubectl][kubectl], the Kubernetes command-line client. The `kubectl` client is pre-installed in the Azure Cloud Shell. |
72 |
| - |
73 |
| -To open the Cloud Shell, select **Try it** from the upper right corner of a code block. You can also launch Cloud Shell in a separate browser tab by going to [https://shell.azure.com/bash](https://shell.azure.com/bash). Select **Copy** to copy the blocks of code, paste it into the Cloud Shell, and press enter to run it. |
| 61 | +The Azure Cloud Shell is a free interactive shell you can use to run the steps in this article. It has common Azure tools preinstalled and configured to use with your account. To manage a Kubernetes cluster, use [kubectl][kubectl], the Kubernetes command-line client. The `kubectl` client is pre-installed in the Azure Cloud Shell. |
74 | 62 |
|
75 |
| -Use the [az aks get-credentials][az-aks-get-credentials] command to configure `kubectl` to connect to your Kubernetes cluster. The following example gets credentials for the cluster name *myAKSCluster* in the resource group named *myResourceGroup*: |
| 63 | +1. Configure `kubectl` to connect to your Kubernetes cluster using the [`az aks get-credentials`][az-aks-get-credentials] command. The following example gets credentials for the cluster name *myAKSCluster* in the resource group named *myResourceGroup*: |
76 | 64 |
|
77 |
| -```azurecli-interactive |
78 |
| -az aks get-credentials --resource-group myResourceGroup --name myAKSCluster |
79 |
| -``` |
| 65 | + ```azurecli-interactive |
| 66 | + az aks get-credentials --resource-group myResourceGroup --name myAKSCluster |
| 67 | + ``` |
80 | 68 |
|
81 |
| -To verify the connection to your cluster, use the [kubectl get][kubectl-get] command to return a list of the cluster nodes. |
| 69 | +2. Verify the connection to your cluster using the [`kubectl get nodes`][kubectl-get]. |
82 | 70 |
|
83 |
| -```console |
84 |
| -kubectl get nodes |
85 |
| -``` |
| 71 | + ```azurecli-interactive |
| 72 | + kubectl get nodes |
| 73 | + ``` |
86 | 74 |
|
87 |
| -The following example output shows the single VM node created and then the virtual node for Linux, *virtual-node-aci-linux*: |
| 75 | + The following example output shows the single VM node created and the virtual Linux node named *virtual-node-aci-linux*: |
88 | 76 |
|
89 |
| -```output |
90 |
| -NAME STATUS ROLES AGE VERSION |
91 |
| -virtual-node-aci-linux Ready agent 28m v1.11.2 |
92 |
| -aks-agentpool-14693408-0 Ready agent 32m v1.11.2 |
93 |
| -``` |
| 77 | + ```output |
| 78 | + NAME STATUS ROLES AGE VERSION |
| 79 | + virtual-node-aci-linux Ready agent 28m v1.11.2 |
| 80 | + aks-agentpool-14693408-0 Ready agent 32m v1.11.2 |
| 81 | + ``` |
94 | 82 |
|
95 | 83 | ## Deploy a sample app
|
96 | 84 |
|
97 |
| -In the Azure Cloud Shell, create a file named `virtual-node.yaml` and copy in the following YAML. To schedule the container on the node, a [nodeSelector][node-selector] and [toleration][toleration] are defined. These settings allow the pod to be scheduled on the virtual node and confirm that the feature is successfully enabled. |
98 |
| - |
99 |
| -```yaml |
100 |
| -apiVersion: apps/v1 |
101 |
| -kind: Deployment |
102 |
| -metadata: |
103 |
| - name: aci-helloworld |
104 |
| -spec: |
105 |
| - replicas: 1 |
106 |
| - selector: |
107 |
| - matchLabels: |
108 |
| - app: aci-helloworld |
109 |
| - template: |
| 85 | +1. In the Azure Cloud Shell, create a file named `virtual-node.yaml` and copy in the following YAML: |
| 86 | +
|
| 87 | + ```yaml |
| 88 | + apiVersion: apps/v1 |
| 89 | + kind: Deployment |
110 | 90 | metadata:
|
111 |
| - labels: |
112 |
| - app: aci-helloworld |
| 91 | + name: aci-helloworld |
113 | 92 | spec:
|
114 |
| - containers: |
115 |
| - - name: aci-helloworld |
116 |
| - image: mcr.microsoft.com/azuredocs/aci-helloworld |
117 |
| - ports: |
118 |
| - - containerPort: 80 |
119 |
| - nodeSelector: |
120 |
| - kubernetes.io/role: agent |
121 |
| - beta.kubernetes.io/os: linux |
122 |
| - type: virtual-kubelet |
123 |
| - tolerations: |
124 |
| - - key: virtual-kubelet.io/provider |
125 |
| - operator: Exists |
126 |
| -``` |
127 |
| -
|
128 |
| -Run the application with the [kubectl apply][kubectl-apply] command. |
129 |
| -
|
130 |
| -```azurecli-interactive |
131 |
| -kubectl apply -f virtual-node.yaml |
132 |
| -``` |
133 |
| - |
134 |
| -Use the [kubectl get pods][kubectl-get] command with the `-o wide` argument to output a list of pods and the scheduled node. Notice that the `virtual-node-helloworld` pod has been scheduled on the `virtual-node-linux` node. |
135 |
| - |
136 |
| -```console |
137 |
| -kubectl get pods -o wide |
138 |
| -``` |
139 |
| - |
140 |
| -```output |
141 |
| -NAME READY STATUS RESTARTS AGE IP NODE |
142 |
| -virtual-node-helloworld-9b55975f-bnmfl 1/1 Running 0 4m 10.241.0.4 virtual-node-aci-linux |
143 |
| -``` |
144 |
| - |
145 |
| -The pod is assigned an internal IP address from the Azure virtual network subnet delegated for use with virtual nodes. |
| 93 | + replicas: 1 |
| 94 | + selector: |
| 95 | + matchLabels: |
| 96 | + app: aci-helloworld |
| 97 | + template: |
| 98 | + metadata: |
| 99 | + labels: |
| 100 | + app: aci-helloworld |
| 101 | + spec: |
| 102 | + containers: |
| 103 | + - name: aci-helloworld |
| 104 | + image: mcr.microsoft.com/azuredocs/aci-helloworld |
| 105 | + ports: |
| 106 | + - containerPort: 80 |
| 107 | + nodeSelector: |
| 108 | + kubernetes.io/role: agent |
| 109 | + beta.kubernetes.io/os: linux |
| 110 | + type: virtual-kubelet |
| 111 | + tolerations: |
| 112 | + - key: virtual-kubelet.io/provider |
| 113 | + operator: Exists |
| 114 | + ``` |
| 115 | +
|
| 116 | + The YAML defines a [nodeSelector][node-selector] and [toleration][toleration], which allows the pod to be scheduled on the virtual node. The pod is assigned an internal IP address from the Azure virtual network subnet delegated for use with virtual nodes. |
| 117 | +
|
| 118 | +2. Run the application using the [`kubectl apply`][kubectl-apply] command. |
| 119 | +
|
| 120 | + ```azurecli-interactive |
| 121 | + kubectl apply -f virtual-node.yaml |
| 122 | + ``` |
| 123 | +
|
| 124 | +3. View the pods scheduled on the node using the [`kubectl get pods`][kubectl-get] command with the `-o wide` argument. |
| 125 | +
|
| 126 | + ```azurecli-interactive |
| 127 | + kubectl get pods -o wide |
| 128 | + ``` |
| 129 | +
|
| 130 | + The following example output shows the `virtual-node-helloworld` pod scheduled on the `virtual-node-linux` node. |
| 131 | +
|
| 132 | + ```output |
| 133 | + NAME READY STATUS RESTARTS AGE IP NODE |
| 134 | + virtual-node-helloworld-9b55975f-bnmfl 1/1 Running 0 4m 10.241.0.4 virtual-node-aci-linux |
| 135 | + ``` |
146 | 136 |
|
147 | 137 | > [!NOTE]
|
148 |
| -> If you use images stored in Azure Container Registry, [configure and use a Kubernetes secret][acr-aks-secrets]. A current limitation of virtual nodes is that you can't use integrated Azure AD service principal authentication. If you don't use a secret, pods scheduled on virtual nodes fail to start and report the error `HTTP response status code 400 error code "InaccessibleImage"`. |
| 138 | +> If you use images stored in Azure Container Registry, [configure and use a Kubernetes secret][acr-aks-secrets]. A limitation of virtual nodes is you can't use integrated Azure AD service principal authentication. If you don't use a secret, pods scheduled on virtual nodes fail to start and report the error `HTTP response status code 400 error code "InaccessibleImage"`. |
149 | 139 |
|
150 | 140 | ## Test the virtual node pod
|
151 | 141 |
|
152 |
| -To test the pod running on the virtual node, browse to the demo application with a web client. As the pod is assigned an internal IP address, you can quickly test this connectivity from another pod on the AKS cluster. Create a test pod and attach a terminal session to it: |
| 142 | +To test the pod running on the virtual node, browse to the demo application with a web client. The pod is assigned an internal IP address, so you can easily test the connectivity from another pod on the AKS cluster. |
| 143 | +
|
| 144 | +1. Create a test pod and attach a terminal session to it using the following `kubectl run` command. |
| 145 | +
|
| 146 | + ```console |
| 147 | + kubectl run -it --rm virtual-node-test --image=mcr.microsoft.com/dotnet/runtime-deps:6.0 |
| 148 | + ``` |
153 | 149 |
|
154 |
| -```console |
155 |
| -kubectl run -it --rm virtual-node-test --image=mcr.microsoft.com/dotnet/runtime-deps:6.0 |
156 |
| -``` |
| 150 | +2. Install `curl` in the pod using the following `apt-get` command. |
157 | 151 |
|
158 |
| -Install `curl` in the pod using `apt-get`: |
| 152 | + ```console |
| 153 | + apt-get update && apt-get install -y curl |
| 154 | + ``` |
159 | 155 |
|
160 |
| -```console |
161 |
| -apt-get update && apt-get install -y curl |
162 |
| -``` |
| 156 | +3. Access the address of your pod using the following `curl` command and provide your internal IP address. |
163 | 157 |
|
164 |
| -Now access the address of your pod using `curl`, such as *http://10.241.0.4*. Provide your own internal IP address shown in the previous `kubectl get pods` command: |
| 158 | + ```console |
| 159 | + curl -L http://10.241.0.4 |
| 160 | + ``` |
165 | 161 |
|
166 |
| -```console |
167 |
| -curl -L http://10.241.0.4 |
168 |
| -``` |
| 162 | + The following condensed example output shows the demo application. |
169 | 163 |
|
170 |
| -The demo application is displayed, as shown in the following condensed example output: |
| 164 | + ```output |
| 165 | + <html> |
| 166 | + <head> |
| 167 | + <title>Welcome to Azure Container Instances!</title> |
| 168 | + </head> |
| 169 | + [...] |
| 170 | + ``` |
171 | 171 |
|
172 |
| -```output |
173 |
| -<html> |
174 |
| -<head> |
175 |
| - <title>Welcome to Azure Container Instances!</title> |
176 |
| -</head> |
177 |
| -[...] |
178 |
| -``` |
| 172 | +4. Close the terminal session to your test pod with `exit`, which also deletes the pod. |
179 | 173 |
|
180 |
| -Close the terminal session to your test pod with `exit`. When your session is ended, the pod is the deleted. |
| 174 | + ```console |
| 175 | + exit |
| 176 | + ``` |
181 | 177 |
|
182 | 178 | ## Next steps
|
183 | 179 |
|
184 |
| -In this article, a pod was scheduled on the virtual node and assigned a private, internal IP address. You could instead create a service deployment and route traffic to your pod through a load balancer or ingress controller. For more information, see [Create a basic ingress controller in AKS][aks-basic-ingress]. |
| 180 | +In this article, you scheduled a pod on the virtual node and assigned a private, internal IP address. If you want, you can instead create a service deployment and route traffic to your pod through a load balancer or ingress controller. For more information, see [Create a basic ingress controller in AKS][aks-basic-ingress]. |
185 | 181 |
|
186 | 182 | Virtual nodes are one component of a scaling solution in AKS. For more information on scaling solutions, see the following articles:
|
187 | 183 |
|
188 |
| -- [Use the Kubernetes horizontal pod autoscaler][aks-hpa] |
189 |
| -- [Use the Kubernetes cluster autoscaler][aks-cluster-autoscaler] |
190 |
| -- [Check out the Autoscale sample for Virtual Nodes][virtual-node-autoscale] |
191 |
| -- [Read more about the Virtual Kubelet open source library][virtual-kubelet-repo] |
| 184 | +* [Use the Kubernetes horizontal pod autoscaler][aks-hpa] |
| 185 | +* [Use the Kubernetes cluster autoscaler][aks-cluster-autoscaler] |
| 186 | +* [Autoscale for virtual nodes][virtual-node-autoscale] |
| 187 | +* [Virtual Kubelet open source library][virtual-kubelet-repo] |
192 | 188 |
|
193 | 189 | <!-- LINKS - external -->
|
194 | 190 | [kubectl]: https://kubernetes.io/docs/reference/kubectl/
|
195 | 191 | [kubectl-get]: https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#get
|
196 | 192 | [kubectl-apply]: https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#apply
|
197 | 193 | [node-selector]:https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
|
198 | 194 | [toleration]: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
|
199 |
| -[azure-cni]: https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md |
200 |
| -[aks-github]: https://github.com/azure/aks/issues] |
201 | 195 | [virtual-node-autoscale]: https://github.com/Azure-Samples/virtual-node-autoscale
|
202 | 196 | [virtual-kubelet-repo]: https://github.com/virtual-kubelet/virtual-kubelet
|
203 | 197 | [acr-aks-secrets]: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
|
204 | 198 |
|
205 | 199 | <!-- LINKS - internal -->
|
206 |
| -[aks-network]: ./configure-azure-cni.md |
207 | 200 | [az-aks-get-credentials]: /cli/azure/aks#az_aks_get_credentials
|
208 | 201 | [aks-hpa]: tutorial-kubernetes-scale.md
|
209 | 202 | [aks-cluster-autoscaler]: cluster-autoscaler.md
|
|
0 commit comments