You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#Customer intent: As an Azure Kubernetes user, I want to troubleshoot minor revision upgrades of the Istio add-on so that I can use the Istio service mesh successfully.
12
12
---
13
13
# Istio service mesh add-on minor revision upgrade troubleshooting
@@ -29,9 +29,10 @@ The following table lists various problems and the different scenarios and solut
29
29
30
30
| Scenario | Problem | Solution |
31
31
|--|--|--|
32
-
| Data plane workloads are dropped from the mesh. | Data plane and control plane revisions didn't correspond before you completed or rolled back an upgrade. | <p>Follow these steps:</p><ol> <li><p>Relabel namespaces that contain workloads by specifying the revision that's expected to exist after the upgrade completion or rollback. To do this, run the [kubectl label](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_label/) command:</p><pre>kubectl label namespace default istio.io/rev=asm-x-y --overwrite</pre></li> <li><p>Restart the corresponding workload deployments to trigger sidecar reinjection of the correct revision. To do this, run the [kubectl rollout restart](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_rollout/kubectl_rollout_restart/) command:</p><pre>kubectl rollout restart deployment \<deployment name></pre></li> <li><p>Verify that the sidecar images exist. To do this, run the [kubectl get](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_get/) command:</p><pre>kubectl get pods --namespace \<namespace> --output yaml \| grep mcr.microsoft.com/oss/istio/proxyv2:</pre></li> </ol> |
32
+
| Data plane workloads are dropped from the mesh. | Data plane and control plane revisions didn't correspond before you completed or rolled back an upgrade. | <p>Follow these steps:</p><ol> <li><p>Relabel namespaces that contain workloads by specifying the revision that's expected to exist after the upgrade completion or rollback:</p><pre>kubectl label namespace default istio.io/rev=asm-x-y --overwrite</pre></li> <li><p>Restart the corresponding workload deployments using the [kubectl rollout restart](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_rollout/kubectl_rollout_restart/) command to trigger sidecar reinjection of the correct revision:</p><pre>kubectl rollout restart deployment \<deployment name></pre></li> <li><p>Verify that the sidecar images exist using the [kubectl get](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_get/) command:</p><pre>kubectl get pods --namespace \<namespace> --output yaml \| grep mcr.microsoft.com/oss/istio/proxyv2:</pre></li> </ol> |
33
33
| Control plane pods are in the pending state. | The pods lack capacity. | Verify the state of the pods by running the [kubectl describe](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_describe/) command. If capacity is the problem, you can scale up your cluster to add another node. For more information, see [Manually scale the node count in an Azure Kubernetes Service (AKS) cluster](/azure/aks/scale-cluster). |
34
-
| The [az aks mesh get-upgrades](/cli/azure/aks/mesh#az-aks-mesh-get-upgrades) command returns no available upgrades. | The newest Istio revision might be incompatible with the current AKS cluster version. | You can use the [az aks mesh get-revisions](/cli/azure/aks/mesh#az-aks-mesh-get-revisions) command to discover whether newer Istio revisions exist. The output includes a list of compatible cluster versions for each Istio revision. Therefore, you can determine whether a cluster upgrade is necessary. |
34
+
| The [az aks mesh get-upgrades](/cli/azure/aks/mesh#az-aks-mesh-get-upgrades) command returns no available upgrades. | The next Istio revision might be incompatible with the current AKS cluster version. | You can use the [az aks mesh get-revisions](/cli/azure/aks/mesh#az-aks-mesh-get-revisions) command to discover whether newer Istio revisions exist. The output includes a list of compatible cluster versions for each Istio revision. Therefore, you can determine whether a cluster upgrade is necessary. If _both_ mesh and cluster are no longer supported, upgrade the cluster version first, and then the mesh revision. To recover from this scenario, a cluster upgrade is permitted even if it's incompatible with the mesh revision. |
35
+
35
36
36
37
> [!NOTE]
37
38
> To avoid unintended behavior and broken functionality, and also make sure that you're receiving updates for security vulnerabilities, we strongly recommend that you upgrade to a supported and up-to-date [AKS version](/azure/aks/supported-kubernetes-versions) and Istio add-on revision. Remember that the add-on revision should also be within the supported Kubernetes version range for the given AKS cluster. As highlighted in the [Minor revision upgrade](/azure/aks/istio-upgrade#minor-revision-upgrade) section of the Istio upgrade article, you can run the `az aks mesh get-revisions` and `az aks mesh get-upgrades` commands to learn about available add-on revisions, upgrades, and compatibility information.
@@ -40,7 +41,10 @@ The following table lists various problems and the different scenarios and solut
40
41
41
42
- A downgrade to an older revision (outside the canary rollback process) isn't allowed.
42
43
43
-
- Skipping from one revision to a nonconsecutive revision is allowed only if AKS no longer supports both the current revision and the next upgrade revision. At this point, the only upgrade that's available to you is the lowest supported revision.
44
+
- Available upgrades for a revision depend on whether it's currently supported. For example, if `n` is the currently installed revision and `n+2` is the latest revision:
45
+
- If `n` is supported, you may upgrade to the next revision `n+1` or directly to the newest revision `n+2`.
46
+
- If both `n` and `n+1` (next consecutive) are unsupported, the only available upgrade is `n+2` (next supported).
47
+
- If `n` has been unsupported for a while, it's possible both of the next two consecutive revisions are unsupported. In this case, the only available upgrade is the lowest supported revision.
44
48
45
49
- The Istio `sidecar.istio.io/inject` label doesn't enable sidecar injection for the Istio add-on. You must use the `istio.io/rev` label when you label and relabel your namespaces during the canary upgrade.
0 commit comments