Skip to content

Commit 60c6d6e

Browse files
general changes
1 parent bd41dd9 commit 60c6d6e

File tree

1 file changed

+27
-21
lines changed

1 file changed

+27
-21
lines changed

articles/iot-operations/troubleshoot/troubleshoot.md

Lines changed: 27 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ ms.date: 03/07/2025
1313

1414
This article contains troubleshooting tips for Azure IoT Operations.
1515

16-
## General deployment troubleshooting
16+
## Troubleshoot general deployment of Azure IoT Operations
1717

1818
For general deployment and configuration troubleshooting, you can use the Azure CLI IoT Operations *check* and *support* commands.
1919

@@ -25,31 +25,37 @@ For general deployment and configuration troubleshooting, you can use the Azure
2525

2626
### Troubleshoot UnauthorizedNamespaceError error message
2727

28-
```
28+
If you see the following error message, you either didn't enable the required azure-arc custom locations feature, or you enabled the custom locations feature with an incorrect custom locations RP OID.
29+
30+
```ouput
2931
Message: Microsoft.ExtendedLocation resource provider does not have the required permissions to create a namespace on the cluster.
3032
```
3133

32-
This error is commonly caused by either not enabling the required azure-arc custom locations feature, or enabling the custom locations feature with an incorrect custom locations RP OID. To resolve, follow [this guidance](/azure-arc/kubernetes/custom-locations#enable-custom-locations-on-your-cluster) for enabling the custom locations feature with the correct OID.
34+
To resolve, follow [this guidance](/azure-arc/kubernetes/custom-locations#enable-custom-locations-on-your-cluster) for enabling the custom locations feature with the correct OID.
3335

3436
### Troubleshoot MissingResourceVersionOnHost error message
3537

36-
```
38+
If you see the following error message, your custom location resource associated with the deployment isn't properly configured with the API version(s) of resources attempting to be projected to the cluster.
39+
40+
```output
3741
Message: The resource {resource Id} extended location {custom location resource Id} does not support the resource type {IoT Operations resource type} or api version {IoT Operations ARM API}. Please check with the owner of the extended location to ensure the host has the CRD {custom resource name} with group {api group name}.iotoperations.azure.com, plural {custom resource plural name}, and versions [{api group version}] installed.
3842
```
3943

40-
This error happens when the custom location resource associated with the deployment isn't properly configured with the API version(s) of resources attempting to be projected to the cluster. To resolve, delete any provisioned resources associated with prior deployment(s) including custom locations. You can use `az iot ops delete` or alternative mechanism. Due to a potential caching issue, waiting a few minutes after deletion before re-deploying AIO or choosing a custom location name via `az iot ops create --custom-location` is recommended.
44+
To resolve, delete any provisioned resources associated with prior deployment(s) including custom locations. You can use `az iot ops delete` or alternative mechanism. Due to a potential caching issue, waiting a few minutes after deletion before re-deploying AIO or choosing a custom location name via `az iot ops create --custom-location` is recommended.
4145

4246
### Troubleshoot LinkedAuthorizationFailed error message
4347

44-
```
48+
If you see the following error message, the logged-in principal doesn't have the required permissions to deploy resources to the resource group specified in the resource sync resource ID.
49+
50+
```output
4551
Message: The client {principal Id} with object id {principal object Id} has permission to perform action Microsoft.ExtendedLocation/customLocations/resourceSyncRules/write on scope {resource sync resource Id}; however, it does not have permission to perform action(s) Microsoft.Authorization/roleAssignments/write on the linked scope(s) {resource sync resource group} (respectively) or the linked scope(s) are invalid.
4652
```
4753

4854
Deployment of resource sync rules require the logged-in principal to have the `Microsoft.Authorization/roleAssignments/write` permission against the resource group that resources are being deployed to. This is a necessary security constraint as edge to cloud resource hydration will create new resources in the target resource group.
4955

5056
To resolve, either elevate principal permissions, or don't deploy resource sync rules. Current AIO CLI has an opt-in mechanism to deploy resource sync rules via `--enable-rsync`. Simply omit this flag. Legacy AIO CLIs had an opt-out mechanism via `--disable-rsync-rules`.
5157

52-
## Secret management
58+
## Troubleshoot Azure Key Vault secret management
5359

5460
If you see the following error message related to secret management, you need to update your Azure Key Vault contents:
5561

@@ -63,7 +69,7 @@ For help resolving this issue, please see https://go.microsoft.com/fwlink/?linki
6369

6470
This error occurs when Azure IoT Operations tries to synchronize a secret from Azure Key Vault that doesn't exist. To resolve this issue, you need to add the secret in Azure Key Vault before you create resources such as a secret provider class.
6571

66-
## Connector for OPC UA
72+
## Troubleshoot asset connector for OPC UA
6773

6874
An OPC UA server connection fails with a `BadSecurityModeRejected` error if the connector tries to connect to a server that only exposes endpoints with no security. There are two options to resolve this issue:
6975

@@ -76,11 +82,11 @@ An OPC UA server connection fails with a `BadSecurityModeRejected` error if the
7682

7783
- Add a secure endpoint to the OPC UA server and set up the certificate mutual trust to establish the connection.
7884

79-
## Azure IoT Layered Network Management (preview) troubleshooting
85+
## Troubleshoot Azure IoT Layered Network Management (preview)
8086

8187
The troubleshooting guidance in this section is specific to Azure IoT Operations when using the Layered Network Management component. For more information, see [How does Azure IoT Operations work in layered network?](../manage-layered-network/concept-iot-operations-in-layered-network.md).
8288

83-
### Can't install Layered Network Management on the parent level
89+
### You can't install Layered Network Management on the parent level
8490

8591
If the Layered Network Management operator install fails or you can't apply the custom resource for a Layered Network Management instance:
8692

@@ -89,7 +95,7 @@ If the Layered Network Management operator install fails or you can't apply the
8995
1. Verify the Layered Network Management operator is in the *Running and Ready* state.
9096
1. If applying the custom resource `kubectl apply -f cr.yaml` fails, the output of this command lists the reason for error. For example, CRD version mismatch or wrong entry in CRD.
9197

92-
### Can't Arc-enable the cluster through the parent level Layered Network Management
98+
### You can't Arc-enable the cluster through the parent level Layered Network Management
9399

94100
If you repeatedly remove and onboard a cluster with the same machine, you might get an error while Arc-enabling the cluster on nested layers. For example:
95101

@@ -107,12 +113,12 @@ If your cluster is behind an outbound proxy server, please ensure that you have
107113

108114
1. Reboot the host machine.
109115

110-
### Other types of Arc-enablement failures
116+
If you still see the error, check the following:
111117

112118
1. Add the `--debug` parameter when running the `connectedk8s` command.
113119
1. Capture and investigate a network packet trace. For more information, see [capture Layered Network Management packet trace](#capture-layered-network-management-packet-trace).
114120

115-
### Can't install Azure IoT Operations on the isolated cluster
121+
### You can't install Azure IoT Operations on the isolated cluster
116122

117123
You can't install Azure IoT Operations components on nested layers. For example, Layered Network Management on level 4 is running but can't install Azure IoT Operations on level 3.
118124

@@ -128,7 +134,7 @@ You can't install Azure IoT Operations components on nested layers. For example,
128134
1. If the domain is being resolved correctly, verify the domain is added to the allowlist. For more information, see [Check the allowlist of Layered Network Management](#check-the-allowlist-of-layered-network-management).
129135
1. Capture and investigate a network packet trace. For more information, see [capture Layered Network Management packet trace](#capture-layered-network-management-packet-trace).
130136

131-
### A pod fails when installing Azure IoT Operations on an isolated cluster
137+
### You can install Azure IoT Operations on the isolated cluster but the pods fail to start
132138

133139
When installing the Azure IoT Operations components to a cluster, the installation starts and proceeds. However, initialization of one or few of the components (pods) fails.
134140

@@ -150,9 +156,9 @@ When installing the Azure IoT Operations components to a cluster, the installati
150156
Warning Failed 3m14s kubelet Failed to pull image "
151157
```
152158
153-
### Check the allowlist of Layered Network Management
159+
### You can't connect to the Azure IoT Operations service from the child level Layered Network Management
154160
155-
Layered Network Management blocks traffic if the destination domain isn't on the allowlist.
161+
Layered Network Management blocks traffic if the destination domain isn't on the allowlist. The allowlist is a list of domains that are allowed to be accessed from the child level Layered Network Management. Check the allowlist of Layered Network Management to verify if the domain is included. If the domain isn't on the allowlist, you can add it to the allowlist.
156162
157163
1. Run the following command to list the config maps.
158164
@@ -176,18 +182,18 @@ Layered Network Management blocks traffic if the destination domain isn't on the
176182
177183
1. All the allowed domains are listed in the output.
178184
179-
### Capture Layered Network Management packet trace
185+
### You want to capture Layered Network Management to a packet trace
180186
181187
In some cases, you might suspect that Layered Network Management instance at the parent level isn't forwarding network traffic to a particular endpoint. Connection to a required endpoint is causing an issue for the service running on your node. It's possible that the service you enabled is trying to connect to a new endpoint after an update. Or you're trying to install a new Arc extension or service that requires connection to endpoints that aren't on the default allowlist. Usually there would be information in the error message to notify the connection failure. However, if there's no clear information about the missing endpoint, you can capture the network traffic on the child node for detailed debugging.
182188
183-
#### Windows host
189+
#### [Windows host](#tab/tabid-windows)
184190
185191
1. Install Wireshark network traffic analyzer on the host.
186192
1. Run Wireshark and start capturing.
187193
1. Reproduce the installation or connection failure.
188194
1. Stop capturing.
189195
190-
#### Linux host
196+
#### [Linux host](#tab/tabid-linux)
191197
192198
1. Run the following command to start capturing:
193199
@@ -198,14 +204,14 @@ In some cases, you might suspect that Layered Network Management instance at the
198204
1. Reproduce the installation or connection failure.
199205
1. Stop capturing.
200206
201-
#### Analyze the packet trace
207+
***
202208
203209
Use Wireshark to open the trace file. Look for connection failures or unresponsive connections.
204210
205211
1. Filter the packets with the *ip.addr == [IP address]* parameter. Input the IP address of your custom DNS service address.
206212
1. Review the DNS query and response, check if there's a domain name that isn't on the allowlist of Layered Network Management.
207213
208-
## Operations experience
214+
## IoT Operations experience
209215
210216
To sign in to the [operations experience](https://iotoperations.azure.com) web UI, you need a Microsoft Entra ID account with at least contributor permissions for the resource group that contains your **Kubernetes - Azure Arc** instance.
211217

0 commit comments

Comments
 (0)