You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title: How to troubleshoot with Azure Web PubSub service resource logs
3
-
description: Learn how to get and troubleshoot with resource logs
3
+
description: Learn what resource logs are and how to use them for troubleshooting common problems.
4
4
author: wanlwanl
5
5
ms.author: wanl
6
6
ms.service: azure-web-pubsub
7
7
ms.topic: how-to
8
-
ms.date: 04/01/2022
8
+
ms.date: 07/21/2022
9
9
---
10
10
11
11
# How to troubleshoot with resource logs
12
12
13
-
This how-to guide shows you the options to get the resource logs and how to troubleshoot with them.
13
+
This how-to guide provides an overview of Azure Web PubSub resource logs and some tips for using the logs to troubleshoot certain problems. Logs can be used for issue identification, connection tracking, message tracing, HTTP request tracing, and analysis.
14
14
15
-
## What's the resource logs?
15
+
## <aid="whats-are-resource-logs">What are resource logs?</a>
16
16
17
-
The resource logs provide richer view of connectivity, messaging and HTTP request information to the Azure Web PubSub service instance. They can be used for issue identification, connection tracking, message tracing, HTTP request tracing and analysis.
17
+
There are three types of resource logs: *Connectivity*, *Messaging*, and *HTTP requests*.
18
+
-**Connectivity** logs provide detailed information for Azure Web PubSub hub connections. For example, basic information (user ID, connection ID, and so on) and event information (connect, disconnect, and so on).
19
+
-**Messaging** logs provide tracing information for the Azure Web PubSub hub messages received and sent via Azure Web PubSub service. For example, tracing ID and message type of the message.
20
+
-**HTTP requests** logs provide tracing information for HTTP requests to the Azure Web PubSub service. For example, HTTP method and status code. Typically the HTTP request is recorded when it arrives at or leave from service.
18
21
19
-
There are three types of logs: connectivity log, messaging log and HTTP request logs.
22
+
## <aid="capture-resource-logs-with-live-trace-tool">Capture resource logs by using the live trace tool</a>
20
23
21
-
### Connectivity logs
22
-
23
-
Connectivity logs provide detailed information for Azure Web PubSub hub connections. For example, basic information (user ID, connection ID, and so on) and event information (connect, disconnect, and abort event, and so on). That's why the connectivity log is helpful to troubleshoot connection-related issues.
24
-
25
-
### Messaging logs
26
-
27
-
Messaging logs provide tracing information for the Azure Web PubSub hub messages received and sent via Azure Web PubSub service. For example, tracing ID and message type of the message. Typically the message is recorded when it arrives at or leaves from service. So messaging logs are helpful for troubleshooting message-related issues.
28
-
29
-
### HTTP request logs
30
-
31
-
Http request logs provide tracing information for HTTP requests to the Azure Web PubSub service. For example, HTTP method and status code. Typically the HTTP request is recorded when it arrives at or leave from service. So HTTP request logs are helpful for troubleshooting request-related issues.
32
-
33
-
## Capture resource logs with live trace tool
34
-
35
-
The Azure Web PubSub service live trace tool has ability to collect resource logs in real time, and is helpful to trace with customer's development environment. The live trace tool could capture connectivity logs, messaging logs and HTTP request logs.
36
-
37
-
> [!NOTE]
38
-
> The real-time resource logs captured by live trace tool will be billed as messages (outbound traffic).
24
+
The Azure Web PubSub service live trace tool has ability to collect resource logs in real time, which is helpful for troubleshooting problems in your development environment. The live trace tool can capture connectivity logs, messaging logs, and HTTP request logs.
39
25
40
26
> [!NOTE]
41
-
> The Azure Web PubSub service instance created as free tier has the daily limit of messages (outbound traffic).
27
+
> The following considerations apply to using the live trace tool:
28
+
> - The real-time resource logs captured by live trace tool will be billed as messages (outbound traffic).
29
+
> - The live trace tool does not currently support Azure Active Directory authentication. You must enable access keys to use live trace. Under **Settings**, select **Keys**, and then enable **Access Key**.
30
+
> - The Azure Web PubSub service Free Tier instance has a daily limit of 20,000 messages (outbound traffic). Live trace can cause you to unexpectedly reach the daily limit.
42
31
43
32
### Launch the live trace tool
44
33
45
-
1. Go to the Azure portal.
46
-
2. On the **Live trace settings** page of your Azure Web PubSub service instance, check **Enable Live Trace** if it's disabled.
47
-
3. Check any log category you need.
48
-
4. Click **Save** button and wait until the settings take effect.
49
-
5. Click *Open Live Trace Tool*
34
+
1. Go to the Azure portal and your Web PubSub service.
35
+
1. On the left menu, under **Monitoring**, select **Live trace settings.**
36
+
1. On the **Live trace settings** page, select **Enable Live Trace**.
37
+
1. Choose the log categories to collect.
38
+
1. Select **Save** and then wait until the settings take effect.
39
+
1. Select **Open Live Trace Tool**.
50
40
51
41
:::image type="content" source="./media/howto-troubleshoot-diagnostic-logs/diagnostic-logs-with-live-trace-tool.png" alt-text="Screenshot of launching the live trace tool.":::
52
42
53
-
> Azure Active Directory access to live trace tool is not yet supported, please enable `Access Key` in `Keys` menu.
54
-
55
43
### Capture the resource logs
56
44
57
-
The live trace tool provides some fundamental functionalities to help you capture the resource logs for troubleshooting.
45
+
The live trace tool provides functionality to help you capture the resource logs for troubleshooting.
58
46
59
-
***Capture**: Begin to capture the real-time resource logs from Azure Web PubSub instance with live trace tool.
47
+
***Capture**: Begin to capture the real-time resource logs from Azure Web PubSub.
60
48
***Clear**: Clear the captured real-time resource logs.
61
-
***Log filter**: The live trace tool allows you filtering the captured real-time resource logs with one specific key word. The common separator (for example, space, comma, semicolon, and so on) will be treated as part of the key word.
49
+
***Log filter**: The live trace tool lets you filter the captured real-time resource logs with one specific key word. The common separators (for example, space, comma, semicolon, and so on) will be treated as part of the key word.
62
50
***Status**: The status shows whether the live trace tool is connected or disconnected with the specific instance.
63
51
64
52
:::image type="content" source="./media/howto-troubleshoot-diagnostic-logs/live-trace-tool-capture.png" alt-text="Screenshot of capturing resource logs with live trace tool.":::
@@ -68,47 +56,43 @@ The real-time resource logs captured by live trace tool contain detailed informa
| Log Level | Log event level, can be [Trace\|Debug\|Informational\|Warning\|Error]|
72
60
| Event Name | Operation name of the event |
73
-
| Message | Detailed message of log event |
61
+
| Message | Detailed message for the event |
74
62
| Exception | The run-time exception of Azure Web PubSub service |
75
-
| Hub | User-defined Hub Name|
63
+
| Hub | User-defined hub name|
76
64
| Connection ID | Identity of the connection |
77
-
| User ID |Identity of the user |
78
-
| IP |The IP address of client|
65
+
| User ID |User identity|
66
+
| IP |Client IP address |
79
67
| Route Template | The route template of the API |
80
68
| Http Method | The Http method (POST/GET/PUT/DELETE) |
81
69
| URL | The uniform resource locator |
82
70
| Trace ID | The unique identifier to the invocation |
83
71
| Status Code | The Http response code |
84
-
| Duration | The duration between the request is received and processed|
72
+
| Duration | The duration between receiving the request and processing the request|
85
73
| Headers | The additional information passed by the client and the server with an HTTP request or response |
86
74
87
-
After the Azure Web PubSub service is GA, the live trace tool will also support to export the logs as a specific format and then help you share with others for troubleshooting.
88
-
89
75
## Capture resource logs with Azure Monitor
90
76
91
77
### How to enable resource logs
92
78
93
-
Currently Azure Web PubSub supports integrate with [Azure Storage](../azure-monitor/essentials/resource-logs.md#send-to-azure-storage).
79
+
Currently Azure Web PubSub supports integration with [Azure Storage](../azure-monitor/essentials/resource-logs.md#send-to-azure-storage).
94
80
95
81
1. Go to Azure portal.
96
-
2. On **Diagnostic settings** page of your Azure Web PubSub service instance, click **+ Add diagnostic setting** link.
97
-
3. In **Diagnostic setting name**, input the setting name.
98
-
4. In **Category details**, select any log category you need.
99
-
5. In **Destination details**, check **Archive to a storage account**.
100
-
6. Click **Save** button to save the diagnostic setting.
101
-
102
-
:::image type="content" source="./media/howto-troubleshoot-diagnostic-logs/diagnostic-settings-list.png" alt-text="Screenshot of viewing diagnostic settings and create a new one":::
103
-
104
-
:::image type="content" source="./media/howto-troubleshoot-diagnostic-logs/diagnostic-settings-details.png" alt-text="Screenshot of configuring diagnostic setting detail":::
105
-
82
+
1. On **Diagnostic settings** page of your Azure Web PubSub service instance, select **+ Add diagnostic setting**.
83
+
:::image type="content" source="./media/howto-troubleshoot-diagnostic-logs/diagnostic-settings-list.png" alt-text="Screenshot of viewing diagnostic settings and create a new one":::
84
+
1. In **Diagnostic setting name**, input the setting name.
85
+
1. In **Category details**, select any log category you need.
86
+
1. In **Destination details**, check **Archive to a storage account**.
87
+
88
+
:::image type="content" source="./media/howto-troubleshoot-diagnostic-logs/diagnostic-settings-details.png" alt-text="Screenshot of configuring diagnostic setting detail":::
89
+
1. Select **Save** to save the diagnostic setting.
106
90
> [!NOTE]
107
-
> The storage account should be the same region to Azure Web PubSub service.
91
+
> The storage account should be in the same region as Azure Web PubSub service.
108
92
109
-
### Archive to Azure Storage Account
93
+
### Archive to an Azure Storage Account
110
94
111
-
Logs are stored in the storage account that configured in **Diagnostics setting** pane. A container named `insights-logs-<CATEGORY_NAME>` is created automatically to store resource logs. Inside the container, logs are stored in the file `resourceId=/SUBSCRIPTIONS/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/RESOURCEGROUPS/XXXX/PROVIDERS/MICROSOFT.SIGNALRSERVICE/SIGNALR/XXX/y=YYYY/m=MM/d=DD/h=HH/m=00/PT1H.json`. Basically, the path is combined by `resource ID` and `Date Time`. The log files are split by `hour`. Therefore, the minutes always be`m=00`.
95
+
Logs are stored in the storage account that's configured in the **Diagnostics setting** pane. A container named `insights-logs-<CATEGORY_NAME>` is created automatically to store resource logs. Inside the container, logs are stored in the file `resourceId=/SUBSCRIPTIONS/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/RESOURCEGROUPS/XXXX/PROVIDERS/MICROSOFT.SIGNALRSERVICE/SIGNALR/XXX/y=YYYY/m=MM/d=DD/h=HH/m=00/PT1H.json`. The path is combined by `resource ID` and `Date Time`. The log files are split by `hour`. The minute value is always`m=00`.
112
96
113
97
All logs are stored in JavaScript Object Notation (JSON) format. Each entry has string fields that use the format described in the following sections.
114
98
@@ -124,7 +108,7 @@ resourceId | Resource ID of your Azure SignalR Service
124
108
location | Location of your Azure SignalR Service
125
109
category | Category of the log event
126
110
operationName | Operation name of the event
127
-
callerIpAddress | IP address of your server/client
111
+
callerIpAddress | IP address of your server or client
128
112
properties | Detailed properties related to this log event. For more detail, see the properties table below
129
113
130
114
**Properties Table**
@@ -166,25 +150,31 @@ The following code is an example of an archive log JSON string:
166
150
167
151
### Archive to Azure Log Analytics
168
152
169
-
Once you check `Send to Log Analytics`, and select target Azure Log Analytics, the logs will be stored in the target. To view resource logs, follow these steps:
153
+
To send logs to a Log Analytics workspace:
154
+
1. On the **Diagnostic setting** page, under **Destination details**, select **Send to Log Analytics workspace.
155
+
1. Select the **Subscription** you want to use.
156
+
1. Select the **Log Analytics workspace** to use as the destination for the logs.
170
157
171
-
1. Select `Logs` in your target Log Analytics.
158
+
To view the resource logs, follow these steps:
159
+
160
+
1. Select `Logs` in your target Log Analytics workspace.
172
161
173
162
:::image type="content" alt-text="Log Analytics menu item" source="./media/howto-troubleshoot-diagnostic-logs/log-analytics-menu-item.png" lightbox="./media/howto-troubleshoot-diagnostic-logs/log-analytics-menu-item.png":::
174
163
175
-
2. Enter `WebPubSubConnectivity`, `WebPubSubMessaging` or `WebPubSubHttpRequest` and select time range to query [connectivity log](#connectivity-logs), [messaging log](#messaging-logs) or [http request logs](#http-request-logs) correspondingly. For advanced query, see [Get started with Log Analytics in Azure Monitor](../azure-monitor/logs/log-analytics-tutorial.md)
164
+
1. Enter `WebPubSubConnectivity`, `WebPubSubMessaging` or `WebPubSubHttpRequest`, and then select the time range to query the log. For advanced queries, see [Get started with Log Analytics in Azure Monitor](../azure-monitor/logs/log-analytics-tutorial.md).
176
165
177
166
:::image type="content" alt-text="Query log in Log Analytics" source="./media/howto-troubleshoot-diagnostic-logs/query-log-in-log-analytics.png" lightbox="./media/howto-troubleshoot-diagnostic-logs/query-log-in-log-analytics.png":::
178
167
179
-
To use sample query for SignalR service, please follow the steps below:
168
+
169
+
To use a sample query for SignalR service, follow the steps below.
180
170
1. Select `Logs` in your target Log Analytics.
181
-
2. Select `Queries` to open query explorer.
182
-
3. Select `Resource type` to group sample queries in resource type.
183
-
4. Select `Run` to run the script.
171
+
1. Select `Queries` to open query explorer.
172
+
1. Select `Resource type` to group sample queries in resource type.
173
+
1. Select `Run` to run the script.
184
174
:::image type="content" alt-text="Sample query in Log Analytics" source="./media/howto-troubleshoot-diagnostic-logs/log-analytics-sample-query.png" lightbox="./media/howto-troubleshoot-diagnostic-logs/log-analytics-sample-query.png":::
185
175
186
176
187
-
Archive log columns include elements listed in the following table:
177
+
Archive log columns include elements listed in the following table.
188
178
189
179
Name | Description
190
180
------- | -------
@@ -202,15 +192,15 @@ TransportType | Transport type of the connection. Allowed values are: `Websocket
202
192
203
193
## Troubleshoot with the resource logs
204
194
205
-
When finding connection unexpected growing or dropping situation, you can take advantage of resource logs to troubleshoot. Typical issues are often about connections' unexpected quantity changes, connections reach connection limits and authorization failure.
195
+
If you find unexpected changes in the number of connections, either increasing or decreasing, you can take advantage of resource logs to troubleshoot the problem. Typical issues are often about connections' unexpected quantity changes, connections reach connection limits, and authorization failure.
206
196
207
-
### Unexpected connection number changes
197
+
### Unexpected changes in number of connections
208
198
209
199
#### Unexpected connection dropping
210
200
211
-
If a connection disconnects, the resource logs will record this disconnecting event with `ConnectionAborted` or `ConnectionEnded` in `operationName`.
201
+
If a connection disconnects, the resource logs will record the disconnection event with `ConnectionAborted` or `ConnectionEnded` in `operationName`.
212
202
213
-
The difference between `ConnectionAborted` and `ConnectionEnded` is that `ConnectionEnded` is an expected disconnecting which is triggered by client or server side. While the `ConnectionAborted` is usually an unexpected connection dropping event, and aborting reason will be provided in `message`.
203
+
The difference between `ConnectionAborted` and `ConnectionEnded` is that `ConnectionEnded` is an expected disconnection that is triggered by the client or server side. While the `ConnectionAborted` is usually an unexpected connection dropping event, and the reason for disconnection will be provided in `message`.
214
204
215
205
The abort reasons are listed in the following table:
216
206
@@ -220,14 +210,14 @@ The abort reasons are listed in the following table:
220
210
| Service reloading, reconnect | Azure Web PubSub service is reloading. You need to implement your own reconnect mechanism or manually reconnect to Azure Web PubSub service |
221
211
| Internal server transient error | Transient error occurs in Azure Web PubSub service, should be auto recovered
222
212
223
-
#### Unexpected connection growing
213
+
#### Unexpected increase in connections
224
214
225
-
To troubleshoot about unexpected connection growing, the first thing you need to do is to filter out the extra connections. You can add unique test user ID to your test client connection. Then verify it in with resource logs, if you see more than one client connections have the same test user ID or IP, then it is likely the client side create and establish more connections than expectation. Check your client side.
215
+
When the number of client connections unexpectedly increases, the first thing you need to do is to filter out the superfluous connections. Add a unique test user ID to your test client connection. Then check the resource logs; if you see more than one client connection has the same test user ID or IP, then it's likely the client is creating more connections than expected. Check your client code to find the source of the extra connections.
226
216
227
217
### Authorization failure
228
218
229
-
If you get 401 Unauthorized returned for client requests, check your resource logs. If you meet`Failed to validate audience. Expected Audiences: <valid audience>. Actual Audiences: <actual audience>`, it means all audiences in your access token are invalid. Try to use the valid audiences suggested in the log.
219
+
If you get 401 Unauthorized returned for client requests, check your resource logs. If you find`Failed to validate audience. Expected Audiences: <valid audience>. Actual Audiences: <actual audience>`, it means all audiences in your access token are invalid. Try to use the valid audiences suggested in the log.
230
220
231
221
### Throttling
232
222
233
-
If you find that you cannot establish client connections to Azure Web PubSub service, check your resource logs. If you meet`Connection count reaches limit` in resource log, you establish too many connections to Azure Web PubSub service, which reach the connection count limit. Consider scaling up your Azure Web PubSub service instance. If you meet`Message count reaches limit` in resource log, it means you use free tier, and you use up the quota of messages. If you want to send more messages, consider changing your Azure Web PubSub service instance to standard tier to send additional messages. For more information, see [Azure Web PubSub service Pricing](https://azure.microsoft.com/pricing/details/web-pubsub/).
223
+
If you find that you can't establish client connections to Azure Web PubSub service, check your resource logs. If you see`Connection count reaches limit` in the resource log, you established too many connections to Azure Web PubSub service and reached the connection count limit. Consider scaling up your Azure Web PubSub service instance. If you see`Message count reaches limit` in the resource log and you're using the Free tier, it means you used up the quota of messages. If you want to send more messages, consider changing your Azure Web PubSub service instance to Standard tier. For more information, see [Azure Web PubSub service Pricing](https://azure.microsoft.com/pricing/details/web-pubsub/).
0 commit comments