You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cdn/cdn-azure-diagnostic-logs.md
+38-49Lines changed: 38 additions & 49 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,53 +20,47 @@ ms.author: v-deasim
20
20
21
21
# Azure diagnostic logs
22
22
23
-
Azure diagnostic logs is available for all CDN endpoints belonging to Verizon (Standard and Premium) and Akamai (Standard) CDN Profiles. For more information, see [Diagnostic Logs](https://docs.microsoft.com/azure/monitoring-and-diagnostics/monitoring-overview-of-diagnostic-logs).
24
-
25
-
With this feature, you can view core analytics and save them into one or more destinations including:
23
+
With Azure diagnostic logs, you can view core analytics and save them into one or more destinations including:
This feature is available for all CDN endpoints belonging to Verizon (Standard and Premium) and Akamai (Standard) CDN Profiles.
32
-
33
-
For more information about diagnostic logs, see [Diagnostic Logs](https://docs.microsoft.com/azure/monitoring-and-diagnostics/monitoring-overview-of-diagnostic-logs).
29
+
This feature is available for all CDN endpoints belonging to Verizon (Standard and Premium) and Akamai (Standard) CDN Profiles.
34
30
35
31
Azure diagnostics logs allow you to export basic usage metrics from your CDN endpoint to a variety of sources so that you can consume them in a customized way. For example, you can do the following types of data export:
36
32
37
33
- Export data to blob storage, export to CSV, and generate graphs in Excel.
38
34
- Export data to Event Hubs and correlate with data from other Azure services.
39
35
- Export data to log analytics and view data in your own OMS work space
40
36
41
-
The following figure shows a typical CDN core analytics view into data.
37
+
The following figure shows a typical CDN core analytics view of data.
The following walkthrough goes through the schema of the core analytics data, steps involved in enabling the feature and delivering them to various destinations, and consuming from these destinations.
43
+
For more information about diagnostic logs, see [Diagnostic Logs](https://docs.microsoft.com/azure/monitoring-and-diagnostics/monitoring-overview-of-diagnostic-logs).
48
44
49
45
## Enable logging with Azure portal
50
46
51
-
> [!NOTE]
52
-
> The diagnostics logs are turned **off** by default.
53
-
54
-
Follow the steps below to enable logging with CDN core analytics:
47
+
Follow these steps enable logging with CDN core analytics:
55
48
56
49
Sign in to the [Azure portal](http://portal.azure.com). If you don't already have CDN enabled for your workflow, [Enable Azure CDN](cdn-create-new-endpoint.md) before you continue.
57
50
58
51
1. In the portal, navigate to **CDN profile**.
59
-
2. Select a CDN profile, then select the CDN endpoint that you want to enable **Diagnostics Logs**.
52
+
2. Select a CDN profile, then select the CDN endpoint for which you want to enable **Diagnostics Logs**.
To use Azure Storage to store the logs, select **Archive to a storage account**, select retention days, and click **CoreAnalytics** under **Log**.
62
+
1. To use Azure storage to store the logs, select **Archive to a storage account**, select **CoreAnalytics**, and then choose the number of retention days under **Retention (days)**. A retention of zero days stores the logs indefinitely.
63
+
2. Enter a name for your setting, then click **Storage account**. After you have selected a storage account, click **Save**.
2. Configure the Log Analytics logging by clicking on Configure. This takes you to a dialog where you can select a previous workspace or create a new one.
77
+
2.Click **Configure** to configure log analytics logging. In the OMS Workspaces dialog box, you can select a previous workspace or create a new one.
4. Next you must select a new workspace name, existing subscription, resource group (new or existing), location, and pricing tier. You have the option of pinning this configuration to your dashboard. Click OK to complete the configuration.
92
-
93
-
Next you should see your workspace with your OMS Workspace and Resource group names. Names must be unique and can only use letters, numbers, and hyphens. Spaces and underscores are not allowed.
85
+
4. Enter a new OMS workspace name. An OMS workspace name must be unique and contain only letters, numbers, and hyphens; spaces and underscores are not allowed.
86
+
5. Next, select an existing subscription, resource group (new or existing), location, and pricing tier. You also have the option of pinning this configuration to your dashboard. Click **OK** to complete the configuration.
5.You next get a short message saying that your workspace has been created and you are returned to your logging configuration screen. You can confirm the name of your Log Analytics workspace.
90
+
5. After your workspace is created, you are returned to your Diagnostic logs windows. Confirm the name of your new log analytics workspace.
Once you have set up the log analytics configuration, verify that you have selected **CoreAnalytics**.
106
95
107
-
The **Save** button is no longer active and that the ON/OFF button is now ON, but blue instead of purple.
96
+
6. Click **Save**.
108
97
109
-
7.If you want to see your new OMS workspace, go to your Azure portal Dashboard, click the name of your Log Analytics workspace. Next you will see your workspace (make sure that OMS Workspace is highlighted on the left). Click on the OMS Portal tile to see your workspace in the OMS repository.
98
+
7.To view your new OMS workspace, go to your Azure portal dashboard and click the name of your log analytics workspace. Click the OMS Portal tile to view your workspace in the OMS repository.
@@ -116,7 +105,7 @@ For more information about log data delays, see [Log data delays](#log-data-dela
116
105
117
106
## Enable logging with PowerShell
118
107
119
-
Below is an example on how to enable and get Diagnostic Logs via the Azure PowerShell Cmdlets.
108
+
The following example shows how to enable Diagnostic Logs via the Azure PowerShell Cmdlets.
120
109
121
110
###Enabling Diagnostic Logs in a Storage Account
122
111
@@ -142,10 +131,10 @@ To Enable Diagnostics Logs in an OMS workspace, use this command:
142
131
143
132
144
133
## Consuming diagnostics logs from Azure Storage
145
-
This section describes the schema of the CDN core analytics, how these are organized inside of an Azure Storage Account and provides sample code to download the logs into a CSV file.
134
+
This section describes the schema of CDN core analytics, how it is organized inside of an Azure storage account, and provides sample code to download the logs in a CSV file.
146
135
147
136
### Using Microsoft Azure Storage Explorer
148
-
Before you can access the core analytics data from the Azure Storage Account, you first need a tool to access the contents in a storage account. While there are several tools available in the market, the one that we recommend is the Microsoft Azure Storage Explorer. You can download the tool from [here](http://storageexplorer.com/). After downloading and installing the software, configure it to use the same Azure Storage Account that was configured as a destination to the CDN Diagnostics Logs.
137
+
Before you can access the core analytics data from the Azure Storage Account, you first need a tool to access the contents in a storage account. While there are several tools available in the market, the one that we recommend is the Microsoft Azure Storage Explorer. To download the tool, see [Azure Storage Explorer](http://storageexplorer.com/). After downloading and installing the software, configure it to use the same Azure storage account that was configured as a destination to the CDN Diagnostics Logs.
149
138
150
139
1. Open **Microsoft Azure Storage Explorer**
151
140
2. Locate the storage account
@@ -159,34 +148,34 @@ Before you can access the core analytics data from the Azure Storage Account, yo
159
148
> [!NOTE]
160
149
> **Blob path format**
161
150
>
162
-
> Core analytics logs are generated every hour. All data for an hour are collected and stored inside a single Azure Blob as a JSON payload. The path to this Azure Blob appears as if there is a hierarchical structure. This is because the Storage explorer tool interprets '/' as a directory separator and shows the hierarchy for convenience. Actually, the whole path just represents the blob name. This name of the blob follows the following naming convention
151
+
> Core analytics logs are generated every hour and the data is collected and stored inside a single Azure blob as a JSON payload. Because the Storage explorer tool interprets '/' as a directory separator and shows the hierarchy, the path to the Azure blob appears as if there is a hierarchical structure and represents the blob name. The name of the blob follows the following naming convention:
163
152
164
153
resourceId=/SUBSCRIPTIONS/{Subscription Id}/RESOURCEGROUPS/{Resource Group Name}/PROVIDERS/MICROSOFT.CDN/PROFILES/{Profile Name}/ENDPOINTS/{Endpoint Name}/ y={Year}/m={Month}/d={Day}/h={Hour}/m={Minutes}/PT1H.json
165
154
166
155
**Description of fields:**
167
156
168
157
|value|description|
169
158
|-------|---------|
170
-
|Subscription ID |ID of the Azure Subscription. This is in the Guid format.|
159
+
|Subscription ID |ID of the Azure subscription in Guid format.|
171
160
|Resource |Group Name Name of the resource group to which the CDN resources belong.|
172
161
|Profile Name |Name of the CDN Profile|
173
162
|Endpoint Name |Name of the CDN Endpoint|
174
-
|Year| 4-digit representation of the year for example, 2017|
163
+
|Year| 4-digit representation of the year, for example, 2017|
175
164
|Month| 2-digit representation of the month number. 01=January ... 12=December|
176
-
|Day| 2digit representation of the day of the month|
165
+
|Day| 2-digit representation of the day of the month|
177
166
|PT1H.json| Actual JSON file where the analytics data is stored|
178
167
179
168
### Exporting the core analytics data to a CSV file
180
169
181
-
To make it easy to access core analytics, we provide a sample code for a tool, which allows downloading the JSON files into a flat comma-separated file format, which can be used to easily create charts or other aggregations.
170
+
To make it easy to access core analytics, sample code for a tool is provided. This tool allows downloading the JSON files into a flat comma-separated file format, which can be used to easily create charts or other aggregations.
182
171
183
172
Here is how you can use the tool:
184
173
185
174
1. Visit the github link: [https://github.com/Azure-Samples/azure-cdn-samples/tree/master/CoreAnalytics-ExportToCsv](https://github.com/Azure-Samples/azure-cdn-samples/tree/master/CoreAnalytics-ExportToCsv)
186
-
2. Download the code
187
-
3. Follow instructions to compile and configure
188
-
4. Run the tool
189
-
5. Resulting CSV file shows the analytics data in a simple flat hierarchy.
175
+
2. Download the code.
176
+
3. Follow the instructions to compile and configure.
177
+
4. Run the tool.
178
+
5. The resulting CSV file shows the analytics data in a simple flat hierarchy.
190
179
191
180
## Consuming diagnostics logs from an OMS Log Analytics repository
192
181
Log Analytics is a service in Operations Management Suite (OMS) that monitors your cloud and on-premises environments to maintain their availability and performance. It collects data generated by resources in your cloud and on-premises environments and from other monitoring tools to provide analysis across multiple sources.
@@ -244,7 +233,7 @@ Follow these steps to add a Management Solution:
@@ -274,11 +263,11 @@ You can see offers and pricing tiers for OMS management solutions [here](https:/
274
263
275
264
### Customizing views
276
265
277
-
You can customize the view into your data by using the **View Designer**. Go to your OMS workspace and begin designing by clicking the **View Designer** tile.
266
+
You can customize the view into your data by using the **View Designer**. To begin designing, go to your OMS workspace and click the **View Designer** tile.
@@ -287,7 +276,7 @@ You can drag and drop types of charts from the left and fill in the data details
287
276
288
277
Verizon log data delays | Akamai log data delays
289
278
--- | ---
290
-
Verizon log data is 1 hour delayed, and take up to 2 hours to start appearing after endpoint propagation completion. | Akamai log data is 24 hours delayed, and takes up to 2 hours to start appearing if it was created more than 24 hours ago. If it was recently created, it can take up to 25 hours for the logs to start appearing.
279
+
Verizon log data is 1 hour delayed, and take up to 2 hours to start appearing after endpoint propagation completion. | Akamai log data is delayed by 24 hours; if it was created more than 24 hours ago, it takes up to 2 hours to start appearing. If it was recently created, it can take up to 25 hours for the logs to start appearing.
291
280
292
281
## Diagnostic log types for CDN core analytics
293
282
@@ -310,7 +299,7 @@ The following table shows a list of metrics available in the core analytics logs
310
299
| RequestCountHttpStatus302 | Count of all requests that resulted in a 302 HTTP code response |No |Yes |
311
300
| RequestCountHttpStatus304 | Count of all requests that resulted in a 304 HTTP code response |No |Yes |
312
301
| RequestCountHttpStatus404 | Count of all requests that resulted in a 404 HTTP code response |No |Yes |
313
-
| RequestCountCacheHit |Count of all requests that resulted in a Cache Hit. This means the asset was served directly from the POP to the Client. | Yes |No |
302
+
| RequestCountCacheHit |Count of all requests that resulted in a Cache Hit. The asset was served directly from the POP to the client. | Yes |No |
314
303
| RequestCountCacheMiss | Count of all requests that resulted in a Cache Miss. This means the asset was not found on the POP closest to the client, and therefore was retrieved from the Origin. |Yes | No |
315
304
| RequestCountCacheNoCache | Count of all requests to an asset that are prevented from being cached due to a user configuration on the edge. |Yes | No |
316
305
| RequestCountCacheUncacheable | Count of all requests to assets that are prevented from being cached by the asset's Cache-Control and Expires headers, which indicate that it should not be cached on a POP or by the HTTP client |Yes |No |
@@ -324,15 +313,15 @@ The following table shows a list of metrics available in the core analytics logs
324
313
| EgressCacheHit | Outbound data transfer for responses that were delivered directly from the CDN cache on the CDN POPs/Edges |Yes | No |
325
314
| EgressCacheMiss | Outbound data transfer for responses that were not found on the nearest POP server, and retrieved from the origin server |Yes | No |
326
315
| EgressCacheNoCache | Outbound data transfer for assets that are prevented from being cached due to a user configuration on the edge. |Yes |No |
327
-
| EgressCacheUncacheable | Outbound data transfer for assets that are prevented from being cached by the asset's Cache-Control and/or Expires headers, which indicate that it should not be cached on a POP or by the HTTP client|Yes | No |
316
+
| EgressCacheUncacheable | Outbound data transfer for assets that are prevented from being cached by the asset's Cache-Control and/or Expires headers. Indicates that it should not be cached on a POP or by the HTTP client.|Yes | No |
328
317
| EgressCacheOthers | Outbound data transfers for other cache scenarios. |Yes | No |
329
318
330
319
*Outbound data transfer refers to traffic delivered from CDN POP servers to the client.
331
320
332
321
333
322
### Schema of the core analytics logs
334
323
335
-
All logs are stored in JSON format and each entry has string fields following the below schema:
324
+
All logs are stored in JSON format and each entry has string fields according to the following schema:
336
325
337
326
```json
338
327
"records": [
@@ -377,9 +366,9 @@ All logs are stored in JSON format and each entry has string fields following th
377
366
}
378
367
```
379
368
380
-
Where the ‘time’ represents the start time of the hour boundary for which the statistics is reported. When a metric is not supported by a CDN provider, instead of a double or integer value, there will be a null value. This null value indicates the absence of a metric, and this is different from a 0 value. Also note that there will be one set of these metrics per domain configured on the endpoint.
369
+
Where the ‘time’ represents the start time of the hour boundary for which the statistics is reported. When a metric is not supported by a CDN provider, instead of a double or integer value, there is a null value. This null value indicates the absence of a metric, and is different from a value of 0. There is one set of these metrics per domain configured on the endpoint.
0 commit comments