Skip to content

Commit 968ab7c

Browse files
guywi-msguywi-ms
committed
Update data-collection-text-log.md
1 parent d6a6d14 commit 968ab7c

File tree

1 file changed

+8
-187
lines changed

1 file changed

+8
-187
lines changed

‎articles/azure-monitor/agents/data-collection-text-log.md

Lines changed: 8 additions & 187 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,26 @@
11
---
2-
title: Collect text and IIS logs with Azure Monitor agent (preview)
3-
description: Configure collection of filed-based text logs using a data collection rule on virtual machines with the Azure Monitor agent.
2+
title: Collect text logs with Azure Monitor Agent
3+
description: Configure collection of filed-based text logs using a data collection rule on virtual machines with the Azure Monitor Agent.
44
ms.topic: conceptual
55
ms.date: 06/22/2022
66
ms.reviewer: shseth
77

88
---
99

10-
# Collect text and IIS logs with Azure Monitor agent (preview)
11-
This article describes how to configure the collection of file-based text logs, including logs generated by IIS on Windows computers, with the [Azure Monitor agent](azure-monitor-agent-overview.md). Many applications log information to text files instead of standard logging services such as Windows Event log or Syslog.
10+
# Collect text logs with Azure Monitor Agent
11+
This article describes how to configure the collection of file-based text logs with [Azure Monitor Agent](azure-monitor-agent-overview.md). Many applications log information to text files instead of standard logging services such as Windows Event log or Syslog.
1212

1313

1414
## Prerequisites
1515
To complete this procedure, you need the following:
1616

1717
- Log Analytics workspace where you have at least [contributor rights](../logs/manage-access.md#azure-rbac) .
1818
- [Permissions to create Data Collection Rule objects](../essentials/data-collection-rule-overview.md#permissions) in the workspace.
19-
- An agent with supported log file as described in the next section.
20-
- Azure Monitor collects entries from log files created by IIS, so you must configure IIS for logging. Azure Monitor only supports IIS log files stored in W3C format and does not support custom fields or IIS Advanced Logging. It does not collect logs in NCSA or IIS native format. Configure IIS logs in Azure Monitor from the Agent configuration menu for the Log Analytics agent. There is no configuration required other than selecting Collect W3C format IIS log files.
21-
22-
## Log files supported
23-
IIS logs must be in W3C format. Other log files must meet the following criteria to be collected:
24-
25-
- The log file must be stored on a local drive of a virtual machine, virtual machine scale set, or Arc enabled server with the Azure Monitor installed.
26-
- Each entry in the log file must be delineated with an [ISO 8601 formatted](https://www.iso.org/standard/40874.html) time stamp or an end of line.
27-
- The log file must not allow circular logging, log rotation where the file is overwritten with new entries, or the file is renamed and the same file name is reused for continued logging.
28-
29-
19+
- A machine that generates file-based text logs.
3020
## Steps to collect text logs
3121
The steps to configure log collection are as follows. The detailed steps for each are provided in the sections below:
3222

33-
1. Create a new table in your workspace to receive the collected data. (not required for IIS logs)
23+
1. Create a new table in your workspace to receive the collected data.
3424
2. Create a data collection endpoint for the Azure Monitor agent to connect.
3525
3. Create a data collection rule to define the structure of the log file and destination of the collected data.
3626
4. Create association between the data collection rule and the agent collecting the log file.
@@ -41,9 +31,6 @@ The steps to configure log collection are as follows. The detailed steps for eac
4131
## Create new table in Log Analytics workspace
4232
The custom table must be created before you can send data to it. When you create the table, you provide its name and a definition for each of its columns.
4333

44-
>[!NOTE]
45-
> This step isn't required to collect an IIS log. The table [W3CIISLog](/azure/azure-monitor/reference/tables/w3ciislog) will be used for IIS logs.
46-
4734
Use the **Tables - Update** API to create the table with the PowerShell code below. This code creates a table called *MyTable_CL* with two columns. Modify this schema to collect a different table.
4835

4936
> [!IMPORTANT]
@@ -308,96 +295,6 @@ The [data collection rule (DCR)](../essentials/data-collection-rule-overview.md)
308295
}
309296
```
310297
311-
**Data collection rule for IIS log**
312-
313-
```json
314-
{
315-
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
316-
"contentVersion": "1.0.0.0",
317-
"parameters": {
318-
"dataCollectionRuleName": {
319-
"type": "string",
320-
"metadata": {
321-
"description": "Specifies the name of the Data Collection Rule to create."
322-
}
323-
},
324-
"location": {
325-
"type": "string",
326-
"metadata": {
327-
"description": "Specifies the location in which to create the Data Collection Rule."
328-
}
329-
},
330-
"workspaceName": {
331-
"type": "string",
332-
"metadata": {
333-
"description": "Name of the Log Analytics workspace to use."
334-
}
335-
},
336-
"workspaceResourceId": {
337-
"type": "string",
338-
"metadata": {
339-
"description": "Specifies the Azure resource ID of the Log Analytics workspace to use."
340-
}
341-
},
342-
"endpointResourceId": {
343-
"type": "string",
344-
"metadata": {
345-
"description": "Specifies the Azure resource ID of the Data Collection Endpoint to use."
346-
}
347-
}
348-
},
349-
"resources": [
350-
{
351-
"type": "Microsoft.Insights/dataCollectionRules",
352-
"name": "[parameters('dataCollectionRuleName')]",
353-
"location": "[parameters('location')]",
354-
"apiVersion": "2021-09-01-preview",
355-
"properties": {
356-
"dataCollectionEndpointId": "[parameters('endpointResourceId')]",
357-
"dataSources": {
358-
"iisLogs": [
359-
{
360-
"streams": [
361-
"Microsoft-W3CIISLog"
362-
],
363-
"logDirectories": [
364-
"C:\\inetpub\\logs\\LogFiles\\W3SVC1\\"
365-
],
366-
"name": "myIisLogsDataSource"
367-
}
368-
]
369-
},
370-
"destinations": {
371-
"logAnalytics": [
372-
{
373-
"workspaceResourceId": "[parameters('workspaceResourceId')]",
374-
"name": "[parameters('workspaceName')]"
375-
}
376-
]
377-
},
378-
"dataFlows": [
379-
{
380-
"streams": [
381-
"Microsoft-W3CIISLog"
382-
],
383-
"destinations": [
384-
"[parameters('workspaceName')]"
385-
],
386-
"transformKql": "source"
387-
}
388-
]
389-
}
390-
}
391-
],
392-
"outputs": {
393-
"dataCollectionRuleId": {
394-
"type": "string",
395-
"value": "[resourceId('Microsoft.Insights/dataCollectionRules', parameters('dataCollectionRuleName'))]"
396-
}
397-
}
398-
}
399-
```
400-
401298
5. On the **Custom deployment** screen, specify a **Subscription** and **Resource group** to store the data collection rule and then provide values defined in the template. This includes a **Name** for the data collection rule and the **Workspace Resource ID** and **Endpoint Resource ID**. The **Location** should be the same location as the workspace. The **Region** will already be populated and is used for the location of the data collection rule.
402299
403300
:::image type="content" source="media/data-collection-text-log/custom-deployment-values.png" lightbox="media/data-collection-text-log/custom-deployment-values.png" alt-text="Screenshot that shows the Custom Deployment screen in the portal to edit custom deployment values for data collection rule.":::
@@ -429,10 +326,9 @@ The final step is to create a data collection association that associates the da
429326
430327
:::image type="content" source="media/data-collection-text-log/select-resources.png" lightbox="media/data-collection-text-log/select-resources.png" alt-text="Screenshot that shows the Resources pane in the portal to add resources to the data collection rule.":::
431328
432-
## Troubleshooting - text logs
329+
## Troubleshooting
433330
Use the following steps to troubleshoot collection of text logs.
434331
435-
436332
### Check if any custom logs have been received
437333
Start by checking if any records have been collected for your custom log table by running the following query in Log Analytics. If no records are returned then check the other sections for possible causes. This query looks for entires in the last two days, but you can modify for another time range. It can take 5-7 minutes for new data from your tables to be uploaded. Only new data will be uploaded any log file last written to prior to the DCR rules being created will not be uploaded.
438334
@@ -533,83 +429,8 @@ If everything is configured properly, but you're still not collecting log data,
533429
4. Share the `AMAFiles.zip` file generated on the desktop.
534430

535431

536-
## Troubleshoot - IIS logs
537-
Use the following steps to troubleshoot collection of IIS logs.
538-
539-
### Check if any IIS logs have been received
540-
Start by checking if any records have been collected for your IIS logs by running the following query in Log Analytics. If no records are returned then check the other sections for possible causes. This query looks for entires in the last two days, but you can modify for another time range.
541-
542-
``` kusto
543-
W3CIISLog
544-
| where TimeGenerated > ago(48h)
545-
| order by TimeGenerated desc
546-
```
547-
548-
### Verify that the agent is sending heartbeats successfully
549-
Verify that Azure Monitor agent is communicating properly by running the following query in Log Analytics to check if there are any records in the Heartbeat table.
550-
551-
``` kusto
552-
Heartbeat
553-
| where TimeGenerated > ago(24h)
554-
| where Computer has "<computer name>"
555-
| project TimeGenerated, Category, Version
556-
| order by TimeGenerated desc
557-
```
558-
559-
### Verify that IIS logs are being created
560-
Look at the timestamps of the log files and open the latest to see that latest timestamps are present in the log files. The default location for IIS log files is C:\\inetpub\\LogFiles\\W3SVC1.
561-
562-
:::image type="content" source="media/data-collection-text-log/iis-log-timestamp.png" lightbox="media/data-collection-text-log/iis-log-timestamp.png" alt-text="Screenshot of I I S log on agent machine showing the timestamp.":::
563-
564-
### Verify that you specified the correct log location in the data collection rule
565-
The data collection rule will have a section similar to the following. The `logDirectories` element specifies the path to the log file to collect from the agent computer. Check the agent computer to verify that this is correct.
566-
567-
``` json
568-
"dataSources": [
569-
{
570-
"configuration": {
571-
"logDirectories": ["C:\\scratch\\demo\\W3SVC1"]
572-
},
573-
"id": "myIisLogsDataSource",
574-
"kind": "iisLog",
575-
"streams": [{
576-
"stream": "ONPREM_IIS_BLOB_V2"
577-
}
578-
],
579-
"sendToChannels": ["gigl-dce-6a8e34db54bb4b6db22d99d86314eaee"]
580-
}
581-
]
582-
```
583-
584-
This directory should correspond to the location of the IIS logs on the agent machine.
585-
586-
:::image type="content" source="media/data-collection-text-log/iis-log-files.png" lightbox="media/data-collection-text-log/iis-log-files.png" alt-text="Screenshot of I I S log files on agent machine.":::
587-
588-
### Verify that the IIS logs are W3C formatted
589-
Open IIS Manager and verify that the logs are being written in W3C format.
590-
591-
:::image type="content" source="media/data-collection-text-log/iis-log-format-setting.png" lightbox="media/data-collection-text-log/iis-log-format-setting.png" alt-text="Screenshot of I I S logging configuration dialog box on agent machine.":::
592-
593-
Open IIS log on the agent machine to verify logs are in W3C format.
594-
595-
:::image type="content" source="media/data-collection-text-log/iis-log-format.png" lightbox="media/data-collection-text-log/iis-log-format.png" alt-text="Screenshot of I I S log on agent machine showing the header specifies W3C format.":::
596-
597-
598-
599-
### Share logs with Microsoft
600-
If everything is configured properly, but you're still not collecting log data, use the following procedure to collect diagnostics logs for Azure Monitor agent to share with the Azure Monitor group.
601-
602-
1. Open an elevated PowerShell window.
603-
2. Change to directory `C:\Packages\Plugins\Microsoft.Azure.Monitor.AzureMonitorWindowsAgent\[version]\`.
604-
3. Execute the script: `.\CollectAMALogs.ps1`.
605-
4. Share the `AMAFiles.zip` file generated on the desktop.
606-
607-
608-
609-
610-
611432
## Next steps
612433

613-
- Learn more about the [Azure Monitor agent](azure-monitor-agent-overview.md).
434+
- Learn more about the [Azure Monitor Agent](azure-monitor-agent-overview.md).
614435
- Learn more about [data collection rules](../essentials/data-collection-rule-overview.md).
615436
- Learn more about [data collection endpoints](../essentials/data-collection-endpoint-overview.md).

0 commit comments

Comments
 (0)