Skip to content

Commit 37c54a3

Browse files
AnnaMHuffAnnaMHuff
authored
Merge pull request #203564 from bwren/edit-dcr
Edit data collection rules
2 parents 69170c1 + f7eed3a commit 37c54a3

File tree

2 files changed

+126
-0
lines changed

2 files changed

+126
-0
lines changed

‎articles/azure-monitor/essentials/data-collection-rule-edit.md

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
---
2+
title: Tutorial - Editing Data Collection Rules
3+
description: This article describes how to make changes in Data Collection Rule definition using command line tools and simple API calls.
4+
ms.topic: tutorial
5+
author: bwren
6+
ms.author: bwren
7+
ms.reviewer: ivankh
8+
ms.date: 05/31/2022
9+
---
10+
11+
# Tutorial: Editing Data Collection Rules
12+
This tutorial will describe how to edit the definition of Data Collection Rule (DCR) that has been already provisioned using command line tools.
13+
14+
In this tutorial, you learn how to:
15+
> [!div class="checklist"]
16+
> * Leverage existing portal functionality to pre-create DCRs
17+
> * Get the content of a Data Collection Rule using ARM API call
18+
> * Apply changes to a Data Collection Rule using ARM API call
19+
> * Automate the process of DCR update using PowerShell scripts
20+
21+
## Prerequisites
22+
To complete this tutorial you need the following:
23+
- Log Analytics workspace where you have at least [contributor rights](../logs/manage-access.md#azure-rbac).
24+
- [Permissions to create Data Collection Rule objects](data-collection-rule-overview.md#permissions) in the workspace.
25+
- Up to date version of PowerShell. Using Azure Cloud Shell is recommended.
26+
27+
## Overview of tutorial
28+
While going through the wizard on the portal is the simplest way to set up the ingestion of your custom data to Log Analytics, in some cases you might want to update your Data Collection Rule later to:
29+
- Change data collection settings (e.g. Data Collection Endpoint, associated with the DCR)
30+
- Update data parsing or filtering logic for your data stream
31+
- Change data destination (e.g. send data to an Azure table, as this option is not directly offered as part of the DCR-based custom log wizard)
32+
33+
In this tutorial, you will, first, set up ingestion of a custom log, then. you will modify the KQL transformation for your custom log to include additional filtering and apply the changes to your DCR. Finally, we are going to combine all editing operations into a single PowerShell script, which can be used to edit any DCR for any of the above mentioned reasons.
34+
35+
## Set up new custom log
36+
Start by setting up a new custom log. Follow [Tutorial: Send custom logs to Azure Monitor Logs using the Azure portal (preview)]( ../logs/tutorial-custom-logs.md). Note the resource ID of the DCR created.
37+
38+
## Retrieve DCR content
39+
In order to update DCR, we are going to retrieve its content and save it as a file, which can be further edited.
40+
1. Click the **Cloud Shell** button in the Azure portal and ensure the environment is set to **PowerShell**.
41+
42+
:::image type="content" source="../logs/media/tutorial-ingestion-time-transformations-api/open-cloud-shell.png" lightbox="../logs/media/tutorial-ingestion-time-transformations-api/open-cloud-shell.png" alt-text="Screenshot of opening cloud shell":::
43+
44+
2. Execute the following commands to retrieve DCR content and save it to a file. Replace `<ResourceId>` with DCR ResourceID and `<FilePath>` with the name of the file to store DCR.
45+
46+
```PowerShell
47+
$ResourceId = “<ResourceId>” # Resource ID of the DCR to edit
48+
$FilePath = “<FilePath>” # Store DCR content in this file
49+
$DCR = Invoke-AzRestMethod -Path ("$ResourceId"+"?api-version=2021-09-01-preview") -Method GET
50+
$DCR.Content | ConvertFrom-Json | ConvertTo-Json -Depth 20 | Out-File -FilePath $FilePath
51+
```
52+
## Edit DCR
53+
Now, when DCR content is stored as a JSON file, you can use an editor of your choice to make changes in the DCR. You may [prefer to download the file from the Cloud Shell environment](../../cloud-shell/using-the-shell-window.md#upload-and-download-files), if you are using one.
54+
55+
Alternatively you can use code editors supplied with the environment. For example, if you saved your DCR in a file named `temp.dcr` on your Cloud Drive, you could use the following command to open DCR for editing right in the Cloud Shell window:
56+
```PowerShell
57+
code "temp.dcr"
58+
```
59+
60+
Let’s modify the KQL transformation within DCR to drop rows where RequestType is anything, but “GET”.
61+
1. Open the file created in the previous part for editing using an editor of your choice.
62+
2. Locate the line containing `”transformKql”` attribute, which, if you followed the tutorial for custom log creation, should look similar to this:
63+
``` JSON
64+
"transformKql": " source\n | extend TimeGenerated = todatetime(Time)\n | parse RawData with \n ClientIP:string\n ' ' *\n ' ' *\n ' [' * '] \"' RequestType:string\n \" \" Resource:string\n \" \" *\n '\" ' ResponseCode:int\n \" \" *\n | where ResponseCode != 200\n | project-away Time, RawData\n"
65+
```
66+
3. Modify KQL transformation to include additional filter by RequestType
67+
``` JSON
68+
"transformKql": " source\n | where RawData contains \"GET\"\n | extend TimeGenerated = todatetime(Time)\n | parse RawData with \n ClientIP:string\n ' ' *\n ' ' *\n ' [' * '] \"' RequestType:string\n \" \" Resource:string\n \" \" *\n '\" ' ResponseCode:int\n \" \" *\n | where ResponseCode != 200\n | project-away Time, RawData\n"
69+
```
70+
4. Save the file with modified DCR content.
71+
72+
## Apply changes
73+
Our final step is to update DCR back in the system. This is accomplished by “PUT” HTTP call to ARM API, with updated DCR content sent in the HTTP request body.
74+
1. If you are using Azure Cloud Shell, save the file and close the embedded editor, or [upload modified DCR file back to the Cloud Shell environment](../../cloud-shell/using-the-shell-window.md#upload-and-download-files).
75+
2. Execute the following commands to load DCR content from the file and place HTTP call to update the DCR in the system. Replace `<ResourceId>` with DCR ResourceID and `<FilePath>` with the name of the file modified in the previous part of the tutorial. You can omit first two lines if you read and write to the DCR within the same PowerShell session.
76+
```PowerShell
77+
$ResourceId = “<ResourceId>” # Resource ID of the DCR to edit
78+
$FilePath = “<FilePath>” # Store DCR content in this file
79+
$DCRContent = Get-Content $FilePath -Raw
80+
Invoke-AzRestMethod -Path ("$ResourceId"+"?api-version=2021-09-01-preview") -Method PUT -Payload $DCRContent
81+
```
82+
3. Upon successful call, you should get the response with status code “200”, indicating that your DCR is now updated.
83+
4. You can now navigate to your DCR and examine its content on the portal via “JSON View” function, or you could repeat the first part of the tutorial to retrieve DCR content into a file.
84+
85+
## Putting everything together
86+
Now, when we know how to read and update the content of a DCR, let’s put everything together into utility script, which can be used to perform both operations together.
87+
88+
```PowerShell
89+
param ([Parameter(Mandatory=$true)] $ResourceId)
90+
91+
# get DCR content and put into a file
92+
$FilePath = "temp.dcr"
93+
$DCR = Invoke-AzRestMethod -Path ("$ResourceId"+"?api-version=2021-09-01-preview") -Method GET
94+
$DCR.Content | ConvertFrom-Json | ConvertTo-Json -Depth 20 | Out-File $FilePath
95+
96+
# Open DCR in code editor
97+
code $FilePath | Wait-Process
98+
99+
#Wait for confirmation to apply changes
100+
$Output = Read-Host "Apply changes to DCR (Y/N)? "
101+
if ("Y" -eq $Output.toupper())
102+
{
103+
#write DCR content back from the file
104+
$DCRContent = Get-Content $FilePath -Raw
105+
Invoke-AzRestMethod -Path ("$ResourceId"+"?api-version=2021-09-01-preview") -Method PUT -Payload $DCRContent
106+
}
107+
108+
#Delete temporary file
109+
Remove-Item $FilePath
110+
```
111+
### How to use this utility
112+
113+
Assuming you saved the script as a file, named `DCREditor.ps1` and need to modify a Data Collection Rule with resource ID of `/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/foo/providers/Microsoft.Insights/dataCollectionRules/bar`, this could be accomplished by running the following command:
114+
115+
```PowerShell
116+
.\DCREditor.ps1 "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/foo/providers/Microsoft.Insights/dataCollectionRules/bar"
117+
```
118+
119+
DCR content will open in embedded code editor. Once editing is complete, entering "Y" on script prompt will apply changes back to the DCR.
120+
121+
## Next steps
122+
123+
- [Read more about data collection rules and options for creating them.](data-collection-rule-overview.md)

‎articles/azure-monitor/toc.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -431,6 +431,9 @@ items:
431431
- name: Overview
432432
displayName: Data collection rules
433433
href: essentials/data-collection-rule-overview.md
434+
- name: Edit
435+
displayName: Data collection rules
436+
href: essentials/data-collection-rule-edit.md
434437
- name: Structure
435438
displayName: Data collection rules
436439
href: essentials/data-collection-rule-structure.md

0 commit comments

Comments
 (0)