Skip to content

Commit 3749bb1

Browse files
committed
Copy edit article
1 parent 59252e5 commit 3749bb1

File tree

1 file changed

+45
-33
lines changed

1 file changed

+45
-33
lines changed
Lines changed: 45 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,40 +1,48 @@
11
---
2-
title: 'Tutorial: Add a workspace transformation to Cosmos DB Azure Monitor Logs by using the Azure portal'
3-
description: Describes how to add a custom transformation to data flowing through Azure Monitor Logs by using the Azure portal.
4-
ms.topic: tutorial
5-
ms.date: 03/17/2023
2+
title: |
3+
Tutorial: Add a transformation for workspace data
4+
titleSuffix: Azure Cosmos DB
5+
description: In this tutorial, add a custom transformation to data flowing through Azure Monitor Logs from Azure Cosmos DB by using the Azure portal.
66
author: StefArroyo
77
ms.author: esarroyo
8+
ms.reviewer: sidandrews
89
ms.service: cosmos-db
10+
ms.topic: tutorial
11+
ms.date: 03/22/2023
912
---
10-
# Cost Optimizations for Log Analytics Workbooks
11-
To help improve costs for enabling Log Analytics, we now support adding Data Collection Rules and transformations on your Log Analytics resources to filter out columns, reduce number of results returned, and create new columns before the data is sent to the destination.
1213

14+
# Tutorial: Add a transformation for Azure Cosmos DB workspace data by using the Azure portal
1315

14-
## Tutorial: Add a transformation in a workspace data collection rule by using the Azure portal
15-
This tutorial walks you through configuration of a sample [transformation in a workspace data collection rule (DCR)](../articles/azure-monitor/essentials/data-collection-transformations.md) by using the Azure portal.
16+
This tutorial walks you through configuration of a sample [transformation in a workspace data collection rule (DCR)](../azure-monitor/essentials/data-collection-transformations.md) by using the Azure portal.
17+
18+
> [!NOTE]
19+
> To help improve costs for enabling Log Analytics, we now support adding Data Collection Rules and transformations on your Log Analytics resources to filter out columns, reduce number of results returned, and create new columns before the data is sent to the destination.
1620
17-
Workspace transformations are stored together in a single [DCR](../articles/azure-monitor/essentials/data-collection-rule-overview.md) for the workspace, which is called the workspace DCR. Each transformation is associated with a particular table. The transformation will be applied to all data sent to this table from any workflow not using a DCR.
21+
Workspace transformations are stored together in a single [DCR](../azure-monitor/essentials/data-collection-rule-overview.md) for the workspace, which is called the workspace DCR. Each transformation is associated with a particular table. The transformation is applied to all data sent to this table from any workflow not using a DCR.
1822

1923
> [!NOTE]
20-
> This tutorial uses the Azure portal to configure a workspace transformation. For the same tutorial using Azure Resource Manager templates and REST API, see [Tutorial: Add transformation in workspace data collection rule to Azure Monitor using resource manager templates](tutorial-workspace-transformations-api.md).
24+
> This tutorial uses the Azure portal to configure a workspace transformation. For the same tutorial using Azure Resource Manager templates and REST API, see [Tutorial: Add transformation in workspace data collection rule to Azure Monitor using resource manager templates](../azure-monitor/logs/tutorial-workspace-transformations-api.md).
2125
2226
In this tutorial, you learn how to:
2327

2428
> [!div class="checklist"]
25-
> * Configure a [workspace transformation](../articles/azure-monitor/essentials/data-collection-transformations.md#workspace-transformation-dcr) for a table in a Log Analytics workspace.
26-
> * Write a log query for a workspace transformation.
29+
>
30+
> - Configure a [workspace transformation](../azure-monitor/essentials/data-collection-transformations.md#workspace-transformation-dcr) for a table in a Log Analytics workspace.
31+
> - Write a log query for a workspace transformation.
32+
>
2733
2834
## Prerequisites
35+
2936
To complete this tutorial, you need:
3037

31-
- A Log Analytics workspace where you have at least [contributor rights](../articles/azure-monitor/logs/manage-access.md#azure-rbac).
32-
- [Permissions to create DCR objects](../articles/azure-monitor/data-collection-rule-overview.md#permissions) in the workspace.
38+
- A Log Analytics workspace where you have at least [contributor rights](../azure-monitor/logs/manage-access.md#azure-rbac).
39+
- [Permissions to create DCR objects](../azure-monitor/essentials/data-collection-rule-overview.md#permissions) in the workspace.
3340
- A table that already has some data.
34-
- The table can't be linked to the [workspace transformation DCR](../articles/azure-monitor/essentials/data-collection-transformations.md#workspace-transformation-dcr).
41+
- The table can't be linked to the [workspace transformation DCR](../azure-monitor/essentials/data-collection-transformations.md#workspace-transformation-dcr).
3542

3643
## Overview of the tutorial
37-
In this tutorial, you'll reduce the storage requirement for the `CDBDataPlaneRequests` table by filtering out certain records. You'll also remove the contents of a column while parsing the column data to store a piece of data in a custom column. The [CDBDataPlaneRequests table](monitor-resource-logs.md) is created when you enable [log analytics](monitor-resource-logs.md) in a workspace.
44+
45+
In this tutorial, you reduce the storage requirement for the `CDBDataPlaneRequests` table by filtering out certain records. You also remove the contents of a column while parsing the column data to store a piece of data in a custom column. The [CDBDataPlaneRequests table](monitor-resource-logs.md) is created when you enable [log analytics](monitor-resource-logs.md) in a workspace.
3846

3947
This tutorial uses the Azure portal, which provides a wizard to walk you through the process of creating an ingestion-time transformation. After you finish the steps, you'll see that the wizard:
4048

@@ -43,26 +51,26 @@ This tutorial uses the Azure portal, which provides a wizard to walk you through
4351
- Creates an ingestion-time transformation and adds it to the DCR.
4452

4553
## Enable query audit logs
46-
You need to enable [log analytics](monitor-resource-logs.md) for your workspace to create the `CDBDataPlaneRequests` table that you'll be working with. This step isn't required for all ingestion time transformations. It's just to generate the sample data that we'll be working with.
54+
55+
You need to enable [log analytics](monitor-resource-logs.md) for your workspace to create the `CDBDataPlaneRequests` table that you're working with. This step isn't required for all ingestion time transformations. It's just to generate the sample data that we're working with.
4756

4857
## Add a transformation to the table
58+
4959
Now that the table's created, you can create the transformation for it.
5060

5161
1. On the **Log Analytics workspaces** menu in the Azure portal, select **Tables**. Locate the `CDBDataPlaneRequests` table and select **Create transformation**.
5262

53-
:::image type="content" source="media/tutorial-workspace-transformations-portal/create-transformation.png" lightbox="media/tutorial-workspace-transformations-portal/create-transformation.png" alt-text="Screenshot that shows creating a new transformation.":::
54-
55-
2. Because this transformation is the first one in the workspace, you must create a [workspace transformation DCR](../articles/azure-monitor/essentials/data-collection-transformations.md#workspace-transformation-dcr). If you create transformations for other tables in the same workspace, they'll be stored in this same DCR. Select **Create a new data collection rule**. The **Subscription** and **Resource group** will already be populated for the workspace. Enter a name for the DCR and select **Done**.
63+
:::image type="content" source="media/tutorial-log-transformation/create-transformation.png" lightbox="media/tutorial-log-transformation/create-transformation.png" alt-text="Screenshot that shows creating a new transformation.":::
5664

57-
:::image type="content" source="media/tutorial-workspace-transformations-portal/new-data-collection-rule.png" lightbox="media/tutorial-workspace-transformations-portal/new-data-collection-rule.png" alt-text="Screenshot that shows creating a new data collection rule.":::
65+
1. Because this transformation is the first one in the workspace, you must create a [workspace transformation DCR](../azure-monitor/essentials/data-collection-transformations.md#workspace-transformation-dcr). If you create transformations for other tables in the same workspace, they're stored in this same DCR. Select **Create a new data collection rule**. The **Subscription** and **Resource group** are already populated for the workspace. Enter a name for the DCR and select **Done**.
5866

59-
3. Select **Next** to view sample data from the table. As you define the transformation, the result will be applied to the sample data. For this reason, you can evaluate the results before you apply it to actual data. Select **Transformation editor** to define the transformation.
67+
1. Select **Next** to view sample data from the table. As you define the transformation, the result is applied to the sample data. For this reason, you can evaluate the results before you apply it to actual data. Select **Transformation editor** to define the transformation.
6068

61-
:::image type="content" source="media/tutorial-workspace-transformations-portal/transformation-query-results.png" lightbox="media/tutorial-workspace-transformations-portal/transformation-query-results.png" alt-text="Screenshot that shows sample data from the log table.":::
69+
:::image type="content" source="media/tutorial-log-transformation/transformation-query-results.png" lightbox="media/tutorial-log-transformation/transformation-query-results.png" alt-text="Screenshot that shows sample data from the log table.":::
6270

63-
4. In the transformation editor, you can see the transformation that will be applied to the data prior to its ingestion into the table. The incoming data is represented by a virtual table named `source`, which has the same set of columns as the destination table itself. The transformation initially contains a simple query that returns the `source` table with no changes.
71+
1. In the transformation editor, you can see the transformation that is applied to the data prior to its ingestion into the table. A virtual table named `source` represents the incoming data, which has the same set of columns as the destination table itself. The transformation initially contains a simple query that returns the `source` table with no changes.
6472

65-
5. Modify the query to the following example:
73+
1. Modify the query to the following example:
6674

6775
``` kusto
6876
source
@@ -78,32 +86,36 @@ Now that the table's created, you can create the transformation for it.
7886
7987
> [!Note]
8088
> Using the Azure portal, the output of the transformation will initiate changes to the table schema if required. Columns will be added to match the transformation output if they don't already exist. Make sure that your output doesn't contain any columns that you don't want added to the table. If the output doesn't include columns that are already in the table, those columns won't be removed, but data won't be added.
81-
>
89+
>
8290
> Any custom columns added to a built-in table must end in `_CF`. Columns added to a custom table don't need to have this suffix. A custom table has a name that ends in `_CL`.
8391
84-
6. Copy the query into the transformation editor and select **Run** to view results from the sample data. You can verify that the new `Workspace_CF` column is in the query.
92+
1. Copy the query into the transformation editor and select **Run** to view results from the sample data. You can verify that the new `Workspace_CF` column is in the query.
8593
86-
:::image type="content" source="media/tutorial-workspace-transformations-portal/select-transformation-editor.png" lightbox="media/tutorial-workspace-transformations-portal/select-transformation-editor.png" alt-text="Screenshot that shows the transformation editor.":::
94+
:::image type="content" source="media/tutorial-log-transformation/select-transformation-editor.png" lightbox="media/tutorial-log-transformation/select-transformation-editor.png" alt-text="Screenshot that shows the transformation editor.":::
8795
88-
6. Select **Apply** to save the transformation and then select **Next** to review the configuration. Select **Create** to update the DCR with the new transformation.
96+
1. Select **Apply** to save the transformation and then select **Next** to review the configuration. Select **Create** to update the DCR with the new transformation.
8997
90-
:::image type="content" source="media/tutorial-workspace-transformations-portal/transformation-configuration-created.png" lightbox="media/tutorial-workspace-transformations-portal/transformation-configuration-created.png" alt-text="Screenshot that shows saving the transformation.":::
98+
:::image type="content" source="media/tutorial-log-transformation/transformation-configuration-created.png" lightbox="media/tutorial-log-transformation/transformation-configuration-created.png" alt-text="Screenshot that shows saving the transformation.":::
9199
92100
## Test the transformation
93-
Allow about 30 minutes for the transformation to take effect and then test it by running a query against the table. Only data sent to the table after the transformation was applied will be affected.
101+
102+
Allow about 30 minutes for the transformation to take effect and then test it by running a query against the table. This transformation affects only data sent to the table after the transformation was applied.
94103
95104
For this tutorial, run some sample queries to send data to the `CDBDataPlaneRequests` table. Include some queries against `CDBDataPlaneRequests` so that you can verify that the transformation filters these records.
96105
97106
## Troubleshooting
107+
98108
This section describes different error conditions you might receive and how to correct them.
99109
100110
### IntelliSense in Log Analytics not recognizing new columns in the table
111+
101112
The cache that drives IntelliSense might take up to 24 hours to update.
102113
103114
### Transformation on a dynamic column isn't working
115+
104116
A known issue currently affects dynamic columns. A temporary workaround is to explicitly parse dynamic column data by using `parse_json()` prior to performing any operations against them.
105117
106118
## Next steps
107119
108-
- [Read more about transformations](../articles/azure-monitor/essentials/data-collection-transformations.md)
109-
- [Learn more about writing transformation queries](../articles/azure-monitor/essentials/data-collection-transformations-structure.md)
120+
> [!div class="nextstepaction"]
121+
> [Data collection transformations](../azure-monitor/essentials/data-collection-transformations.md)

0 commit comments

Comments
 (0)