You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/azure-monitor/essentials/data-collection-rule-structure.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ description: Details on the structure of different kinds of data collection rule
4
4
ms.topic: conceptual
5
5
author: bwren
6
6
ms.author: bwren
7
-
ms.date: 01/06/2022
7
+
ms.date: 02/22/2022
8
8
9
9
---
10
10
@@ -17,17 +17,17 @@ ms.date: 01/06/2022
17
17
## Custom logs
18
18
A DCR for [custom logs](../logs/custom-logs-overview.md) contains the following sections:
19
19
### streamDeclarations
20
-
This section contains the declaration of all the different types of data that will be sent via the HTTP endpoint directly into Log Analytics. Each stream is an object whose key represents the stream name (Must begin with *Custom-*) and whose value is the full list of top-level properties that the JSON data that will be sent will contain. Note that the shape of the data you send to the endpoint doesn't need to match that of the destination table. Rather, the output of the transform that is applied on top of the input data needs to match the destination shape. The possible data types that can be assigned to the properties are `string`, `int`, `long`, `real`, `boolean`, `dynamic`, and `dateTime`. Dynamic data should be represented by a `string`.
20
+
This section contains the declaration of all the different types of data that will be sent via the HTTP endpoint directly into Log Analytics. Each stream is an object whose key represents the stream name (Must begin with *Custom-*) and whose value is the full list of top-level properties that the JSON data that will be sent will contain. Note that the shape of the data you send to the endpoint doesn't need to match that of the destination table. Rather, the output of the transform that is applied on top of the input data needs to match the destination shape. The possible data types that can be assigned to the properties are `string`, `int`, `long`, `real`, `boolean`, `dynamic`, and `datetime`.
21
21
22
22
### destinations
23
-
This section contains a declaration of all the destinations where the data will be sent. Only Log Analytics is currently supported as a destination. Each Log Analytics destination will require the full Workspace Resource ID, as well as a friendly name that will be used elsewhere in the DCR to refer to this workspace. Only one destination is allowed per stream.
23
+
This section contains a declaration of all the destinations where the data will be sent. Only Log Analytics is currently supported as a destination. Each Log Analytics destination will require the full Workspace Resource ID, as well as a friendly name that will be used elsewhere in the DCR to refer to this workspace.
24
24
25
25
### dataFlows
26
26
This section ties the other sections together. Defines the following for each stream declared in the `streamDeclarations` section:
27
27
28
28
-`destination` from the `destinations` section where the data will be sent.
29
29
-`transformKql` which is the [transformation](data-collection-rule-transformations.md) applied to the data that was sent in the input shape described in the `streamDeclarations` section to the shape of the target table.
30
-
-`outputStream` section, which describes which table in the workspace specified under the `destination` property the data will be ingested into. The value of the outputStream will have the `Microsoft-[tableName]` shape when data is being ingested into a standard Log Analytics table, or `Custom-[tableName]` when ingesting data into a custom-created table.
30
+
-`outputStream` section, which describes which table in the workspace specified under the `destination` property the data will be ingested into. The value of the outputStream will have the `Microsoft-[tableName]` shape when data is being ingested into a standard Log Analytics table, or `Custom-[tableName]` when ingesting data into a custom-created table. Only one destination is allowed per stream.
31
31
32
32
## Azure Monitor agent
33
33
A DCR for [Azure Monitor agent](../agents/data-collection-rule-azure-monitor-agent.md) contains the following sections:
Copy file name to clipboardExpand all lines: articles/azure-monitor/logs/custom-logs-overview.md
+8-2Lines changed: 8 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,18 +9,24 @@ ms.date: 01/06/2022
9
9
# Custom logs API in Azure Monitor Logs (Preview)
10
10
With the DCR based custom logs API in Azure Monitor, you can send data to a Log Analytics workspace from any REST API client. This allows you to send data from virtually any source to [supported built-in tables](tables-feature-support.md) or to custom tables that you create. You can even extend the schema of built-in tables with custom columns.
11
11
12
+
[!INCLUDE [Sign up for preview](../../includes/azure-monitor-custom-logs-signup.md)]
13
+
12
14
> [!NOTE]
13
15
> The custom logs API should not be confused with [custom logs](../agents/data-sources-custom-logs.md) data source with the legacy Log Analytics agent.
14
16
## Basic operation
15
17
Your application sends data to a [data collection endpoint](../essentials/data-collection-endpoint-overview.md) which is a unique connection point for your subscription. The payload of your API call includes the source data formatted in JSON. The call specifies a [data collection rule](../essentials/data-collection-rule-overview.md) that understands the format of the source data, potentially filters and transforms it for the target table, and then directs it to a specific table in a specific workspace. You can modify the target table and workspace by modifying the data collection rule without any change to the REST API call or source data.
16
18
17
19
18
-
19
20
## Authentication
20
21
Authentication for the custom logs API is performed at the data collection endpoint which uses standard Azure Resource Manager authentication. A common strategy is to use an Application ID and Application Key as described in [Tutorial: Add ingestion-time transformation to Azure Monitor Logs (preview)](tutorial-custom-logs.md).
21
22
22
23
## Tables
23
-
Custom logs can send data to any custom table that you create and to [certain built-in tables](tables-feature-support.md) in your Log Analytics workspace. The target table must exist before you can send data to it.
24
+
Custom logs can send data to any custom table that you create and to certain built-in tables in your Log Analytics workspace. The target table must exist before you can send data to it. The following built-in tables are currently supported:
The source data sent by your application is formatted in JSON and must match the structure expected by the data collection rule. It doesn't necessarily need to match the structure of the target table since the DCR can include a transformation to convert the data to match the table's structure.
Copy file name to clipboardExpand all lines: articles/azure-monitor/logs/ingestion-time-transformations.md
+2-1Lines changed: 2 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,8 +7,9 @@ ms.date: 01/19/2022
7
7
---
8
8
9
9
# Ingestion-time transformations in Azure Monitor Logs (preview)
10
-
[Ingestion-time transformations](ingestion-time-transformations.md) allow you to manipulate incoming data before it's stored in a Log Analytics workspace. You can add data filtering, parsing and extraction, and control the structure of the data that gets ingested.
10
+
[Ingestion-time transformations](ingestion-time-transformations.md) allow you to manipulate incoming data before it's stored in a Log Analytics workspace. You can add data filtering, parsing and extraction, and control the structure of the data that gets ingested.in
11
11
12
+
[!INCLUDE [Sign up for preview](../../includes/azure-monitor-custom-logs-signup.md)]
12
13
13
14
## Basic operation
14
15
The transformation is a [KQL query](../essentials/data-collection-rule-transformations.md) that runs against the incoming data and modifies it before it's stored in the workspace. Transformations are defined separately for each table in the workspace. This article provides an overview of this feature and guidance for further details and samples. Configuration for ingestion-time transformation is stored in a workspace transformation DCR. You can either [create this DCR directly](tutorial-ingestion-time-transformations-api.md) or configure transformation [through the Azure portal](tutorial-ingestion-time-transformations.md).
> This feature is currently in preview. Submit a request for it to be enabled in your subscriptions. Please sign up at https://aka.ms/CustomLogsOnboard.
0 commit comments