You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/sentinel/data-transformation.md
+4-35Lines changed: 4 additions & 35 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ Azure Monitor Logs serves as the platform behind the Microsoft Sentinel workspac
16
16
17
17
Log Analytics' custom data ingestion process gives you a high level of control over the data that gets ingested. It uses [**data collection rules (DCRs)**](/azure/azure-monitor/essentials/data-collection-rule-overview) to collect your data and manipulate it even before it's stored in your workspace. This allows you to filter or enrich data being collected in standard tables and to create highly customizable tables for storing data from sources that produce unique log formats.
18
18
19
-
Microsoft Sentinel leverages two tools from the underlying Azure Monitor platform to control this process:
19
+
Microsoft Sentinel uses two tools from the underlying Azure Monitor platform to control this process:
20
20
21
21
-[**Transformations**](/azure/azure-monitor/essentials/data-collection-transformations) are defined in DCRs and apply KQL queries to incoming data before it's stored in your workspace. These transformations can filter out irrelevant data, enrich existing data with analytics or external data, or mask sensitive or personal information.
22
22
@@ -93,43 +93,12 @@ The following table describes DCR support for Microsoft Sentinel data connector
93
93
|**Built-in, API-based data connectors**, such as: <li>[Legacy codeless data connectors](create-codeless-connector-legacy.md)<li>[Azure Functions-based data connectors](connect-azure-functions-template.md)| Not currently supported |
94
94
95
95
96
-
## Data transformation support for custom data connectors
97
96
98
-
If you've created custom data connectors for Microsoft Sentinel, you can use DCRs to configure how the data will be parsed and stored in Log Analytics in your workspace.
97
+
## Limitations and considerations
99
98
100
-
Only the following tables are currently supported for custom log ingestion:
- Transformations in Microsoft Sentinel have the same limitations as Azure Monitor. See [Limitations and considerations](/azure/azure-monitor/essentials/data-collection-transformations-create#limitations-and-considerations) for details.
100
+
- Log Analytic workspaces enabled for Microsoft Sentinel aren't subject to the [filtering ingestion charge](/azure/azure-monitor/essentials/data-collection-transformations#cost-for-transformations), regardless of how much data the transformation filters.
111
101
112
-
For more information, see [Tables that support ingestion-time transformations](/azure/azure-monitor/logs/tables-feature-support).
113
-
114
-
## Limitations
115
-
116
-
Ingestion-time data transformation currently has the following known issues for Microsoft Sentinel data connectors:
117
-
118
-
- Data transformations using *workspace transformation DCRs* are supported only per table, and not per connector.
119
-
120
-
There can only be one workspace transformation DCR for an entire workspace. Within that DCR, each table can use a separate input stream with its own transformation. Splitting data to multiple destinations (Log Analytics workspaces) with a workspace transformation DCR is not possible. AMA-based data connectors use the configuration you define in the associated DCR for input and output streams and transformations, and ignore the workspace transformation DCR.
121
-
122
-
- The following configurations are supported only via API:
123
-
124
-
- Standard DCRs for AMA-based connectors like [Windows Security Events](./data-connectors/windows-security-events-via-ama.md) and [Windows Forwarded Events](./data-connectors/windows-forwarded-events.md).
125
-
126
-
- Standard DCRs for custom log ingestion to a standard table.
127
-
128
-
- It may take up to 60 minutes for the data transformation configurations to apply.
129
-
130
-
- KQL syntax: Not all operators are supported. For more information, see [**KQL limitations** and **Supported KQL features**](/azure/azure-monitor/essentials/data-collection-transformations-structure#kql-limitations) in the Azure Monitor documentation.
131
-
132
-
- You can only send logs from one specific data source to one workspace. To send data from a single data source to multiple workspaces (destinations) with a standard DCR, please create one DCR per workspace.
0 commit comments