Skip to content

Commit bf5e053

Browse files
committed
image
1 parent 1a546e1 commit bf5e053

File tree

2 files changed

+4
-35
lines changed

2 files changed

+4
-35
lines changed

articles/sentinel/data-transformation.md

Lines changed: 4 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Azure Monitor Logs serves as the platform behind the Microsoft Sentinel workspac
1616

1717
Log Analytics' custom data ingestion process gives you a high level of control over the data that gets ingested. It uses [**data collection rules (DCRs)**](/azure/azure-monitor/essentials/data-collection-rule-overview) to collect your data and manipulate it even before it's stored in your workspace. This allows you to filter or enrich data being collected in standard tables and to create highly customizable tables for storing data from sources that produce unique log formats.
1818

19-
Microsoft Sentinel leverages two tools from the underlying Azure Monitor platform to control this process:
19+
Microsoft Sentinel uses two tools from the underlying Azure Monitor platform to control this process:
2020

2121
- [**Transformations**](/azure/azure-monitor/essentials/data-collection-transformations) are defined in DCRs and apply KQL queries to incoming data before it's stored in your workspace. These transformations can filter out irrelevant data, enrich existing data with analytics or external data, or mask sensitive or personal information.
2222

@@ -93,43 +93,12 @@ The following table describes DCR support for Microsoft Sentinel data connector
9393
| **Built-in, API-based data connectors**, such as: <li>[Legacy codeless data connectors](create-codeless-connector-legacy.md)<li>[Azure Functions-based data connectors](connect-azure-functions-template.md) | Not currently supported |
9494

9595

96-
## Data transformation support for custom data connectors
9796

98-
If you've created custom data connectors for Microsoft Sentinel, you can use DCRs to configure how the data will be parsed and stored in Log Analytics in your workspace.
97+
## Limitations and considerations
9998

100-
Only the following tables are currently supported for custom log ingestion:
101-
- [**WindowsEvent**](/azure/azure-monitor/reference/tables/windowsevent)
102-
- [**SecurityEvent**](/azure/azure-monitor/reference/tables/securityevent)
103-
- [**CommonSecurityLog**](/azure/azure-monitor/reference/tables/commonsecuritylog)
104-
- [**Syslog**](/azure/azure-monitor/reference/tables/syslog)
105-
- [**ASimAuditEventLogs**](/azure/azure-monitor/reference/tables/asimauditeventlogs)
106-
- [**ASimAuthenticationEventLogs**](/azure/azure-monitor/reference/tables/asimauthenticationeventlogs)
107-
- [**ASimDnsActivityLogs**](/azure/azure-monitor/reference/tables/asimdnsactivitylogs)
108-
- [**ASimFileEventLogs**](/azure/azure-monitor/reference/tables/asimfileeventlogs)
109-
- [**ASimNetworkSessionLogs**](/azure/azure-monitor/reference/tables/asimnetworksessionlogs)
110-
- [**ASimWebSessionLogs**](/azure/azure-monitor/reference/tables/asimwebsessionlogs)
99+
- Transformations in Microsoft Sentinel have the same limitations as Azure Monitor. See [Limitations and considerations](/azure/azure-monitor/essentials/data-collection-transformations-create#limitations-and-considerations) for details.
100+
- Log Analytic workspaces enabled for Microsoft Sentinel aren't subject to the [filtering ingestion charge](/azure/azure-monitor/essentials/data-collection-transformations#cost-for-transformations), regardless of how much data the transformation filters.
111101

112-
For more information, see [Tables that support ingestion-time transformations](/azure/azure-monitor/logs/tables-feature-support).
113-
114-
## Limitations
115-
116-
Ingestion-time data transformation currently has the following known issues for Microsoft Sentinel data connectors:
117-
118-
- Data transformations using *workspace transformation DCRs* are supported only per table, and not per connector.
119-
120-
There can only be one workspace transformation DCR for an entire workspace. Within that DCR, each table can use a separate input stream with its own transformation. Splitting data to multiple destinations (Log Analytics workspaces) with a workspace transformation DCR is not possible. AMA-based data connectors use the configuration you define in the associated DCR for input and output streams and transformations, and ignore the workspace transformation DCR.
121-
122-
- The following configurations are supported only via API:
123-
124-
- Standard DCRs for AMA-based connectors like [Windows Security Events](./data-connectors/windows-security-events-via-ama.md) and [Windows Forwarded Events](./data-connectors/windows-forwarded-events.md).
125-
126-
- Standard DCRs for custom log ingestion to a standard table.
127-
128-
- It may take up to 60 minutes for the data transformation configurations to apply.
129-
130-
- KQL syntax: Not all operators are supported. For more information, see [**KQL limitations** and **Supported KQL features**](/azure/azure-monitor/essentials/data-collection-transformations-structure#kql-limitations) in the Azure Monitor documentation.
131-
132-
- You can only send logs from one specific data source to one workspace. To send data from a single data source to multiple workspaces (destinations) with a standard DCR, please create one DCR per workspace.
133102

134103
## Related content
135104

-52.3 KB
Loading

0 commit comments

Comments
 (0)