Merge pull request #304044 from EdB-MSFT/managing-xdr-data

Managing xdr data

Author: v-ccolin

articles/sentinel/manage-data-overview.md

@@ -42,18 +42,19 @@ You can retain data in Microsoft Sentinel in one of two tiers:
   - **Analytics retention**: In this "hot" state, data is fully available for real-time analytics - including high-performance queries and analytics rules - and threat hunting. By default, Microsoft Sentinel and Microsoft Defender XDR retain data in this tier for 30 days. You can extend the retention period of all tables to up to two years at a prorated monthly long-term retention charge. You can extend the retention period of Microsoft Sentinel solution tables to 90 days for free. 
   - **Total retention**: By default, all data in the analytics tier is mirrored to the data lake for the same retention period. You can extend the retention of your data in the lake beyond the analytics retention, for up to 12 years of total retention at a low cost. 
 
-* **Data lake tier**: In this low-cost "cold" tier, Microsoft Sentinel retains your data in the lake only.  Data in the data lake tier isn't available for real-time analytics features and threat hunting. However, you can access data in the lake whenever you need it through [KQL jobs](datalake/kql-jobs.md), analyze trends over time by running scheduled KQL or Spark jobs, and aggregate insights from incoming data at a regular cadence by using summary rules. 
+* **Data lake tier**: In this low-cost "cold" tier, Microsoft Sentinel retains your data in the lake only. Data in the data lake tier isn't available for real-time analytics features and threat hunting. However, you can access data in the lake whenever you need it through [KQL jobs](datalake/kql-jobs.md), analyze trends over time by running scheduled KQL or Spark jobs, and aggregate insights from incoming data at a regular cadence by using summary rules. 
 
-For more information about the differences between these two retention types, see [Compare the analytics and data lake tiers](#compare-the-analytics-and-data-lake-tiers).
+* **XDR default tier**: By default, Microsoft Defender XDR retains threat hunting data in the XDR default tier, which includes 30 days of analytics retention, included in the XDR license. This data is not ingested into the analytics or data lake tiers. You can extend the retention period of [supported Defender XDR tables](#preview-limitations) beyond 30 days and ingest the data into the analytics tier. For more information see [Manage XDR data in Microsoft Sentinel](#manage-xdr-data-in-microsoft-sentinel)
 
-By default, Microsoft Defender XDR retains threat hunting data in the **XDR default tier**, which includes 30 days of analytics retention, included in the XDR license. You can extend the retention period of [supported Defender XDR tables](#preview-limitations) beyond 30 days. When you extend the retention period of supported Microsoft Defender XDR tables Microsoft automatically creates the table in your Microsoft Sentinel workspace in the analytics tier.
+For more information about the differences between these two retention types, see [Compare the analytics and data lake tiers](#compare-the-analytics-and-data-lake-tiers).
 
 This diagram shows the retention components of the analytics, data lake, and XDR default tiers, and which table types apply to each tier:
 
 :::image type="content" source="media/manage-data-overview/tiers-retention-defender-portal.png" lightbox="media/manage-data-overview/tiers-retention-defender-portal.png" alt-text="Diagram that depicts the analytics and data lake tiers in the Microsoft Defender portal.":::
 
 For more information about the Microsoft Sentinel data lake, see [What is Microsoft Sentinel data lake](datalake/sentinel-lake-overview.md).
 
+
 ## Compare the analytics and data lake tiers
 
 This table compares the two analytics and data lake tiers and their key characteristics:
@@ -91,6 +92,40 @@ When you change the analytics retention settings of a table with existing data,
 - You change the analytics retention to 90 days without changing the Total retention period of 180 days. 
 - Microsoft Sentinel removes the last 90 days of data from in analytics retention automatically, but continues to store data that's 90-180 days in the data lake.
 
+## Manage XDR data in Microsoft Sentinel
+
+By default, Microsoft Defender XDR retains threat hunting data in the **XDR default tier**, for 30 days. This data isn't ingested into the analytics or data lake tiers by default. If you extend the retention period of the supported XDR tables beyond 30 days, the tables are created in your Microsoft Sentinel workspace in the analytics tier and mirrored to the data lake tier. You can't ingest XDR tables directly to the data lake tier without ingesting into the analytics tier first.
+
+If the Microsoft Sentinel XDR connector was already enabled in the Azure portal, the tables you selected when setting up the connector are automatically ingested into the analytics tier and mirrored to the data lake tier. The default retention is 30 days, and it can be extended up to 12 years. For a list of tables see [Microsoft Defender XDR integration with Microsoft Sentinel](connect-microsoft-365-defender.md?tabs=MDE#connect-events). Supported XDR tables that weren't selected during the connector deployment can be ingested into the analytics tier and mirrored to the data lake tier by setting the retention to more than 30 days.
+    
+If the Microsoft Sentinel XDR connector isn't enabled, XDR tables aren't automatically ingested but can still be ingested by setting analytics retention for more than 30 days in the Defender portal. The data is automatically mirrored to the data lake tier for the same period.
+
+Stop ingesting data into the analytics tier by resetting the analytics tier retention and total retention to the default 30 days. This disables the connector in the Azure portal.
+
+For more information about managing your tables and data, see [Manage your existing tables and data](manage-table-tiers-retention.md).
+
+### XDR data retention and costs
+
+The following tables summarize the free retention periods and cost implications for the different tiers in Microsoft Sentinel:
+
+|Tier|	Retention| Notes|
+|---|---|---|
+|Advanced Hunting (Default)|	30 days	| Default,  included in XDR license|
+|Analytics tier | 90 days | Free storage for Sentinel-enabled workspaces. Ingestion charges apply.|
+|Data lake	| Configurable. By default, the same as the analytics tier. | Free storage when total retention is the same as analytics tier retention. Retaining data in the data lake beyond the analytics tier retention period incurs additional storage costs.|
+
+For more information on billing and costs, see [Understand the full billing model for Microsoft Sentinel](billing.md#understand-the-full-billing-model-for-microsoft-sentinel)
+
+In the following examples, XDR data is available through advanced hunting for at least 30 days, regardless of the retention settings in the analytics or data lake tiers.
+
+ Analytics tier retention | Total retention |  Analytics tier ingestion costs| Analytics tier storage costs | Data lake tier costs |
+|---|---|---|---|---|
+| 30 days default | 30 days default| No additional costs | N/A  | N/A |
+| 90 days | 90 days | Costs apply for analytics tier ingestion. | No additional costs. 90 days included free. | No additional costs. Total retention matches analytics tier retention.|
+| 90 days | 180 days | Costs apply for analytics tier ingestion. | No additional costs; 90 days included free. | Costs apply for 90 days of additional data lake retention (180 - 90 days). |
+| 30 days with connector enabled | 180 days | Costs apply for analytics tier ingestion. | No additional costs; 90 days included free. | Costs apply for 150 days of additional data lake retention (180 - 30 days). |
+| 180 days | 1 year | Costs apply for analytics tier ingestion. | Costs apply for 90 days of additional analytics tier retention. | Costs apply for 185 days of additional data lake retention (365 - 180 days). |
+
 
 ## Preview limitations