Merge pull request #302743 from EdB-MSFT/lake-updates-batami-review

updates

Author: AnnaMHuff

articles/sentinel/TOC.yml

@@ -5,7 +5,7 @@
   items:
   - name: What is Microsoft Sentinel?
     href: overview.md
-  - name: What is Microsoft Sentinel data lake (Preview)?
+  - name: Microsoft Sentinel data lake overview
     href: graph/sentinel-lake-overview.md
     displayName: data lake
   - name: What's new
@@ -14,46 +14,44 @@
     href: best-practices.md
   - name: Experience in Defender portal
     href: microsoft-sentinel-defender-portal.md
-- name: Microsoft Sentinel data lake (Preview)
+- name: Data lake exploration
   items:  
-  - name: Data lake exploration
+  - name: KQL for the Microsoft Sentinel data lake
     items:
-    - name: KQL for the Microsoft Sentinel data lake (Preview)
-      items:
-      - name: Overview 
-        href: graph/kql-overview.md
-        displayName: data lake
-      - name: Run KQL queries (Preview)
-        href: graph/kql-queries.md
-        displayName: data lake
-      - name: Sample data lake queries (Preview)
-        href: graph/kql-samples.md
-        displayName: data lake  
-      - name: Create KQL jobs (Preview)
-        href: graph/kql-jobs.md 
-        displayName: data lake 
-      - name: Manage KQL jobs (Preview)
-        href: graph/kql-manage-jobs.md  
-        displayName: data lake      
-      - name: Troubleshoot KQL for the lake (Preview)
-        href: graph/kql-troubleshoot.md
-        displayName: data lake  
-    - name: Notebooks for data lake exploration (Preview)
-      items:
-      - name: Overview
-        href: graph/notebooks-overview.md
-        displayName: data lake
-      - name: Run notebooks (Preview)
-        href: graph/notebooks.md
-        displayName: data lake
-      - name: Microsoft Sentinel provider class reference (Preview)
-        href: graph/sentinel-provider-class-reference.md
-        displayName: data lake
-      - name: Create and manage notebook jobs (Preview)
-        href: graph/notebook-jobs.md
-        displayName: data lake
-      - name: Notebook examples for data lake exploration (Preview)
-        href: graph/notebook-examples.md 
+    - name: Overview 
+      href: graph/kql-overview.md
+      displayName: data lake
+    - name: Run KQL queries
+      href: graph/kql-queries.md
+      displayName: data lake
+    - name: Sample data lake queries
+      href: graph/kql-samples.md
+      displayName: data lake  
+    - name: Create KQL jobs
+      href: graph/kql-jobs.md 
+      displayName: data lake 
+    - name: Manage KQL jobs
+      href: graph/kql-manage-jobs.md
+      displayName: data lake
+    - name: Troubleshoot KQL for the lake
+      href: graph/kql-troubleshoot.md
+      displayName: data lake  
+  - name: Notebooks for data lake exploration
+    items:
+    - name: Overview
+      href: graph/notebooks-overview.md
+      displayName: data lake
+    - name: Run notebooks
+      href: graph/notebooks.md
+      displayName: data lake
+    - name: Microsoft Sentinel provider class reference
+      href: graph/sentinel-provider-class-reference.md
+      displayName: data lake
+    - name: Create and manage notebook jobs
+      href: graph/notebook-jobs.md
+      displayName: data lake
+    - name: Notebook examples for data lake exploration
+      href: graph/notebook-examples.md
 - name: Plan
   items:
   - name: Deployment planning guide  
@@ -97,7 +95,7 @@
     href: quickstart-onboard.md
   - name: Connect Microsoft Sentinel to the Defender portal
     href: /unified-secops-platform/microsoft-sentinel-onboard?toc=/azure/sentinel/TOC.json&bc=/azure/sentinel/breadcrumb/toc.json
-  - name: Onboard to Microsoft Sentinel data lake (Preview)
+  - name: Onboard to Microsoft Sentinel data lake
     href: graph/sentinel-lake-onboarding.md
     displayName: data lake
   - name: Set up connectors for the Microsoft Sentinel data lake

articles/sentinel/basic-logs-use-cases.md

@@ -1,10 +1,10 @@
 ---
-title: When to use data lake in Microsoft Sentinel
-description: Learn what log sources might be appropriate for the Microsoft Sentinel data lake and what  attributes to look for, to decide about other sources.
+title: When to use the Microsoft Sentinel data lake
+description: Learn what log sources might be appropriate for the Microsoft Sentinel data lake and what attributes to look for, to decide about other sources.
 author: EdB-MSFT
 ms.author: edbaynash
 ms.topic: conceptual
-ms.date: 07/07/2025
+ms.date: 07/15/2025
 appliesto:
     - Microsoft Sentinel in the Microsoft Defender portal
     - Microsoft Sentinel in the Azure portal
@@ -16,11 +16,11 @@ ms.collection: usx-security
 ---
 # Log sources to use for the Microsoft Sentinel data lake
 
-This article highlights log sources to consider configuring as data lake tier only when enabling a connector. Before choosing a tier for which to configure a given table, do the research to see which is most appropriate. For more information about data categories and data tiers, see [Data tiers in Microsoft Sentinel](log-plans.md).
+This article highlights log sources to consider configuring as data lake tier only when enabling a connector. Before choosing a tier for which to configure a given table, check which tier is most appropriate for your use case. For more information about data categories and data tiers, see [Log retention plans in Microsoft Sentinel](log-plans.md).
 
 [!INCLUDE [unified-soc-preview](includes/unified-soc-preview.md)]
->[!NOTE]
->The Microsoft Sentinel data lake is currently in Public Preview.
+
+[!INCLUDE [sentinel-lake-preview](includes/sentinel-lake-preview.md)]
 
 ## Storage access logs for cloud providers
 

articles/sentinel/best-practices.md

@@ -23,7 +23,7 @@ Start with the [deployment guide for Microsoft Sentinel](deploy-overview.md). Th
 
 ## Adopt a single-platform architecture
 
-Microsoft Sentinel is now integrated with a modern data lake that offers affordable, long-term storage enabling teams to simplify data management, optimize costs, and accelerate the adoption of AI. Microsoft Sentinel data lake (Preview) enables a single-platform architecture for security data and empowers analysts with a unified query experience while leveraging Microsoft Sentinel’s rich connector ecosystem. For more information, see [Microsoft Sentinel data lake (Preview)](graph/sentinel-lake-overview.md).
+Microsoft Sentinel is integrated with a modern data lake that offers affordable, long-term storage enabling teams to simplify data management, optimize costs, and accelerate the adoption of AI. The Microsoft Sentinel data lake (preview) enables a single-platform architecture for security data and empowers analysts with a unified query experience while leveraging Microsoft Sentinel’s rich connector ecosystem. For more information, see [Microsoft Sentinel data lake (preview)](graph/sentinel-lake-overview.md).
 
 ## Microsoft security service integrations
 

articles/sentinel/billing-reduce-costs.md

@@ -53,8 +53,7 @@ Microsoft Sentinel analyzes all the data ingested into Microsoft Sentinel-enable
 
 While the analytics tier is most appropriate for continuous, real-time threat detection, the Microsoft Sentinel data lake is well-suited for query and analytics of secondary security data that is not needed for real time threat detection. Microsoft Sentinel data lake offers ingestion and storage at a significantly reduced cost. For more information, see [Microsoft Sentinel Pricing](https://azure.microsoft.com/pricing/details/microsoft-sentinel/).
 
->[!NOTE]
->The Microsoft Sentinel data lake is currently in Public Preview.
+[!INCLUDE [sentinel-lake-preview](includes/sentinel-lake-preview.md)]
 
 ## Optimize Log Analytics costs with dedicated clusters
 

articles/sentinel/billing.md

@@ -33,7 +33,7 @@ This article is part of the [Deployment guide for Microsoft Sentinel](deploy-ove
 
 Enable Microsoft Sentinel on an Azure Monitor Log Analytics workspace and the first 10 GB/day ingested using the Analytics logs plan is free for 31 days. The cost for both Log Analytics data ingestion and Microsoft Sentinel analysis charges up to the 10 GB/day limit, are waived during the 31-day trial period. This free trial is subject to a 20 workspace limit per Azure tenant.
 
-See [Microsoft Sentinel pricing](https://azure.microsoft.com/pricing/details/azure-sentinel) page for information on how usage beyond these limits is charged. Charges related to extra capabilities for [automation](automation.md) and [bring your own machine learning](bring-your-own-ml.md) are still applicable during the free trial, as well as any Microsoft Sentinel data lake related charges.
+See the [Microsoft Sentinel pricing](https://azure.microsoft.com/pricing/details/azure-sentinel) page for information on how usage beyond these limits is charged. Charges related to extra capabilities for [automation](automation.md) and [bring your own machine learning](bring-your-own-ml.md) are still applicable during the free trial, as well as any Microsoft Sentinel data lake related charges.
 
 During your free trial, find resources for cost management, training, and more on the [**News & guides > Free trial**](https://portal.azure.com/#view/Microsoft_Azure_Security_Insights/MainMenuBlade/~/NewsAndGuides) tab in Microsoft Sentinel on the Azure portal. This tab also displays details about the dates of your free trial, and how many days left until the trial expires.
 
@@ -78,7 +78,7 @@ The data lake tier incurs charges based on usage of various data like capabiliti
 Once onboarded, usage from Microsoft Sentinel workspaces begins to be billed through the above described meters rather than existing long-term retention (formerly known as Archive), search or auxiliary logs ingestion meters.
 
 >[!IMPORTANT]
->While in Public Preview, once onboarded to the Microsoft Sentinel data lake, billing through new meters will be billed at the respective meters' list rate. Pricing from previous meters doesn't carry over. For more details on pricing, see [Microsoft Sentinel pricing](https://azure.microsoft.com/pricing/details/microsoft-sentinel/).
+>While in preview, once onboarded to the Microsoft Sentinel data lake, billing through new meters is billed at the respective meters' list rate. Pricing from previous meters doesn't carry over. For more details on pricing, see [Microsoft Sentinel pricing](https://azure.microsoft.com/pricing/details/microsoft-sentinel/).
 >Existing customers that are currently billed for Auxiliary logs ingestion, long-term retention and search, will see charges transition to the new data lake ingestion, data lake storage and data lake query meters respectively.
 
 For customers that haven't onboarded to the Microsoft Sentinel data lake and are currently using Auxiliary or Basic logs, see [Manage data retention in a Log Analytics workspace](/azure/azure-monitor/logs/data-retention-archive) and [Azure Monitor pricing](https://azure.microsoft.com/pricing/details/monitor/) for relevant information.

articles/sentinel/configure-data-connector.md

@@ -17,7 +17,7 @@ ms.collection: usx-security
 
 # Connect data sources to Microsoft Sentinel by using data connectors
 
-Connect data sources to Microsoft Sentinel by installing and configuring data connectors. This article generally explains how to install data connectors available in the Microsoft Sentinel **Content hub** to ingest and analyze data for improved threat detection.
+To connect data sources to Microsoft Sentinel, you need to install and configure data connectors. This article generally explains how to install data connectors available in the Microsoft Sentinel **Content hub** to ingest and analyze data for improved threat detection.
 
 - [Microsoft Sentinel data connectors](connect-data-sources.md)
 - [Find your Microsoft Sentinel data connector](data-connectors-reference.md)
@@ -58,7 +58,7 @@ After you or someone in your organization installs the solution that includes th
    - [Connect Microsoft Sentinel to Azure, Windows, Microsoft, and Amazon services](connect-azure-windows-microsoft-services.md)
    - [Data connector prerequisites](data-connectors-reference.md#windows-security-events-via-ama)
 
-### Configure data retention and tiering.
+### Configure data retention and tiering
 
 If you have onboarded to the Microsoft Sentinel data lake (preview), you can configure data retention and tiering for the data connector. The data lake consists of an analytics tier - your current Microsoft Sentinel workspaces, and a data lake tier where you can store data for up to 12 years. For more information on onboarding, see [Onboarding to Microsoft Sentinel data lake](graph/sentinel-lake-onboarding.md). 
 

articles/sentinel/graph/kql-jobs.md

@@ -6,7 +6,7 @@ author: EdB-MSFT
 ms.service: microsoft-sentinel  
 ms.topic: conceptual
 ms.subservice: sentinel-graph
-ms.date: 07/09/2025
+ms.date: 07/15/2025
 ms.author: edbaynash  
 
 ms.collection: ms-security  
@@ -19,7 +19,7 @@ ms.collection: ms-security
 #  Create KQL jobs in the Microsoft Sentinel data lake (preview)
 
 
-A job is a one-time or scheduled task that runs a KQL (Kusto Query Language) query against the data in the lake tier to promote the results to the analytics tier. Once in the analytics tier, use the advanced hunting KQL editor to query the data. Promoting data to the analytics tier has the following benefits:
+A job is a one-time or repeatedly scheduled task that runs a KQL (Kusto Query Language) query against the data in the data lake tier to promote the results to the analytics tier. Once in the analytics tier, use the advanced hunting KQL editor to query the data. Promoting data to the analytics tier has the following benefits:
 
 + Combine current and historical data in the analytics tier to run advanced analytics and machine learning models on your data.
 
@@ -32,9 +32,6 @@ A job is a one-time or scheduled task that runs a KQL (Kusto Query Language) que
 
 When promoting data to the analytics tier, make sure that the destination workspace is visible in the advanced hunting query editor. You can only query connected workspaces in the advanced hunting query editor. You will not be able to see data promoted to workspaces that aren't connected or to the default workspace in advance hunting. For more information on connected workspaces, see [Connect a workspace](/defender-xdr/advanced-hunting-microsoft-defender#connect-a-workspace). You can promote data to a new table or append the results to an existing table in the analytics tier. When creating a new table, the table name is suffixed with *_KQL_CL* to indicate that the table was created by a KQL job.  
 
-
-You can create a job by selecting the **Create job** button a KQL query tab or directly from the **Jobs** management page or by. For more information on the Jobs management page, see [Manage jobs in the Microsoft Sentinel data lake](kql-manage-jobs.md).
-
 ## Prerequisites
 
 The following prerequisites are required to create and manage KQL jobs in the Microsoft Sentinel data lake.
@@ -85,7 +82,7 @@ You can create and manage jobs from the **Jobs** management page under **Data la
     1. To append to an existing table, select **Add to an existing table** and select the table name form the drop-down list. When adding to an existing table, the query results must match the schema of the existing table. 
 
 1. Select **Next**.
-    :::image type="content" source="media/kql-jobs/enter-job-name-details.png" alt-text="A screenshot showing the new job details page." lightbox="media/kql-jobs/enter-job-name-details.png":::
+    :::image type="content" source="media/kql-jobs/enter-job-details.png" alt-text="A screenshot showing the new job details page." lightbox="media/kql-jobs/enter-job-details.png":::
 
 1. Review or write your query in the Review the query panel. Check that the time picker is set to the required time range for the job if the date range isn't specified in the query.
 1. Select the workspace to run the query against from the **Selected workspace** drop-down.
@@ -96,7 +93,7 @@ You can create and manage jobs from the **Jobs** management page under **Data la
 
     :::image type="content" source="media/kql-jobs/review-query.png" alt-text="A screenshot showing the review query panel." lightbox="media/kql-jobs/review-query.png":::
 
-In the **Schedule the query job** panel, select whether you want to run the job once or on a schedule. If you select **One time**, the job runs as soon as the job definition is complete. If you select **Schedule**, you can specify a date and time for the job to run, or run the job on a recurring schedule.
+    In the **Schedule the query job** panel, select whether you want to run the job once or on a schedule. If you select **One time**, the job runs as soon as the job definition is complete. If you select **Schedule**, you can specify a date and time for the job to run, or run the job on a recurring schedule.
 
 1. Select **One time** or **Scheduled job**.
     >[!NOTE]
@@ -150,6 +147,8 @@ For service limits, see [Microsoft Sentinel data lake (preview) service limits](
 > [!NOTE]
 >  Partial results may be promoted if the job's query exceeds the one hour limit.
 
+[!INCLUDE [limitations for KQL jobs](../includes/service-limits-kql-jobs.md)]
+
 For troubleshooting tips and error messages, see [Troubleshooting KQL queries for the Microsoft Sentinel data lake (preview)](kql-troubleshoot.md).
 
 

articles/sentinel/graph/kql-manage-jobs.md

@@ -48,28 +48,12 @@ To create a job, select **Create new job**. For more information on creating job
 
 ### Job details
 
-To see a job's details, select a job. The job detail panel opens, showing the following information:
-
-- **Job name**: The name of the job.
-- **Job description**: A description of the job, providing context and purpose.
-- **Job type**: The type of job, either a KQL query job or a notebook job.
-- **Job status**: The status of the job, either enabled or disabled.
-- **Run status**: The status of the last run of the job.  Status values are:
-    - `Succeeded`
-    - `Failed`
-    - `In progress`
-    - `Queued` - The job is queued and waiting to run when resources are available.
-- **Repeat frequency**: The frequency at which the job runs, such as daily, weekly, or monthly.
-- **Destination table**: The table in the analytics tier where the job results are written to.
-<!-- **Destination workspace**: The workspace in the analytics tier where the job results are written to -->
-- **Job start on (UTC)**: The date and time in UTC when the job is first scheduled to start.
-- **Target tier**: The destination tier of the job's results, such as data lake or analytics tier.
-- **Date range**: The date range set for the query. 
-- **KQL query**: The KQL query that the job runs.
-
-    :::image type="content" source="media/kql-manage-jobs/manage-job-details.png" alt-text="A screenshot showing the job details page." lightbox="media/kql-manage-jobs/manage-job-details.png":::
-
-Select the **Destination table** link to open the table in the KQL query editor in Advanced hunting. The query can be copied by selecting **Copy query**.  
+To see a job's details, select the job from the table.
+
+:::image type="content" source="media/kql-manage-jobs/manage-job-details.png" alt-text="A screenshot showing the job details page." lightbox="media/kql-manage-jobs/manage-job-details.png":::
+
+Select the **Destination table** link to open the table in the KQL query editor in Advanced hunting.  
+The query can be copied by selecting **Copy query**.  
 
 ### Edit a job
 
@@ -102,7 +86,7 @@ To delete a job, select  **Delete** in the job details panel. A confirmation dia
 
 ## Considerations and limitations
 
-For  information on considerations and limitations when managing KQL jobs in the Microsoft Sentinel data lake, see [KQL jobs](kql-jobs.md#considerations-and-limitations).
+For information on considerations and limitations when managing KQL jobs in the Microsoft Sentinel data lake, see [KQL jobs](kql-jobs.md#considerations-and-limitations).
 
 ## Next steps