Skip to content

Commit d251120

Browse files
Merge pull request #304064 from EdB-MSFT/rework-overview
revised
2 parents 693aeb7 + 68030d1 commit d251120

File tree

2 files changed

+32
-27
lines changed

2 files changed

+32
-27
lines changed
685 KB
Loading

articles/sentinel/datalake/sentinel-lake-overview.md

Lines changed: 32 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ ms.service: microsoft-sentinel
77
ms.subservice: sentinel-graph
88
ms.topic: conceptual
99
ms.custom: references_regions
10-
ms.date: 07/16/2025
10+
ms.date: 08/11/2025
1111
ms.author: edbaynash
1212

1313
ms.collection: ms-security
@@ -16,39 +16,42 @@ ms.collection: ms-security
1616

1717
# What is Microsoft Sentinel data lake (preview)?
1818

19-
Microsoft Sentinel data lake is a purpose-built, cloud-native security data lake that transforms how organizations manage and analyze security data. Architected as a true data lake, it is designed to ingest, store, and analyze large volumes of diverse security data at scale. By centralizing all your security data into a single, open, and extensible platform, it delivers deep visibility, long-term retention, and advanced analytics.
19+
Microsoft Sentinel data lake is a purpose-built, cloud-native security data lake that transforms how organizations manage and analyze security data. Designed as a true data lake, it ingests, stores, and analyzes large volumes of diverse security data at scale. By centralizing security data into a single, open-format, extensible platform, it provides deep visibility, long-term retention, and advanced analytics.
2020

21-
The data lake makes it cost-effective to bring all your security data into Microsoft Sentinel, eliminating the need to choose between coverage and cost. Retain more data for longer, detect threats with greater context and historical depth, and respond faster, without compromising on security.
21+
The data lake lets you bring all your security data into Microsoft Sentinel cost-effectively, removing the need to choose between coverage and cost. You can retain more data for longer, detect threats with greater context and historical depth, and respond faster without compromising security.
2222

23-
The Microsoft Sentinel data lake is fully managed, without the need to deploy or maintain your data infrastructure. It provides a unified data platform for end-to-end threat analysis and response. It enables you to store one copy of security data across assets, activity logs, and threat intelligence in the lake and leverage multiple analytics tools like KQL and notebooks for deep security analytics.
23+
The Microsoft Sentinel data lake is fully managed, so you don't need to deploy or maintain data infrastructure. It provides a unified data platform for end-to-end threat analysis and response. It stores a single copy of security data across assets, activity logs, and threat intelligence in the lake and leverages multiple analytics tools like KQL and Jupyter notebooks for deep security analytics.
2424

25-
Traditional SIEM solutions struggle with the cost and complexity of storing and querying long-term data. Microsoft Sentinel data lake addresses these challenges in the following ways:
25+
Traditional SIEM solutions struggle with the cost and complexity of storing and querying long-term security data. Microsoft Sentinel data lake solves these challenges in the following ways:
2626

27-
+ Unifying security data across Microsoft Defender XDR, third-party sources and across assets, activity logs, and threat intelligence
28-
+ Optimizing costs through tiered storage and on-demand data promotion.
29-
+ Enabling deep security insights with up to 12 years of security data and telemetry that can be queried and deeply analyzed.
27+
+ Unifying security data across Microsoft Defender XDR, third-party sources and assets, activity logs, and threat intelligence
28+
+ Optimizing costs with tiered storage, on-demand data promotion, and a single copy of the data
29+
+ Enabling deep security insights with up to 12 years of security data and telemetry you can query and analyze
3030
+ Powering AI and automation for faster detection and response.
3131

32-
With a single copy of data, Microsoft Sentinel data lake empowers you to run queries in KQL and conduct deeper analysis for forensics, incidence response, and anomaly detection in Jupyter notebooks using sophisticated Python libraries and machine learning tools.
32+
With a single copy of data, use KQL to run queries and Jupyter notebooks with sophisticated Python libraries and machine learning tools to conduct deeper analysis for forensics, incidence response, and anomaly detection.
3333

3434
## Architecture
3535

3636
Microsoft Sentinel data lake, built on Azure's scalable infrastructure, facilitates centralized ingestion, analysis, and action across diverse data sources. The Microsoft Sentinel data lake technical architecture includes the following key benefits:
3737

38-
+ Single, open-format data copy for efficient and cost-effective storage.
39-
+ Separation of storage and compute for greater flexibility.
40-
+ Support for multiple analytics engines to unlock insights from your security data.
41-
+ Native integration with Microsoft Sentinel SIEM and its security operations workflows.
38+
+ Open format Parquet data files for interoperability and extensibility
39+
+ Single copy of data for efficient and cost effective storage
40+
+ Separation of storage and compute for greater flexibility
41+
+ Support for multiple analytics engines to unlock insights from your security data
42+
+ Native integration with Microsoft Sentinel SIEM and its security operations workflows
4243

4344
### Storage tiers
4445

4546
Microsoft Sentinel is designed with two distinct storage tiers to optimize cost and performance:
4647

47-
+ Analytics tier: The existing Microsoft Sentinel data tier enabling querying, visualization, and alerting capabilities to help you proactively identify and resolve issues across your infrastructure and applications.
48-
+ Data lake tier: A centralized security data lake offering long-term data storage for querying and python-based advanced analytics. The data lake tier is designed for cost-effective storage of large volumes of security data, enabling you to retain data for up to 12 years. For more information on data tiers and retention, see [Manage data tiers and retention in Microsoft Defender portal (preview)](https://aka.ms/manage-data-defender-portal-overview).
48+
+ Analytics tier: The existing Microsoft Sentinel data tier supporting advanced hunting, alerting, and incident management to help you proactively identify and resolve issues across your infrastructure and applications. This tier is designed for high-performance analytics and real-time data processing.
49+
+ Data lake tier: Provides centralized long-term storage for querying and Python-based advanced analytics. It's designed for cost effective retention of large volumes of security data for up to 12 years. Data in the analytics tier is mirrored to the lake tier, preserving a single copy of the data.
50+
51+
For more information on data tiers and retention, see [Manage data tiers and retention in Microsoft Defender portal (preview)](https://aka.ms/manage-data-defender-portal-overview).
4952

5053

51-
### Supported Data Sources
54+
### Supported data sources
5255

5356
Microsoft Sentinel data lake works with all existing Sentinel data connectors, including:
5457
+ All Microsoft Defender and Microsoft Sentinel data sources
@@ -64,39 +67,41 @@ Microsoft Sentinel data lake works with all existing Sentinel data connectors, i
6467

6568
### Flexible querying with Kusto Query Language
6669

67-
Data lake exploration Kusto Query Language (KQL) queries enable you to write and run KQL queries against your data lake resources. You can use the query editor to explore your data, analyze your data lake, and create jobs to promote data from the data lake tier to the analytics tier.
70+
Data lake exploration Kusto Query Language (KQL) queries let you write and run queries against data lake resources. Use the query editor to explore data, analyze the lake, and create jobs that promote data from the data lake tier to the analytics tier.
6871
KQL queries offer the following key features:
6972

7073
+ KQL query editor: Provides editing and running KQL queries with IntelliSense and autocomplete.
7174
+ Full support for KQL: Use the full range of KQL capabilities, including machine learning functions and advanced analytics.
72-
+ Job Creation: Create one-time or scheduled jobs to promote data from the lake to the analytics tier.
75+
+ Job creation: Create one-time or scheduled jobs to promote data from the lake to the analytics tier.
7376

74-
For more information, see [KQL and the Microsoft Sentinel data lake (preview)](kql-overview.md)
77+
For more information, see [KQL and the Microsoft Sentinel data lake (preview)](kql-overview.md).
7578

76-
:::image type="content" source="media/sentinel-lake-overview/data-lake-exploration.png" lightbox="media/sentinel-lake-overview/data-lake-exploration.png" alt-text="A screenshot showing the KQL query editor in the Microsoft Sentinel data lake.":::
79+
:::image type="content" source="media/sentinel-lake-overview/data-lake-exploration.png" lightbox="media/sentinel-lake-overview/data-lake-exploration.png" alt-text="Screenshot of the KQL query editor in the Microsoft Sentinel data lake.":::
7780

7881
### Powerful analytics using Jupyter notebooks
7982

80-
Jupyter notebooks in the Microsoft Sentinel data lake provide a powerful environment for data analysis and machine learning. Use Python libraries to build and run machine learning models, conduct advanced analytics, and visualize your data. The notebooks support rich visualizations, enabling you to gain insights from your security data. Schedule notebooks to regularly summarize data, run machine learning models, and promote data from the data lake tier to the analytics tier.
83+
Jupyter notebooks in the Microsoft Sentinel data lake offer a powerful environment for data analysis and machine learning. Use Python libraries to build and run machine learning models, conduct advanced analytics, and visualize your data. The notebooks support rich visualizations, enabling you to gain insights from your security data. Schedule notebooks to summarize data regularly, run machine learning models, and promote data from the data lake tier to the analytics tier.
8184

8285
For more information, see [Jupyter notebooks in the Microsoft Sentinel data lake (preview)](notebooks-overview.md).
8386

84-
:::image type="content" source="media/sentinel-lake-overview/notebook.png" lightbox="media/sentinel-lake-overview/notebook.png" alt-text="A screenshot showing a Jupyter notebook.":::
87+
:::image type="content" source="media/sentinel-lake-overview/notebook.png" lightbox="media/sentinel-lake-overview/notebook.png" alt-text="Screenshot of a Jupyter notebook showing data analysis and visualization.":::
8588

8689
### Activity audit
87-
The Microsoft Sentinel data lake provides audit functionality that tracks activities performed in the data lake. The audit log captures events related to data access, job management, and queries, enabling you to monitor and investigate activities in the data lake.
90+
The Microsoft Sentinel data lake provides auditing that tracks activities in the lake. The audit log captures data access, job management, and query events, letting you monitor and investigate activity.
8891

8992
Some of the activities audited are:
90-
+ Accessing data in lake via KQL queries
93+
+ Accessing data in lake with KQL queries
9194
+ Running notebooks on data lake
9295
+ Create, edit, run, and delete jobs
9396

94-
Auditing is automatically turned on for Microsoft Sentinel data lake. Features that are audited are logged in the audit log automatically.
95-
For more information on audited data lake activities, see [Audit log for Microsoft Sentinel data lake](./auditing-lake-activities.md)
97+
Auditing is enabled by default for the Microsoft Sentinel data lake. Audited actions are shown in the audit log.
98+
99+
For more information on audited data lake activities, see [Audit log for Microsoft Sentinel data lake](./auditing-lake-activities.md).
96100

97101
## Supported regions
98102

99-
For a list of supported regions, see [Regions supported for Microsoft Sentinel data lake](../geographical-availability-data-residency.md#regions-supported-for-microsoft-sentinel-data-lake)
103+
See [Regions supported for Microsoft Sentinel data lake](../geographical-availability-data-residency.md#regions-supported-for-microsoft-sentinel-data-lake) for supported regions.
104+
100105

101106

102107
## Get started

0 commit comments

Comments
 (0)