You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/sentinel/investigate-large-datasets.md
+7-32Lines changed: 7 additions & 32 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
title: Start an investigation by searching large datasets - Microsoft Sentinel
3
-
description: Learn about search jobs and restoring archived data in Microsoft Sentinel.
3
+
description: Learn about search jobs and restoring data from long-term retention in Microsoft Sentinel.
4
4
author: cwatson-cat
5
5
ms.topic: conceptual
6
6
ms.date: 03/03/2024
@@ -11,7 +11,7 @@ appliesto:
11
11
ms.collection: usx-security
12
12
13
13
14
-
#Customer intent: As a security analyst, I want to search and restore archived log data so that I can conduct thorough investigations on historical events.
14
+
#Customer intent: As a security analyst, I want to search and restore log data from long-term retention so that I can conduct thorough investigations on historical events.
15
15
16
16
---
17
17
@@ -47,26 +47,11 @@ You can also search analytics or basic log data stored in [long-term retention](
47
47
48
48
### Limitations of a search job
49
49
50
-
Before you start a search job, be aware of the following limitations:
50
+
See [Search job limitations](/azure/azure-monitor/logs/search-jobs#limitations) in the Azure Monitor documentation.
51
51
52
-
- Optimized to query one table at a time.
53
-
- Search date range is up to seven years.
54
-
- Supports long running searches up to a 24-hour time-out.
55
-
- Results are limited to one million records in the record set.
56
-
- Concurrent execution per user is limited to five search jobs per workspace.
57
-
- Limited to 100 search results tables per workspace.
58
-
- Limited to 100 search job executions per day per workspace.
52
+
## Restore log data from long-term retention
59
53
60
-
Search jobs aren't currently supported for the following workspaces:
61
-
62
-
- Customer-managed key enabled workspaces
63
-
- Workspaces in the China East 2 region
64
-
65
-
To learn more, see [Search job in Azure Monitor](/azure/azure-monitor/logs/search-jobs) in the Azure Monitor documentation.
66
-
67
-
## Restore historical data from archived logs
68
-
69
-
When you need to do a full investigation on data stored in archived logs, restore a table from the **Search** page in Microsoft Sentinel. Specify a target table and time range for the data you want to restore. Within a few minutes, the log data is restored and available within the Log Analytics workspace. Then you can use the data in high-performance queries that support full KQL.
54
+
When you need to do a full investigation on log data in long-term retention, restore a table from the **Search** page in Microsoft Sentinel. Specify a target table and time range for the data you want to restore. Within a few minutes, the log data is restored and available within the Log Analytics workspace. Then you can use the data in high-performance queries that support full KQL.
70
55
71
56
A restored log table is available in a new table that has a *_RST suffix. The restored data is available as long as the underlying source data is available. But you can delete restored tables at any time without deleting the underlying source data. To save costs, we recommend you delete the restored table when you no longer need it.
72
57
@@ -76,17 +61,7 @@ The following image shows the restore option on a saved search.
76
61
77
62
### Limitations of log restore
78
63
79
-
Before you start to restore an archived log table, be aware of the following limitations:
80
-
81
-
82
-
- Restore data for a minimum of two days.
83
-
- Restore data more than 14 days old.
84
-
- Restore up to 60 TB.
85
-
- Restore is limited to one active restore per table.
86
-
- Restore up to four archived tables per workspace per week.
87
-
- Limited to two concurrent restore jobs per workspace.
88
-
89
-
To learn more, see [Restore logs in Azure Monitor](/azure/azure-monitor/logs/restore).
64
+
See [Restore limitations](/azure/azure-monitor/logs/restore#limitations) in the Azure Monitor documentation.
90
65
91
66
## Bookmark search results or restored data rows
92
67
@@ -95,4 +70,4 @@ Similar to the [threat hunting dashboard](hunting.md#use-the-hunting-dashboard),
95
70
## Next steps
96
71
97
72
-[Search across long time spans in large datasets](search-jobs.md)
98
-
-[Restore archived logs from search](restore.md)
73
+
-[Restore logs from long-term retention](restore.md)
Copy file name to clipboardExpand all lines: articles/sentinel/search-jobs.md
+12-12Lines changed: 12 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,27 +1,27 @@
1
1
---
2
-
title: Search across long time spans in large datasets - Microsoft Sentinel
2
+
title: Search for specific events across large datasets in Microsoft Sentinel
3
3
description: Learn how to use search jobs to search large datasets.
4
-
author: austinmccollum
4
+
author: guywi-ms
5
5
ms.topic: how-to
6
-
ms.date: 03/17/2025
7
-
ms.author: austinmc
6
+
ms.date: 03/06/2025
7
+
ms.author: guywild
8
8
appliesto:
9
9
- Microsoft Sentinel in the Microsoft Defender portal
10
10
- Microsoft Sentinel in the Azure portal
11
11
ms.collection: usx-security
12
12
13
13
14
-
#Customer intent: As a security analyst, I want to search and analyze historical log data across large datasets so that I can investigate and identify specific events.
14
+
#Customer intent: As a security analyst, I want to search through historical log data in a specific table so that I can find and analyze specific events.
15
15
16
16
---
17
17
18
-
# Search across long time spans in large datasets
18
+
# Search for specific events across large datasets in Microsoft Sentinel
19
19
20
-
Use a search job when you start an investigation to find specific events in logs up to seven years ago. You can search events across all your logs, including events in Analytics, Basic, and Archived log plans. Filter and look for events that match your criteria.
20
+
Use a search job when you start an investigation to scan through up to a year of data in a table for specific events. You can a run search job on any table, including tables with the Analytics, Basic, and Auxiliary log plans. The search job sends its results to a new Analytics table in the same workspace as the source data.
21
21
22
-
- For more information on search job concepts and limitations, see [Start an investigation by searching large datasets](investigate-large-datasets.md) and [Search jobs in Azure Monitor](/azure/azure-monitor/logs/search-jobs).
22
+
This article explains how to run a search job in Microsoft Sentinel and how to work with the search job results.
23
23
24
-
-Search jobs across certain data sets might incur extra charges. For more information, see [Microsoft Sentinel pricing page](billing.md).
24
+
Search jobs across certain data sets might incur extra charges. For more information, see [Microsoft Sentinel pricing page](billing.md).
@@ -51,7 +51,7 @@ Go to **Search** in Microsoft Sentinel from the Azure portal or the Microsoft De
51
51
52
52
:::image type="content" source="media/search-jobs/search-job-advanced-kql-ellipsis.png" alt-text="Screenshot of KQL editor with revised search with ellipsis highlighted for Search job mode." lightbox="media/search-jobs/search-job-advanced-kql-ellipsis.png":::
53
53
54
-
1. Specify the search job date range using the **Time range** selector. Don't include a time range in your KQL query as it is ignored.
54
+
1. Specify the search job date range using the **Time range** selector. If your query also specifies a time range, Microsoft Sentinel runs the search job on the union of the time ranges.
55
55
56
56
1. Resolve any KQL issues indicated by a squiggly red line in the editor.
57
57
@@ -91,5 +91,5 @@ View the status and results of your search job by going to the **Saved Searches*
91
91
To learn more, see the following articles.
92
92
93
93
-[Hunt with bookmarks](bookmarks.md)
94
-
-[Restore archived logs](restore.md)
95
-
-[Configure data retention and archive policies in Azure Monitor Logs (Preview)](/azure/azure-monitor/logs/data-retention-configure)
94
+
-[Restore logs from long-term retention](restore.md)
95
+
-[Manage data retention in a Log Analytics workspace](/azure/azure-monitor/logs/data-retention-configure)
0 commit comments