Skip to content

Commit 59d8cae

Browse files
committed
Still more cleanup
1 parent 90b564b commit 59d8cae

12 files changed

+36
-38
lines changed

articles/sentinel/billing-reduce-costs.md

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -68,19 +68,15 @@ Here are some other considerations for moving to a dedicated cluster for cost op
6868

6969
For more information about dedicated clusters, see [Log Analytics dedicated clusters](../azure-monitor/logs/cost-logs.md#dedicated-clusters).
7070

71-
## Reduce long-term data retention costs with Azure Data Explorer or archived logs (preview)
71+
## Reduce data retention costs with long-term retention
7272

73-
Microsoft Sentinel data retention is free for the first 90 days. To adjust the data retention period in Log Analytics, select **Usage and estimated costs** in the left navigation, then select **Data retention**, and then adjust the slider.
73+
Microsoft Sentinel interactive data retention is free for the first 90 days. To adjust the data retention period in Log Analytics, select **Usage and estimated costs** in the left navigation, then select **Data retention**, and then adjust the slider.
7474

7575
Microsoft Sentinel security data might lose some of its value after a few months. Security operations center (SOC) users might not need to access older data as frequently as newer data, but still might need to access the data for sporadic investigations or audit purposes.
7676

77-
To help you reduce Microsoft Sentinel data retention costs, Azure Monitor now offers archived logs. Archived logs store log data for long periods of time, up to seven years, at a reduced cost with limitations on its usage. Archived logs are in public preview. For more information, see [Configure data retention and archive policies in Azure Monitor Logs](../azure-monitor/logs/data-retention-archive.md).
77+
To help you reduce Microsoft Sentinel data retention costs, Azure Monitor now offers long-term retention. Data that ages out of its interactive retention state can still be retained for up to twelve years, at a much-reduced cost, and with limitations on its usage. For more information, see [Configure interactive and long-term data retention policies in Azure Monitor Logs](../azure-monitor/logs/data-retention-archive.md).
7878

79-
Alternatively, you can use Azure Data Explorer for long-term data retention at lower cost. Azure Data Explorer provides the right balance of cost and usability for aged data that no longer needs Microsoft Sentinel security intelligence.
80-
81-
With Azure Data Explorer, you can store data at a lower price, but still explore the data using the same Kusto Query Language (KQL) queries as in Microsoft Sentinel. You can also use the Azure Data Explorer proxy feature to do cross-platform queries. These queries aggregate and correlate data spread across Azure Data Explorer, Application Insights, Microsoft Sentinel, and Log Analytics.
82-
83-
For more information, see [Integrate Azure Data Explorer for long-term log retention](store-logs-in-azure-data-explorer.md).
79+
You can reduce costs even further by enrolling tables that contain secondary security data in the **Auxiliary logs** plan (now in Preview). This plan allows you to store high-volume, low-value logs at a low price, with a lower-cost 30-day interactive retention period at the beginning to allow for summarization and basic querying. To learn more about the Auxiliary logs plan and other plans, see [Log retention plans in Microsoft Sentinel](log-plans.md). While the auxiliary logs plan remains in Preview, you also have the option of enrolling these tables in the **Basic logs** plan. Basic logs offers similar functionality to auxiliary logs, but with less of a cost savings.
8480

8581
## Use data collection rules for your Windows Security Events
8682

articles/sentinel/billing.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ Use the [Microsoft Sentinel pricing calculator](https://azure.microsoft.com/pric
4444

4545
For example, enter the GB of daily data you expect to ingest in Microsoft Sentinel, and the region for your workspace. The calculator provides the aggregate monthly cost across these components:
4646

47-
- Microsoft Sentinel: Analytics logs and basic logs
47+
- Microsoft Sentinel: Analytics logs and auxiliary/basic logs
4848
- Azure Monitor: Retention
4949
- Azure Monitor: Data Restore
5050
- Azure Monitor: Search Queries and Search Jobs

articles/sentinel/configure-data-retention-archive.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,12 +12,12 @@ ms.author: cwatson
1212

1313
In the previous deployment step, you enabled the User and Entity Behavior Analytics (UEBA) feature to streamline your analysis process. In this article, you learn how to set up interactive and long-term data retention, to make sure your organization retains the data that's important in the long term. This article is part of the [Deployment guide for Microsoft Sentinel](deploy-overview.md).
1414

15-
## Configure data retention and archive
15+
## Configure data retention
1616

1717
Retention policies define when to remove data, or mark it for long-term retention, in a Log Analytics workspace. Long-term retention lets you keep older, less used data in your workspace at a reduced cost. To set up data retention plans, consult [Log retention plans in Microsoft Sentinel](log-plans.md), and use one or both of these methods, depending on your use case:
1818

19-
- [Configure data retention and archive for one or more tables](../azure-monitor/logs/data-retention-archive.md) (one table at a time)
20-
- [Configure data retention and archive for multiple tables](https://github.com/Azure/Azure-Sentinel/tree/master/Tools/Archive-Log-Tool) at once
19+
- [Configure interactive and long-term data retention for one or more tables](../azure-monitor/logs/data-retention-archive.md) (one table at a time)
20+
- [Configure data retention for multiple tables](https://github.com/Azure/Azure-Sentinel/tree/master/Tools/Archive-Log-Tool) at once
2121

2222
## Next steps
2323

articles/sentinel/configure-data-retention.md

Lines changed: 21 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Configure data retention for logs in Microsoft Sentinel or Azure Monitor
3-
description: In this tutorial, you'll configure an archive policy for a table in a Log Analytics workspace.
3+
description: In this tutorial, you'll configure a data retention policy for a table in a Log Analytics workspace.
44
author: cwatson-cat
55
ms.author: cwatson
66
ms.service: microsoft-sentinel
@@ -36,47 +36,48 @@ To complete the steps in this tutorial, you must have the following resources an
3636

3737
- Log Analytics workspace.
3838

39-
## Review interactive and long-term retention policies
40-
41-
On the **Tables** page for the table you updated, review the field values for **Interactive retention** and **Total retention period**. The long-term retention (archive) period equals the total retention period in days minus the interactive retention in days. For example, you set the following values:
42-
43-
| Field | Value |
44-
| ----- | ----- |
45-
| Interactive retention | 90 days |
46-
| Total retention period | 1.1 years |
47-
48-
So the **Tables** page shows the following an archive period of 310 days.
49-
50-
:::image type="content" source="media/configure-data-retention/data-retention-archive-period.png" alt-text="Screenshot of the table view that shows the interactive retention and archive period columns.":::
51-
5239
## Set the retention policy for a table
5340

54-
In your Log Analytics workspace, clear the **Use default workspace settings** setting if you want to change the interactive retention period from its default of 90 days (for Microsoft Sentinel workspaces) or 31 days (for other workspaces). Then, change the total retention policy for a table like **SecurityAlert** to 3 years of data. The *total retention* period is the sum of the *interactive* and *auxiliary* (archive) retention periods.
41+
In your Log Analytics workspace, change the interactive retention policy of the **SecurityEvent** table from the workspace default of 90 days to 180 days, and the total retention policy to 3 years. The *total retention* period is the sum of the *interactive* and *long-term* (archive) retention periods.
5542

5643
1. Sign in to the [Azure portal](https://portal.azure.com).
44+
5745
1. In the Azure portal, search for and open **Log Analytics workspaces**.
46+
5847
1. Select the appropriate workspace.
48+
5949
1. Under **Settings**, select **Tables**.
60-
1. On a table like **SecurityAlert**, open the context menu (...).
50+
51+
1. Find the **SecurityEvent** table in the list, and open the context menu (...).
52+
6153
1. Select **Manage table**.
54+
6255
:::image type="content" source="media/configure-data-retention/data-retention-tables.png" alt-text="Screenshot of the manage table option on the context menu for a table in the tables view.":::
63-
1. Under **Data retention**, enter the following values.
56+
57+
1. Under **Data retention settings**, enter the following values.
6458

6559
| Field | Value |
6660
| ----- | ----- |
67-
| Use default workspace settings | Clear the checkbox |
68-
| Interactive retention | 120 days |
61+
| Interactive retention | 180 days |
6962
| Total retention period | 3 years |
7063

7164
:::image type="content" source="media/configure-data-retention/data-retention-settings.png" alt-text="Screenshot of the data retention settings that shows the changes to the fields under the data retention section.":::
7265

66+
See that the time graph shows that the long-term retention period equals the total retention period in days minus the interactive retention period in days. In this case, 915 days, or 2.5 years.
67+
7368
1. Select **Save**.
7469

70+
## Review interactive and total retention policies
71+
72+
On the **Tables** page for the table you updated, review the field values for **Interactive retention** and **Total retention**.
73+
74+
:::image type="content" source="media/configure-data-retention/data-retention-archive-period.png" alt-text="Screenshot of the table view that shows the interactive retention and archive period columns.":::
75+
7576
## Clean up resources
7677

7778
No resources were created but you might want to restore the data retention settings you changed.
7879

7980
## Next steps
8081

8182
> [!div class="nextstepaction"]
82-
> [Configure data retention and archive policies in Azure Monitor Logs](../azure-monitor/logs/data-retention-archive.md?tabs=portal-1%2cportal-2)
83+
> [Configure interactive and long-term data retention policies in Azure Monitor Logs](../azure-monitor/logs/data-retention-archive.md?tabs=portal-1%2cportal-2)

articles/sentinel/connect-azure-functions-template.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ This article describes how to configure Microsoft Sentinel for using Azure Funct
1616
> [!NOTE]
1717
> - Once ingested in to Microsoft Sentinel, data is stored in the geographic location of the workspace in which you're running Microsoft Sentinel.
1818
>
19-
> For long-term retention, you may also want to store data in archive log types such as *Basic logs*. For more information, see [Data retention and archive in Azure Monitor Logs](../azure-monitor/logs/data-retention-archive.md).
19+
> For long-term retention, you may also want to store data in log types such as *Auxiliary logs* or *Basic logs*. For more information, see [Log retention plans in Microsoft Sentinel](log-plans.md).
2020
>
2121
> - Using Azure Functions to ingest data into Microsoft Sentinel may result in additional data ingestion costs. For more information, see the [Azure Functions pricing](https://azure.microsoft.com/pricing/details/functions/) page.
2222

articles/sentinel/enable-monitoring.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ This article instructs you how to turn on these features.
1717

1818
To implement the health and audit feature using API (Bicep/ARM/REST), review the [Diagnostic Settings operations](/rest/api/monitor/diagnostic-settings).
1919

20-
To configure the retention time for your audit and health events, see [Configure data retention and archive policies in Azure Monitor Logs](../azure-monitor/logs/data-retention-archive.md).
20+
To configure the retention time for your audit and health events, see [Configure a data retention policy for a table in a Log Analytics workspace](configure-data-retention.md).
2121

2222
> [!IMPORTANT]
2323
>

articles/sentinel/investigate-large-datasets.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ ms.collection: usx-security
1515

1616
One of the primary activities of a security team is to search logs for specific events. For example, you might search logs for the activities of a specific user within a given time-frame.
1717

18-
In Microsoft Sentinel, you can search across long time periods in extremely large datasets by using a search job. While you can run a search job on any type of log, search jobs are ideally suited to search archived logs. If you need to do a full investigation on archived data, you can restore that data into the hot cache to run high performing queries and deeper analysis.
18+
In Microsoft Sentinel, you can search across long time periods in extremely large datasets by using a search job. While you can run a search job on any type of log, search jobs are ideally suited to search logs in a long-term retention (formerly known as archive) state. If you need to do a full investigation on such data, you can restore that data into an interactive retention state—like your regular Log Analytics tables— to run high performing queries and deeper analysis.
1919

2020
[!INCLUDE [unified-soc-preview](includes/unified-soc-preview.md)]
2121

@@ -25,7 +25,7 @@ Use a search job when you start an investigation to find specific events in logs
2525

2626
Search in Microsoft Sentinel is built on top of search jobs. Search jobs are asynchronous queries that fetch records. The results are returned to a search table that's created in your Log Analytics workspace after you start the search job. The search job uses parallel processing to run the search across long time spans, in extremely large datasets. So search jobs don't impact the workspace's performance or availability.
2727

28-
Search results are stored in a table that has a *_SRCH suffix.
28+
Search results are stored in a table named with a `_SRCH` suffix.
2929

3030
The following image shows example search criteria for a search job.
3131

@@ -37,6 +37,7 @@ Use search to find events in any of the following log types:
3737

3838
- [Analytics logs](../azure-monitor/logs/data-platform-logs.md)
3939
- [Basic logs](../azure-monitor/logs/basic-logs-configure.md)
40+
- [Auxiliary logs]
4041

4142
You can also search analytics or basic log data stored in [archived logs](../azure-monitor/logs/data-retention-archive.md).
4243

articles/sentinel/log-plans.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ Some examples of primary data sources include logs from antivirus or enterprise
5050

5151
Logs containing primary security data should be stored using the **Analytics logs** plan. This plan keeps data in an **interactive retention** state for **90 days** by default, extensible for up to two years. In this state, your data can be queried in unlimited fashion and with high performance.
5252

53-
When the interactive retention period ends, data goes into a **long-term retention** state, remaining in its original table. Long-term retention is not defined by default, but you can define it to last up to 12 years. This state preserves your data for regulatory compliance or internal policy purposes. Data in this state can be queried in limited fashion and with much slower performance, but you can use a [**search job**](../azure-monitor/logs/search-jobs.md) or [**restore**](../azure-monitor/logs/restore.md) to pull out limited sets of data into interactive retention, where you can bring the full query capabilities to bear on it.
53+
When the interactive retention period ends, data goes into a **long-term retention** state, remaining in its original table. Long-term retention is not defined by default, but you can define it to last up to 12 years. This state preserves your data for regulatory compliance or internal policy purposes. Data in this state can be queried in limited fashion and with much slower performance, but you can use a [**search job**](investigate-large-datasets.md) or [**restore**](restore.md) to pull out limited sets of data into interactive retention, where you can bring the full query capabilities to bear on it.
5454

5555
### Secondary security data
5656

@@ -68,7 +68,7 @@ Some examples of secondary data log sources are cloud storage access logs, NetFl
6868

6969
Logs containing secondary security data should be stored using the **Auxiliary logs** plan. This plan keeps data in an **interactive retention** state for **30 days**. In this state, your data can be queried with limited capabilities and with lower performance.
7070

71-
When the interactive retention period ends, data goes into a **long-term retention** state, remaining in its original table. Long-term retention in the auxiliary logs plan is similar to long-term retention in the analytics logs plan, except that the only option to rehydrate data is with a [**search job**](../azure-monitor/logs/search-jobs.md). [Restore](../azure-monitor/logs/restore.md) is not supported for auxiliary logs.
71+
When the interactive retention period ends, data goes into a **long-term retention** state, remaining in its original table. Long-term retention in the auxiliary logs plan is similar to long-term retention in the analytics logs plan, except that the only option to rehydrate data is with a [**search job**](investigate-large-datasets.md). [Restore](restore.md) is not supported for auxiliary logs.
7272

7373
## Log management plans
7474

-53.6 KB
Loading
32.1 KB
Loading

0 commit comments

Comments
 (0)