Merge pull request #268479 from austinmccollum/austinmc-splunk-migrate

initial article creation

Author: AnnaMHuff

articles/sentinel/TOC.yml

@@ -64,7 +64,9 @@
   - name: Plan and design your migration
     items:
     - name: Plan your migration
-      href: migration.md      
+      href: migration.md
+    - name: Use the SIEM migration experience
+      href: siem-migration.md
     - name: Track migration with a workbook
       href: migration-track.md                
   - name: Migrate from ArcSight      

articles/sentinel/media/siem-migration/configure-rules.png

@@ -1,10 +1,12 @@
 ---
-title: Migrate Splunk detection rules to Microsoft Sentinel | Microsoft Docs
+title: Migrate Splunk detection rules to Microsoft Sentinel
+titleSuffix: Microsoft Sentinel
 description: Learn how to identify, compare, and migrate your Splunk detection rules to Microsoft Sentinel built-in rules.
 author: limwainstein
 ms.author: lwainstein
 ms.topic: how-to
-ms.date: 05/03/2022
+ms.date: 03/11/2024
+#customer intent: As a SOC administrator, I want to migrate Splunk detection rules to KQL so I can migrate to Microsoft Sentinel.
 ---
 
 # Migrate Splunk detection rules to Microsoft Sentinel
@@ -22,11 +24,12 @@ Microsoft Sentinel uses machine learning analytics to create high-fidelity and a
 - Check that you understand the [rule terminology](#compare-rule-terminology).
 - Review any rules that haven't triggered any alerts in the past 6-12 months, and determine whether they're still relevant.
 - Eliminate low-level threats or alerts that you routinely ignore.
-- Use existing functionality, and check whether Microsoft Sentinel’s [built-in analytics rules](https://github.com/Azure/Azure-Sentinel/tree/master/Detections) might address your current use cases. Because Microsoft Sentinel uses machine learning analytics to produce high-fidelity and actionable incidents, it’s likely that some of your existing detections won’t be required anymore.
+- Use existing functionality, and check whether Microsoft Sentinel's [built-in analytics rules](https://github.com/Azure/Azure-Sentinel/tree/master/Detections) might address your current use cases. Because Microsoft Sentinel uses machine learning analytics to produce high-fidelity and actionable incidents, it's likely that some of your existing detections won't be required anymore.
 - Confirm connected data sources and review your data connection methods. Revisit data collection conversations to ensure data depth and breadth across the use cases you plan to detect.
+- Test the capabilities of the [SIEM migration experience](siem-migration.md) to determine if the automated translation is suitable.
 - Explore community resources such as the [SOC Prime Threat Detection Marketplace](https://my.socprime.com/platform-overview/) to check whether  your rules are available.
 - Consider whether an online query converter such as Uncoder.io might work for your rules. 
-- If rules aren’t available or can’t be converted, they need to be created manually, using a KQL query. Review the [rules mapping](#map-and-compare-rule-samples) to create new queries. 
+- If rules aren't available or can't be converted, they need to be created manually, using a KQL query. Review the [rules mapping](#map-and-compare-rule-samples) to create new queries. 
 
 Learn more about [best practices for migrating detection rules](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/best-practices-for-migrating-detection-rules-from-arcsight/ba-p/2216417).
 
@@ -41,14 +44,18 @@ Learn more about [best practices for migrating detection rules](https://techcomm
     1. **Confirm that you have any required data sources connected,** and review your data connection methods.
 
 1. Verify whether your detections are available as built-in templates in Microsoft Sentinel:
+    
+    - **Use the SIEM migration experience** to automate translation and migration.
+
+        For more information, see [Use the SIEM migration experience](siem-migration.md).
 
     - **If the built-in rules are sufficient**, use built-in rule templates to create rules for your own workspace.
 
         In Microsoft Sentinel, go to the **Configuration > Analytics > Rule templates** tab, and create and update each relevant analytics rule.
 
         For more information, see [Detect threats out-of-the-box](detect-threats-built-in.md).
 
-    - **If you have detections that aren't covered by Microsoft Sentinel's built-in rules**, try an online query converter, such as [Uncoder.io](https://uncoder.io/) to convert your queries to KQL.
+    - **If you have detections that aren't covered by Microsoft Sentinel's built-in rules**, try an online query converter, such as [Uncoder.io](https://uncoder.io/) or [SPL2KQL](https://azure.github.io/spl2kql) to convert your queries to KQL.
 
         Identify the trigger condition and rule action, and then construct and review your KQL query.
 
@@ -71,7 +78,7 @@ Learn more about [best practices for migrating detection rules](https://techcomm
 Learn more about analytics rules:
 
 - [**Create custom analytics rules to detect threats**](detect-threats-custom.md). Use [alert grouping](detect-threats-custom.md#alert-grouping) to reduce alert fatigue by grouping alerts that occur within a given timeframe.
-- [**Map data fields to entities in Microsoft Sentinel**](map-data-fields-to-entities.md) to enable SOC engineers to define entities as part of the evidence to track during an investigation. Entity mapping also makes it possible for SOC analysts to take advantage of an intuitive [investigation graph (investigate-cases.md#use-the-investigation-graph-to-deep-dive) that can help reduce time and effort.
+- [**Map data fields to entities in Microsoft Sentinel**](map-data-fields-to-entities.md) to enable SOC engineers to define entities as part of the evidence to track during an investigation. Entity mapping also makes it possible for SOC analysts to take advantage of an intuitive [investigation graph] (investigate-cases.md#use-the-investigation-graph-to-deep-dive) that can help reduce time and effort.
 - [**Investigate incidents with UEBA data**](investigate-with-ueba.md), as an example of how to use evidence to surface events, alerts, and any bookmarks associated with a particular incident in the incident preview pane.
 - [**Kusto Query Language (KQL)**](/azure/data-explorer/kusto/query/), which you can use to send read-only requests to your [Log Analytics](../azure-monitor/logs/log-analytics-tutorial.md) database to process data and return results. KQL is also used across other Microsoft services, such as [Microsoft Defender for Endpoint](https://www.microsoft.com/microsoft-365/security/endpoint-defender) and [Application Insights](../azure-monitor/app/app-insights-overview.md).
 
@@ -84,7 +91,7 @@ This table helps you to clarify the concept of a rule in Microsoft Sentinel comp
 |**Rule type** |• Scheduled<br>• Real-time |• Scheduled query<br>• Fusion<br>• Microsoft Security<br>• Machine Learning (ML) Behavior Analytics |
 |**Criteria** |Define in SPL |Define in KQL |
 |**Trigger condition** |• Number of results<br>• Number of hosts<br>• Number of sources<br>• Custom |Threshold: Number of query results |
-|**Action** |• Add to triggered alerts<br>• Log Event<br>• Output results to lookup<br>• And more |• Create alert or incident<br>• Integrates with Logic Apps |
+|**Action** |• Add to triggered alerts<br>• Log Event<br>• Output results to look up<br>• And more |• Create alert or incident<br>• Integrates with Logic Apps |
 
 ## Map and compare rule samples
 

articles/sentinel/media/siem-migration/siem-migration-experience.png

@@ -1,10 +1,12 @@
 ---
-title: "Microsoft Sentinel migration: Export Splunk data to target platform | Microsoft Docs"
-description: Learn how to export your historical data from Splunk.
+title: Export Splunk data to target platform
+titleSuffix: Microsoft Sentinel
+description: Learn how to export your historical data from Splunk for a Microsoft Sentinel migration of security monitoring use cases.
 author: limwainstein
 ms.author: lwainstein
 ms.topic: how-to
-ms.date: 05/03/2022
+ms.date: 03/11/2024
+#customer intent: As a SOC administrator, I want to migrate historical data from Splunk so I have continuity when I migrate to Microsoft Sentinel.
 ---
 
 # Export historical data from Splunk

articles/sentinel/media/siem-migration/upload-file.png

@@ -5,7 +5,7 @@ author: limwainstein
 ms.author: lwainstein
 ms.service: microsoft-sentinel
 ms.topic: how-to
-ms.date: 05/03/2022
+ms.date: 03/11/2024
 ---
 
 # Plan your migration to Microsoft Sentinel
@@ -25,6 +25,7 @@ In this guide, you learn how to migrate your legacy SIEM to Microsoft Sentinel.
 |---------|---------|
 |Plan your migration     |**You are here**         |
 |Track migration with a workbook     |[Track your Microsoft Sentinel migration with a workbook](migration-track.md)         |
+|Use the SIEM Migration experience | [SIEM Migration (Preview)](siem-migration.md)       |
 |Migrate from ArcSight     |• [Migrate detection rules](migration-arcsight-detection-rules.md)<br>• [Migrate SOAR automation](migration-arcsight-automation.md)<br>• [Export historical data](migration-arcsight-historical-data.md)          |
 |Migrate from Splunk     |• [Migrate detection rules](migration-splunk-detection-rules.md)<br>• [Migrate SOAR automation](migration-splunk-automation.md)<br>• [Export historical data](migration-splunk-historical-data.md)<br><br>If you want to migrate your Splunk Observability deployment, learn more about how to [migrate from Splunk to Azure Monitor Logs](../azure-monitor/logs/migrate-splunk-to-azure-monitor-logs.md).      |
 |Migrate from QRadar     |• [Migrate detection rules](migration-qradar-detection-rules.md)<br>• [Migrate SOAR automation](migration-qradar-automation.md)<br>• [Export historical data](migration-qradar-historical-data.md)          |

articles/sentinel/migration-splunk-detection-rules.md

@@ -0,0 +1,129 @@
+---
+title: Use the SIEM migration experience
+titleSuffix: Microsoft Sentinel
+description: Migrate security monitoring use cases from other Security Information and Event Management (SIEM) systems to Microsoft Sentinel. 
+author: austinmccollum
+ms.topic: how-to
+ms.date: 3/11/2024
+ms.author: austinmc
+appliesto: Microsoft Sentinel
+#customer intent: As an SOC administrator, I want to use the SIEM migration experience so I can migrate to Microsoft Sentinel.
+---
+
+# Migrate to Microsoft Sentinel with the SIEM migration experience (preview)
+
+Migrate your SIEM to Microsoft Sentinel for all your security monitoring use cases. Automated assistance from the SIEM Migration experience simplifies your migration. 
+
+These features are currently included in the SIEM Migration experience: 
+
+**Splunk**
+- The experience focuses on migrating Splunk security monitoring to Microsoft Sentinel.
+- The experience only supports migration of Splunk detections to Microsoft Sentinel analytics rules.
+
+## Prerequisites
+
+You need the following from the source SIEM:
+
+**Splunk**
+- The migration experience is compatible with both Splunk Enterprise and Splunk Cloud editions.
+- A Splunk admin role is required to export all Splunk alerts. For more information, see [Splunk role-based user access](https://docs.splunk.com/Documentation/Splunk/9.1.3/Security/Aboutusersandroles).
+- Export the historical data from Splunk to the relevant tables in the Log Analytics workspace. For more information, see [Export historical data from Splunk](migration-splunk-historical-data.md)
+
+You need the following on the target, Microsoft Sentinel:
+
+- The SIEM migration experience deploys analytics rules. This capability requires the **Microsoft Sentinel Contributor** role. For more information, see [Permissions in Microsoft Sentinel](roles.md). 
+- Ingest security data previously used in your source SIEM into Microsoft Sentinel by enabling an out-of-the-box (OOTB) data connector. 
+    - If the data connector isn't installed yet, find the relevant solution in **Content hub**. 
+    - If no data connector exists, create a custom ingestion pipeline.<br>For more information, see [Discover and manage Microsoft Sentinel out-of-the-box content](sentinel-solutions-deploy.md) or [Custom data ingestion and transformation](data-transformation.md).
+
+## Translate Splunk detection rules
+
+At the core of Splunk detection rules is the Search Processing Language (SPL). The SIEM migration experience systematically translates SPL to Kusto query language (KQL) for each Splunk rule. Carefully review translations and make adjustments to ensure migrated rules function as intended in your Microsoft Sentinel workspace. For more information on the concepts important in translating detection rules, see [migrate Splunk detection rules](migration-splunk-detection-rules.md).
+
+Capabilities in public preview:
+
+- Translate simple queries with a single data source
+- Direct translations listed in the article, [Splunk to Kusto cheat sheet](/azure/data-explorer/kusto/query/splunk-cheat-sheet)
+- Review translated query error feedback with edit capability to save time in the detection rule translation process
+
+Here are some of the priorities that are important to us as we continue to develop the translation technology:
+
+- Splunk Common Information Model (CIM) to Microsoft Sentinel's Advanced Security Information Model (ASIM) translation support
+- Translated queries feature a completeness status with translation states 
+- Multiple data sources and index
+- Rule correlations
+- Support for macros
+- Support for lookups 
+- Complex queries with joins
+
+## Start the SIEM migration experience
+
+1. Navigate to Microsoft Sentinel in the [Azure portal](https://portal.azure.com), under **Content management**, select **Content hub**.
+
+1. Select **SIEM Migration (Preview)**. 
+
+:::image type="content" source="media/siem-migration/siem-migration-experience.png" alt-text="Screenshot showing content hub with menu item for the SIEM migration experience.":::
+
+## Upload Splunk detections
+
+1. From Splunk Web, select **Search and Reporting** in the **Apps** panel. 
+
+1. Run the following query: 
+
+    `| rest splunk_server=local count=0 /services/saved/searches | search disabled=0 | table title,search ,*`
+
+1. Select the export button and choose JSON as the format. 
+
+1. Save the file. 
+
+1. Upload the exported Splunk JSON file.
+
+> [!NOTE]
+> The Splunk export must be a valid JSON file and the upload size is limited to 50 MB.
+
+:::image type="content" source="media/siem-migration/upload-file.png" alt-text="Screenshot showing the upload files tab.":::
+
+## Configure rules
+
+1. Select **Configure Rules**.
+
+1. Review the analysis of the Splunk export.
+
+    - **Name** is the original Splunk detection rule name.
+    - **Compatibility** indicates if a Sentinel OOTB analytics rule matches the Splunk detection logic. 
+
+    :::image type="content" source="media/siem-migration/configure-rules.png" alt-text="Screenshot showing the results of the automatic rule mapping." lightbox="media/siem-migration/configure-rules.png":::
+
+    > [!NOTE]
+    > Check the schema of the data types and fields used in the rule logic. Microsoft Sentinel Analytics require that the data type be present in the Log Analytics Workspace before the rule is enabled. It's also important the fields used in the query are accurate for the defined data type schema.
+
+1. When the review is complete, select **Review and migrate**.
+
+## Deploy the Analytics rules
+
+1. Select **Deploy** to start the deployment of analytics rules to your Microsoft Sentinel workspace. 
+
+   The following resources are deployed:
+   - For all OOTB matches, the corresponding solutions with the matched analytics rule are installed, and the matched rules are deployed as active analytics rules.
+   - All custom rules translated to Sentinel analytics rules are deployed as active analytics rules.
+
+1. View the properties of deployed rules from Microsoft Sentinel **Analytics**.
+
+   - All migrated rules are deployed with the Prefix **[Splunk Migrated]**.
+   - All migrated rules are set to disabled.
+   - The following properties are retained from the Splunk export wherever possible:<br>
+     `Severity`<br>
+     `queryFrequency`<br>
+     `queryPeriod`<br>
+     `triggerOperator`<br>
+     `triggerThreshold`<br>
+     `suppressionDuration`
+
+1. Enable rules you've reviewed and verified. 
+
+## Next step
+
+In this article, you learned how to use the SIEM migration experience. 
+
+> [!div class="nextstepaction"]
+> [Migrate Splunk detection rules](migration-splunk-detection-rules.md)

articles/sentinel/migration-splunk-historical-data.md

@@ -1,10 +1,10 @@
 ---
 title: What's new in Microsoft Sentinel
-description: This article describes new features in Microsoft Sentinel from the past few months.
+description: Learn about the latest new features and announcement in Microsoft Sentinel from the past few months.
 author: yelevin
 ms.author: yelevin
-ms.topic: conceptual
-ms.date: 02/28/2024
+ms.topic: concept
+ms.date: 03/11/2024
 ---
 
 # What's new in Microsoft Sentinel
@@ -23,7 +23,15 @@ The listed features were released in the last three months. For information abou
 
 ## March 2024
 
-[Data connectors for Syslog and CEF based on Azure Monitor Agent now generally available (GA)](#data-connectors-for-syslog-and-cef-based-on-azure-monitor-agent-now-generally-available-ga)
+- [SIEM migration experience (preview)](#siem-migration-experience-preview)
+- [Data connectors for Syslog and CEF based on Azure Monitor Agent now generally available (GA)](#data-connectors-for-syslog-and-cef-based-on-azure-monitor-agent-now-generally-available-ga)
+
+### SIEM migration experience (preview)
+
+The new Microsoft Sentinel Migration experience helps customers and partners to automate the process of migrating their security monitoring use cases hosted in non-Microsoft products into Microsoft Sentinel.
+- This first version of the tool supports migrations from Splunk
+
+For more information, see [Migrate to Microsoft Sentinel with the SIEM migration experience](siem-migration.md)
 
 ### Data connectors for Syslog and CEF based on Azure Monitor Agent now generally available (GA)
 
@@ -86,7 +94,7 @@ Incident tasks, which help you standardize your incident investigation and respo
 
 Microsoft Sentinel data connectors for Amazon Web Services (AWS) and Google Cloud Platform (GCP) now include supporting configurations to ingest data into workspaces in Azure Government clouds.
 
-The configurations for these connectors for Azure Government customers differs slightly from the public cloud configuration. See the relevant documentation for details:
+The configurations for these connectors for Azure Government customers differ slightly from the public cloud configuration. See the relevant documentation for details:
 
 - [Connect Microsoft Sentinel to Amazon Web Services to ingest AWS service log data](connect-aws.md)
 - [Ingest Google Cloud Platform log data into Microsoft Sentinel](connect-google-cloud-platform.md)