Merge pull request #289973 from austinmccollum/main

denrea · web-flow · commit 53a3635fd9a3 · 2024-11-06T09:52:24.000-08:00
remove old and confusing information
diff --git a/articles/sentinel/connect-data-sources.md b/articles/sentinel/connect-data-sources.md
@@ -3,16 +3,13 @@ title: Microsoft Sentinel data connectors
 description: Learn about supported data connectors, like Microsoft Defender XDR (formerly Microsoft 365 Defender), Microsoft 365 and Office 365, Microsoft Entra ID, ATP, and Defender for Cloud Apps to Microsoft Sentinel.
 author: yelevin
 ms.topic: conceptual
-ms.date: 03/02/2024
+ms.date: 11/06/2024
 ms.author: yelevin
 appliesto:
     - Microsoft Sentinel in the Azure portal
     - Microsoft Sentinel in the Microsoft Defender portal
 ms.collection: usx-security
-
-
 #Customer intent: As a security eningeer, I want to use data connectors to integrate various data sources into Microsoft Sentinel so that I can enhance threat detection and response capabilities.
-
 ---
 
 # Microsoft Sentinel data connectors
@@ -55,26 +52,16 @@ To add more data connectors, install the solution associated with the data conne
 
 ## REST API integration for data connectors
 
-Many security technologies provide a set of APIs for retrieving log files. Some data sources can use those APIs to connect to Microsoft Sentinel.
-
-Data connectors that use APIs either integrate from the provider side or integrate using Azure Functions, as described in the following sections.
-
-### Integration on the provider side
-
-An API integration built by the provider connects with the provider data sources and pushes data into Microsoft Sentinel custom log tables by using the Azure Monitor Data Collector API. For more information, see [Send log data to Azure Monitor by using the HTTP Data Collector API](/azure/azure-monitor/logs/data-collector-api?branch=main&tabs=powershell).
-
-To learn about REST API integration, read your provider documentation and [Connect your data source to Microsoft Sentinel's REST-API to ingest data](connect-rest-api-template.md).
-
-### Integration using Azure Functions
-
-Integrations that use Azure Functions to connect with a provider API first format the data, and then send it to Microsoft Sentinel custom log tables using the Azure Monitor Data Collector API.
+Many security solutions provide a set of APIs for retrieving log files and other security data from their product or service. Those APIs connect to Microsoft Sentinel with one of the following methods:
+- The data source APIs are configured with the [Codeless Connector Platform](create-codeless-connector.md).
+- The data connector uses the Log Ingestion API for Azure Monitor as part of an Azure Function or Logic App.
 
-For more information, see:
--  [Send log data to Azure Monitor by using the HTTP Data Collector API](/azure/azure-monitor/logs/data-collector-api?branch=main&tabs=powershell)
+For more information about connecting with Azure Functions, see the following articles:
 - [Use Azure Functions to connect your data source to Microsoft Sentinel](connect-azure-functions-template.md)
 - [Azure Functions documentation](../azure-functions/index.yml)
+- [Azure Functions pricing](https://azure.microsoft.com/pricing/details/functions/)
 
-Integrations that use Azure Functions might have extra data ingestion costs, because you host Azure Functions in your Azure organization. Learn more about [Azure Functions pricing](https://azure.microsoft.com/pricing/details/functions/).
+For more information about connecting with Logic Apps, see [Connect with Logic Apps](create-custom-connector.md#connect-with-logic-apps).
 
 ## Agent-based integration for data connectors
 
diff --git a/articles/sentinel/create-custom-connector.md b/articles/sentinel/create-custom-connector.md
@@ -3,7 +3,7 @@ title: Resources for creating Microsoft Sentinel custom connectors
 description: Learn about available resources for creating custom connectors for Microsoft Sentinel. Methods include the Log Analytics API, Logstash, Logic Apps, PowerShell, and Azure Functions.
 author: austinmccollum
 ms.topic: conceptual
-ms.date: 10/01/2024
+ms.date: 11/06/2024
 ms.author: austinmc
 #Customer intent: As a security engineer, I want to know which Microsoft Sentinel custom data connector would be most appropriate to build for ingesting data from sources with no out-of-the-box solution.
 
@@ -27,21 +27,20 @@ The following table compares essential details about each method for creating cu
 |**[Azure Monitor Agent](#connect-with-the-azure-monitor-agent)** <br>Best for collecting files from on-premises and IaaS sources   | File collection, data transformation  |   No      | Low         |
 |**[Logstash](#connect-with-logstash)** <br>Best for on-premises and IaaS sources, any source for which a plugin is available, and organizations already familiar with Logstash  | Supports all capabilities of the Azure Monitor Agent  |   No; requires a VM or VM cluster to run           |   Low; supports many scenarios with plugins      |
 |**[Logic Apps](#connect-with-logic-apps)** <br>High cost; avoid for high-volume data <br>Best for low-volume cloud sources  | Codeless programming allows for limited flexibility, without support for implementing algorithms.<br><br> If no available action already supports your requirements, creating a custom action may add complexity.    |    Yes         |   Low; simple, codeless development      |
-|**[PowerShell](#connect-with-powershell)** <br>Best for prototyping and periodic file uploads | Direct support for file collection. <br><br>PowerShell can be used to collect more sources, but will require coding and configuring the script as a service.      |No               |  Low       |
-|**[Log Analytics API](#connect-with-the-log-analytics-api)** <br>Best for ISVs implementing integration, and for unique collection requirements   | Supports all capabilities available with the code.  | Depends on the implementation           |     High    |
+|**[Log Ingestion API in Azure Monitor](#connect-with-the-log-ingestion-api)** <br>Best for ISVs implementing integration, and for unique collection requirements   | Supports all capabilities available with the code.  | Depends on the implementation           |     High    |
 |**[Azure Functions](#connect-with-azure-functions)** <br>Best for high-volume cloud sources, and for unique collection requirements  | Supports all capabilities available with the code.  |  Yes             |     High; requires programming knowledge    |
 
 
 > [!TIP]
 > For comparisons of using Logic Apps and Azure Functions for the same connector, see:
 >
-> - [Ingest Fastly Web Application Firewall logs into Microsoft Sentinel](https://techcommunity.microsoft.com/t5/azure-sentinel/ingest-fastly-web-application-firewall-logs-into-azure-sentinel/ba-p/1238804)
+> - [Ingest Fastly Web Application Firewall logs into Microsoft Sentinel](https://techcommunity.microsoft.com/blog/microsoftsentinelblog/ingest-fastly-web-application-firewall-logs-into-azure-sentinel/1238804)
 > - Office 365 (Microsoft Sentinel GitHub community): [Logic App connector](https://github.com/Azure/Azure-Sentinel/tree/master/Playbooks/Get-O365Data) | [Azure Function connector](https://github.com/Azure/Azure-Sentinel/tree/master/DataConnectors/O365%20Data)
 >
 
 ## Connect with the Codeless Connector Platform
 
-The Codeless Connector Platform (CCP) provides a configuration file that can be used by both customers and partners, and then deployed to your own workspace, or as a solution to Microsoft Sentinel's solution's gallery.
+The Codeless Connector Platform (CCP) provides a configuration file that can be used by both customers and partners, and then deployed to your own workspace, or as a solution to Microsoft Sentinel's content hub.
 
 Connectors created using the CCP are fully SaaS, without any requirements for service installations, and also include health monitoring and full support from Microsoft Sentinel.
 
@@ -63,7 +62,7 @@ With the Microsoft Sentinel Logstash Output plugin, you can use any Logstash inp
 
 For examples of using Logstash as a custom connector, see:
 
-- [Hunting for Capital One Breach TTPs in AWS logs using Microsoft Sentinel](https://techcommunity.microsoft.com/t5/azure-sentinel/hunting-for-capital-one-breach-ttps-in-aws-logs-using-azure/ba-p/1019767) (blog)
+- [Hunting for Capital One Breach TTPs in AWS logs using Microsoft Sentinel](https://techcommunity.microsoft.com/blog/microsoftsentinelblog/hunting-for-capital-one-breach-ttps-in-aws-logs-using-azure-sentinel---part-i/1014258) (blog)
 - [Radware Microsoft Sentinel implementation guide](https://support.radware.com/ci/okcsFattach/get/1025459_3)
 
 For examples of useful Logstash plugins, see:
@@ -74,7 +73,7 @@ For examples of useful Logstash plugins, see:
 - [Google_pubsub input plugin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-google_pubsub.html)
 
 > [!TIP]
-> Logstash also enables scaled data collection using a cluster. For more information, see [Using a load-balanced Logstash VM at scale](https://techcommunity.microsoft.com/t5/azure-sentinel/scaling-up-syslog-cef-collection/ba-p/1185854).
+> Logstash also enables scaled data collection using a cluster. For more information, see [Using a load-balanced Logstash VM at scale](https://techcommunity.microsoft.com/blog/microsoftsentinelblog/scaling-up-syslog-cef-collection/1185854).
 >
 
 ## Connect with Logic Apps
@@ -120,59 +119,18 @@ For examples of how you can create a custom connector for Microsoft Sentinel usi
 
 - [Create a data pipeline with the Data Collector API](/connectors/azureloganalyticsdatacollector/)
 - [Palo Alto Prisma Logic App connector using a webhook](https://github.com/Azure/Azure-Sentinel/tree/master/Playbooks/Ingest-Prisma) (Microsoft Sentinel GitHub community)
-- [Secure your Microsoft Teams calls with scheduled activation](https://techcommunity.microsoft.com/t5/azure-sentinel/secure-your-calls-monitoring-microsoft-teams-callrecords/ba-p/1574600) (blog)
-- [Ingesting AlienVault OTX threat indicators into Microsoft Sentinel](https://techcommunity.microsoft.com/t5/azure-sentinel/ingesting-alien-vault-otx-threat-indicators-into-azure-sentinel/ba-p/1086566) (blog)
+- [Secure your Microsoft Teams calls with scheduled activation](https://techcommunity.microsoft.com/blog/microsoftsentinelblog/secure-your-calls--monitoring-microsoft-teams-callrecords-activity-logs-using-az/1574600) (blog)
+- [Ingesting AlienVault OTX threat indicators into Microsoft Sentinel](https://techcommunity.microsoft.com/blog/microsoftsentinelblog/ingesting-alien-vault-otx-threat-indicators-into-azure-sentinel/1086566) (blog)
 
-## Connect with PowerShell
-
-The [Upload-AzMonitorLog PowerShell script](https://www.powershellgallery.com/packages/Upload-AzMonitorLog/) enables you to use PowerShell to stream events or context information to Microsoft Sentinel from the command line. This streaming effectively creates a custom connector between your data source and Microsoft Sentinel.
-
-For example, the following script uploads a CSV file to Microsoft Sentinel:
-
-``` PowerShell
-Import-Csv .\testcsv.csv
-| .\Upload-AzMonitorLog.ps1
--WorkspaceId '69f7ec3e-cae3-458d-b4ea-6975385-6e426'
--WorkspaceKey $WSKey
--LogTypeName 'MyNewCSV'
--AddComputerName
--AdditionalDataTaggingName "MyAdditionalField"
--AdditionalDataTaggingValue "Foo"
-```
-
-The [Upload-AzMonitorLog PowerShell script](https://www.powershellgallery.com/packages/Upload-AzMonitorLog/) script uses the following parameters:
-
-|Parameter  |Description  |
-|---------|---------|
-|**WorkspaceId**     |   Your Microsoft Sentinel workspace ID, where you'll be storing your data.  [Find your workspace ID and key](#find-your-workspace-id-and-key).  |
-|**WorkspaceKey**     |   The primary or secondary key for the Microsoft Sentinel workspace where you'll be storing your data. [Find your workspace ID and key](#find-your-workspace-id-and-key).  |
-|**LogTypeName**     |    The name of the custom log table where you want to store the data. A suffix of **_CL** will automatically be added to the end of your table name.  |
-|**AddComputerName**     |   When this parameter exists, the script adds the current computer name to every log record, in a field named **Computer**.      |
-|**TaggedAzureResourceId**     | When this parameter exists, the script associates all uploaded log records with the specified Azure resource. <br><br>This association enables the uploaded log records for resource-context queries, and adheres to resource-centric, role-based access control.       |
-|**AdditionalDataTaggingName**     |      When this parameter exists, the script adds another field to every log record, with the configured name, and the value that's configured for the **AdditionalDataTaggingValue** parameter. <br><br>In this case, **AdditionalDataTaggingValue** must not be empty. |
-|**AdditionalDataTaggingValue**     |   When this parameter exists, the script adds another field to every log record, with the configured value, and the field name configured for the **AdditionalDataTaggingName** parameter. <br><br>If the **AdditionalDataTaggingName** parameter is empty, but a value is configured, the default field name is **DataTagging**.       |
-
-
-### Find your workspace ID and key
-
-Find the details for the **WorkspaceID** and **WorkspaceKey** parameters in Microsoft Sentinel:
-
-1. In Microsoft Sentinel, select **Settings** on the left, and then select the **Workspace settings** tab.
-
-1. Under **Get started with Log Analytics** > **1 Connect a data source**, select **Windows and Linux agents management**.
-
-1. Find your workspace ID, primary key, and secondary key on the **Windows servers** tabs.
-
-## Connect with the Log Analytics API
+## Connect with the Log Ingestion API
 
 You can stream events to Microsoft Sentinel by using the Log Analytics Data Collector API to call a RESTful endpoint directly.
 
 While calling a RESTful endpoint directly requires more programming, it also provides more flexibility.
 
-For more information, see the [Log Analytics Data collector API](/azure/azure-monitor/logs/data-collector-api), especially the following examples:
-
-- [C#](/azure/azure-monitor/logs/data-collector-api#sample-requests)
-- [Python](/azure/azure-monitor/logs/data-collector-api#sample-requests)
+For more information, see the following articles:
+- [Log Ingestion API in Azure Monitor](/azure/azure-monitor/logs/logs-ingestion-api-overview).
+- [Sample code to send data to Azure Monitor using Logs ingestion API](/azure/azure-monitor/logs/tutorial-logs-ingestion-code).
 
 ## Connect with Azure Functions
 
@@ -185,7 +143,7 @@ For examples of this method, see:
 - [Connect your Proofpoint TAP to Microsoft Sentinel with Azure Function](./data-connectors/proofpoint-tap-using-azure-functions.md)
 - [Connect your Qualys VM to Microsoft Sentinel with Azure Function](data-connectors/qualys-vulnerability-management-using-azure-functions.md)
 - [Ingesting XML, CSV, or other formats of data](/azure/azure-monitor/logs/create-pipeline-datacollector-api#ingesting-xml-csv-or-other-formats-of-data)
-- [Monitoring Zoom with Microsoft Sentinel](https://techcommunity.microsoft.com/t5/azure-sentinel/monitoring-zoom-with-azure-sentinel/ba-p/1341516) (blog)
+- [Monitoring Zoom with Microsoft Sentinel](https://techcommunity.microsoft.com/blog/microsoftsentinelblog/monitoring-zoom-with-azure-sentinel/1341516) (blog)
 - [Deploy a Function App for getting Office 365 Management API data into Microsoft Sentinel](https://github.com/Azure/Azure-Sentinel/tree/master/DataConnectors/O365%20Data) (Microsoft Sentinel GitHub community)
 
 ## Parse your custom connector data
@@ -207,6 +165,4 @@ Use the data ingested into Microsoft Sentinel to secure your environment with an
 - [Investigate incidents](investigate-cases.md)
 - [Detect threats](threat-detection.md)
 - [Automate threat prevention](tutorial-respond-threats-playbook.md)
-- [Hunt for threats](hunting.md)
-
-Also, learn about one example of creating a custom connector to monitor Zoom: [Monitoring Zoom with Microsoft Sentinel](https://techcommunity.microsoft.com/t5/azure-sentinel/monitoring-zoom-with-azure-sentinel/ba-p/1341516).
+- [Hunt for threats](hunting.md)
