Merge pull request #172153 from yelevin/yelevin/wef-connector

Prep for event forwarding doc

Author: v-albemi

.openpublishing.redirection.json

@@ -26176,6 +26176,11 @@
       "redirect_url": "/azure/sentinel/data-connectors-reference#salesforce-service-cloud-preview",
       "redirect_document_id": false
     },
+    {
+      "source_path_from_root": "/articles/sentinel/connect-security-events.md",
+      "redirect_url": "/azure/sentinel/data-connectors-reference#security-events-windows",
+      "redirect_document_id": false
+    },
     {
       "source_path_from_root": "/articles/sentinel/connect-sophos-cloud-optix.md",
       "redirect_url": "/azure/sentinel/data-connectors-reference#sophos-cloud-optix-preview",
@@ -26241,6 +26246,11 @@
       "redirect_url": "/azure/sentinel/data-connectors-reference#windows-firewall",
       "redirect_document_id": false
     },
+    {
+      "source_path_from_root": "/articles/sentinel/connect-windows-security-events.md",
+      "redirect_url": "/azure/sentinel/data-connectors-reference#windows-security-events-preview",
+      "redirect_document_id": false
+    },
     {
       "source_path_from_root": "/articles/sentinel/connect-wirex-systems.md",
       "redirect_url": "/azure/sentinel/data-connectors-reference#wirex-network-forensics-platform-preview",

articles/sentinel/TOC.yml

@@ -137,7 +137,7 @@
       - name: Microsoft 365 Defender
         href: connect-microsoft-365-defender.md
       - name: Windows security events
-        href: connect-windows-security-events.md
+        href: data-connectors-reference.md#windows-security-events-via-ama
     - name: Connection instructions by type
       items:
       - name: Azure, Windows, and Microsoft connectors
@@ -318,6 +318,8 @@
         href: registry-event-normalization-schema.md
       - name: Legacy network normalization schema
         href: normalization-schema-v1.md
+    - name: Windows security event sets
+      href: windows-security-event-id-reference.md
   - name: Detection and analysis references
     items:
     - name: Top Azure Sentinel workbooks

articles/sentinel/connect-azure-windows-microsoft-services.md

@@ -131,7 +131,125 @@ Connectors of this type use Azure Policy to apply a single diagnostic settings c
 
 You can find and query the data for each resource type using the table name that appears in the section for the resource's connector in the [Data connectors reference](data-connectors-reference.md) page.
 
-## Log Analytics agent-based connections
+## Windows agent-based connections
+
+# [Azure Monitor Agent](#tab/AMA)
+
+> [!IMPORTANT]
+>
+> - Some connectors based on the Azure Monitor Agent (AMA) are currently in **PREVIEW**. See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.
+
+The [Azure Monitor agent](../azure-monitor/agents/azure-monitor-agent-overview.md) uses **Data collection rules (DCRs)** to define the data to collect from each agent. Data collection rules offer you two distinct advantages:
+
+- **Manage collection settings at scale** while still allowing unique, scoped configurations for subsets of machines. They are independent of the workspace and independent of the virtual machine, which means they can be defined once and reused across machines and environments. See [Configure data collection for the Azure Monitor agent](../azure-monitor/agents/data-collection-rule-azure-monitor-agent.md).
+
+- **Build custom filters** to choose the exact events you want to ingest. The Azure Monitor Agent uses these rules to filter the data *at the source* and ingest only the events you want, while leaving everything else behind. This can save you a lot of money in data ingestion costs!
+
+See below how to create data collection rules.
+
+### Prerequisites
+
+- You must have read and write permissions on the Azure Sentinel workspace.
+
+- To collect events from any system that is not an Azure virtual machine, the system must have [**Azure Arc**](../azure-monitor/agents/azure-monitor-agent-install.md) installed and enabled *before* you enable the Azure Monitor Agent-based connector.
+
+   This includes:
+   
+    - Windows servers installed on physical machines
+    - Windows servers installed on on-premises virtual machines
+    - Windows servers installed on virtual machines in non-Azure clouds
+
+### Instructions
+
+1. From the Azure Sentinel navigation menu, select **Data connectors**. Select your connector from the list, and then select **Open connector page** on the details pane. Then follow the on-screen instructions under the **Instructions** tab, as described through the rest of this section.
+
+1. Verify that you have the appropriate permissions as described under the **Prerequisites** section on the connector page.
+
+1. Under **Configuration**, select **+Add data collection rule**. The **Create data collection rule** wizard will open to the right.
+
+1. Under **Basics**, enter a **Rule name** and specify a **Subscription** and **Resource group** where the data collection rule (DCR) will be created. This *does not* have to be the same resource group or subscription the monitored machines and their associations are in, as long as they are in the same tenant.
+
+1. In the **Resources** tab, select **+Add resource(s)** to add machines to which the Data Collection Rule will apply. The **Select a scope** dialog will open, and you will see a list of available subscriptions. Expand a subscription to see its resource groups, and expand a resource group to see the available machines. You will see Azure virtual machines and Azure Arc-enabled servers in the list. You can mark the check boxes of subscriptions or resource groups to select all the machines they contain, or you can select individual machines. Select **Apply** when you've chosen all your machines. At the end of this process, the Azure Monitor Agent will be installed on any selected machines that don't already have it installed.
+
+1. On the **Collect** tab, choose the events you would like to collect: select **All events** or **Custom** to specify other logs or to filter events using [XPath queries](../azure-monitor/agents/data-collection-rule-azure-monitor-agent.md#limit-data-collection-with-custom-xpath-queries) (see note below). Enter expressions in the box that evaluate to specific XML criteria for events to collect, then select **Add**. You can enter up to 20 expressions in a single box, and up to 100 boxes in a rule.
+
+    Learn more about [data collection rules](../azure-monitor/agents/data-collection-rule-overview.md#create-a-dcr) from the Azure Monitor documentation.
+
+    > [!NOTE]
+    > - The Windows Security Events connector offers two other [**pre-built event sets**](windows-security-event-id-reference.md) you can choose to collect: **Common** and **Minimal**.
+    >
+    > - The Azure Monitor agent supports XPath queries for **[XPath version 1.0](/windows/win32/wes/consuming-events#xpath-10-limitations) only**.
+
+1. When you've added all the filter expressions you want, select **Next: Review + create**.
+
+1. When you see the "Validation passed" message, select **Create**. 
+
+You'll see all your data collection rules (including those created through the API) under **Configuration** on the connector page. From there you can edit or delete existing rules.
+
+> [!TIP]
+> Use the PowerShell cmdlet **Get-WinEvent** with the *-FilterXPath* parameter to test the validity of an XPath query. The following script shows an example:
+> ```powershell
+> $XPath = '*[System[EventID=1035]]'
+> Get-WinEvent -LogName 'Application' -FilterXPath $XPath
+> ```
+> - If events are returned, the query is valid.
+> - If you receive the message "No events were found that match the specified selection criteria," the query may be valid, but there are no matching events on the local machine.
+> - If you receive the message "The specified query is invalid," the query syntax is invalid.
+
+### Create data collection rules using the API
+
+You can also create data collection rules using the API ([see schema](/rest/api/monitor/data-collection-rules)), which can make life easier if you're creating many rules (if you're an MSSP, for example). Here's an example (for the [Windows Security Events via AMA](data-connectors-reference.md#windows-security-events-via-ama) connector) that you can use as a template for creating a rule:
+
+**Request URL and header**
+
+```http
+PUT https://management.azure.com/subscriptions/703362b3-f278-4e4b-9179-c76eaf41ffc2/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionRules/myCollectionRule?api-version=2019-11-01-preview
+```
+
+**Request body**
+
+```json
+{
+    "location": "eastus",
+    "properties": {
+        "dataSources": {
+            "windowsEventLogs": [
+                {
+                    "streams": [
+                        "Microsoft-SecurityEvent"
+                    ],
+                    "xPathQueries": [
+                        "Security!*[System[(EventID=) or (EventID=4688) or (EventID=4663) or (EventID=4624) or (EventID=4657) or (EventID=4100) or (EventID=4104) or (EventID=5140) or (EventID=5145) or (EventID=5156)]]"
+                    ],
+                    "name": "eventLogsDataSource"
+                }
+            ]
+        },
+        "destinations": {
+            "logAnalytics": [
+                {
+                    "workspaceResourceId": "/subscriptions/703362b3-f278-4e4b-9179-c76eaf41ffc2/resourceGroups/myResourceGroup/providers/Microsoft.OperationalInsights/workspaces/centralTeamWorkspace",
+                    "name": "centralWorkspace"
+                }
+            ]
+        },
+        "dataFlows": [
+            {
+                "streams": [
+                    "Microsoft-SecurityEvent"
+                ],
+                "destinations": [
+                    "centralWorkspace"
+                ]
+            }
+        ]
+    }
+}
+```
+See this [complete description of data collection rules](../azure-monitor/agents/data-collection-rule-overview.md) from the Azure Monitor documentation.
+
+
+# [Log Analytics Agent (Legacy)](#tab/LAA)
 
 ### Prerequisites
 
@@ -140,6 +258,8 @@ You can find and query the data for each resource type using the table name that
 
 ### Instructions
 
+#### Install the agent
+
 1. From the Azure Sentinel navigation menu, select **Data connectors**.
 
 1. Select your service (**DNS** or **Windows Firewall**) and then select **Open connector page**.
@@ -149,12 +269,25 @@ You can find and query the data for each resource type using the table name that
     | Machine type  | Instructions  |
     | --------- | --------- |
     | **For an Azure Windows VM** | 1. Under **Choose where to install the agent**, expand **Install agent on Azure Windows virtual machine**. <br><br>2. Select the **Download & install agent for Azure Windows Virtual machines >** link. <br><br>3. In the **Virtual machines** blade, select a virtual machine to install the agent on, and then select **Connect**. Repeat this step for each VM you wish to connect. |
-    | **For any other Windows machine** | 1. Under **Choose where to install the agent**, expand **Install agent on non-Azure Windows Machine** <br><br>2. Select the **Download & install agent for non-Azure Windows machines >** link.  <br><br>3. In the **Agents management** blade, on the **Windows servers** tab, select the **Download Windows Agent** link for either 32-bit or 64-bit systems, as appropriate.      |
+    | **For any other Windows machine** | 1. Under **Choose where to install the agent**, expand **Install agent on non-Azure Windows Machine** <br><br>2. Select the **Download & install agent for non-Azure Windows machines >** link.  <br><br>3. In the **Agents management** blade, on the **Windows servers** tab, select the **Download Windows Agent** link for either 32-bit or 64-bit systems, as appropriate.  <br><br>4. Using the downloaded executable file, install the agent on the Windows systems of your choice, and configure it using the **Workspace ID and Keys** that appear below the download links in the previous step. |
+    | | |
+
+> [!NOTE]
+>
+> To allow Windows systems without the necessary internet connectivity to still stream events to Azure Sentinel, download and install the **Log Analytics Gateway** on a separate machine, using the **Download Log Analytics Gateway** link on the **Agents Management** page, to act as a proxy.  You will still need to install the Log Analytics agent on each Windows system whose events you want to collect.
+>
+> For more information on this scenario, see the [**Log Analytics gateway** documentation](../azure-monitor/agents/gateway.md).
+
+For additional installation options and further details, see the [**Log Analytics agent** documentation](../azure-monitor/agents/agent-windows.md).
 
-1. Select the **Install solution** button for either DNS or Windows Firewall.
 
-You can find and query the data for DNS and Windows Firewall using the **DnsEvents**, **DnsInventory**, and **WindowsFirewall** table names, respectively. You can see this and other information about these two service connectors in their sections in the [Data connectors reference](data-connectors-reference.md) page.
+#### Determine the logs to send
 
+For the Windows DNS Server and Windows Firewall connectors, select the **Install solution** button. For the legacy Security Events connector, choose the [**event set**](windows-security-event-id-reference.md) you wish to send and select **Update**.
+
+You can find and query the data for these services using the table names in their respective sections in the [Data connectors reference](data-connectors-reference.md) page.
+
+---
 
 ## Next steps