Merge pull request #231697 from oshezaf/asim/update-parser-contrib-and-more

asim/update-parser-contrib-and-more

Author: prmerger-automator[bot]

articles/sentinel/normalization-develop-parsers.md

@@ -29,7 +29,7 @@ To understand how parsers fit within the ASIM architecture, refer to the [ASIM a
 > ASIM is currently in PREVIEW. The [Azure Preview Supplemental Terms](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.
 
 
-## Custom parser development process
+## Custom ASIM parser development process
 
 The following workflow describes the high level steps in developing a custom ASIM, source-specific parser:
 
@@ -461,11 +461,13 @@ You may want to contribute the parser to the primary ASIM distribution. If accep
 
 To contribute your parsers:
 
-| Step | Description |
-| ---- | ----------- | 
-| Develop the parsers | - Develop both a filtering parser and a parameter-less parser.<br>- Create a YAML file for the parser as described in [Deploying Parsers](#deploy-parsers) above.|
-| Test the parsers | - Make sure that your parsers pass all [testings](#test-parsers) with no errors.<br>- If any warnings are left, document them in the parser YAML file as described below. |
-| Contribute | - Create a pull request against the [Microsoft Sentinel GitHub repository](https://github.com/Azure/Azure-Sentinel)<br>- Add to the PR your parsers YAML files to the ASIM parser folders (`/Parsers/ASim<schema>/Parsers`)<br>- Adds representative sample data to the sample data folder (`/Sample Data`) |
+- Develop both a filtering parser and a parameter-less parser.
+- Create a YAML file for the parser as described in [Deploying Parsers](#deploy-parsers) above.
+- Make sure that your parsers pass all [testings](#test-parsers) with no errors. If any warnings are left, [document them](#documenting-accepted-warnings) in the parser YAML file.
+- Create a pull request against the [Microsoft Sentinel GitHub repository](https://github.com/Azure/Azure-Sentinel), including:
+  - Your parsers YAML files in the ASIM parser folders (`/Parsers/ASim<schema>/Parsers`)
+  - Representative sample data according to the [samples submission guidelines](#samples-submission-guidelines).
+  - Test results according to the [test results submission guidelines](#test-results-submission-guidelines).
 
 ### Documenting accepted warnings
 
@@ -483,6 +485,45 @@ Exceptions:
 
 The warning specified in the YAML file should be a short form of the warning message uniquely identifying. The value is used to match warning messages when performing automated testings and ignore them.  
 
+### Samples submission guidelines
+
+Sample data is needed when troubleshooting parser issues and for ensuring future updates to the parser conform to older samples. The samples you submit should include any event variant that the parser supports. Make sure that the sample events include all possible event types, event formats and variations such as events representing successful and failed activity. Also make sure that variations in value formats are represented. For example, if a hostname can be represented as an FQDN or a simple hostname, the sample events should include both formats.
+
+To submit the event samples, use the following steps:
+
+- In the `Logs` screen, run a query that will extract from the source table only the events selected by the parser. For example, for the [Infoblox DNS parser](https://github.com/Azure/Azure-Sentinel/blob/master/Parsers/ASimDns/Parsers/ASimDnsInfobloxNIOS.yaml), use the following query:
+
+``` KQL
+    Syslog
+    | where ProcessName == "named"
+```
+
+- Export the results using the **Export to CSV** option to a file named `<EventVendor>_<EventProduct>_<EventSchema>_IngestedLogs.csv`, Where `EventProduct`, `EventProduct`, and `EventSchema` are the values assigned by the parser to those fields.
+
+- In the `Logs` screen, run a query that will output the schema or the parser input table. For example, for the same Infoblox DNS parser, the query is:
+
+``` KQL
+    Syslog
+    | getschema
+```
+
+- Export the results using the **Export to CSV** option to a file named `<TableName>_schema.csv`, where `TableName` is the name of source table the parser uses.
+
+- Include both files in your PR in the folder `/Sample Data/ASIM`. If the file already exists, add your GitHub handle to the name, for example: `<EventVendor>_<EventProduct>_<EventSchema>_SchemaTest_<GitHubHanlde>.csv`
+
+### Test results submission guidelines
+
+Test results are important to verify the correctness of the parser and understand any reported exception. 
+
+To submit your test results, use the following steps:
+
+- Run the parser tests and described in the [testings](#test-parsers) section.
+
+- and export the tests results using the **Export to CSV** option to files named `<EventVendor>_<EventProduct>_<EventSchema>_SchemaTest.csv` and `<EventVendor>_<EventProduct>_<EventSchema>_DataTest.csv` respectively.
+
+- Include both files in your PR in the folder `/Parsers/ASim<schema>/Tests`.
+
+
 ## Next steps
 
 This article discusses developing ASIM parsers.

articles/sentinel/normalization-parsers-list.md

@@ -36,6 +36,7 @@ To use ASIM authentication parsers, deploy the parsers from the [Microsoft Senti
     - reported by Microsoft 365 Defender for Endpoint, collected using the Microsoft 365 Defender connector.
 - **Linux sign-ins** 
     - reported by Microsoft 365 Defender for Endpoint, collected using the Microsoft 365 Defender connector.
+    - `su`, `sudu`, and `sshd` activity reported using Syslog.
     - reported by Microsoft Defender to IoT Endpoint.
 - **Azure Active Directory sign-ins**, collected using the Azure Active Directory connector. Separate parsers are provided for regular, Non-Interactive, Managed Identities and Service Principles Sign-ins.
 - **AWS sign-ins**, collected using the AWS CloudTrail connector.
@@ -135,9 +136,10 @@ ASIM Web Session parsers are available in every workspace. Microsoft Sentinel pr
 
 | **Source** | **Notes** | **Parser** | 
 | --- | --------------------------- | ------------------------------ | 
+| **Palo Alto PanOS threat logs** | Collected using CEF. | `_Im_WebSession_PaloAltoCEF` |
 | **Squid Proxy** | | `_Im_WebSession_SquidProxyVxx` |
 | **Vectra AI Streams** | Supports the [pack](normalization-about-parsers.md#the-pack-parameter) parameter. | `_Im_WebSession_VectraAIVxx`  |
-| **Zscaler ZIA** | Collected using CEF | `_Im_WebSessionZscalerZIAVxx` |
+| **Zscaler ZIA** | Collected using CEF. | `_Im_WebSessionZscalerZIAVxx` |
 
 Deploy the workspace deployed parsers version from the [Microsoft Sentinel GitHub repository](https://aka.ms/DeployASIM).
 

articles/sentinel/normalization-schema-web.md

@@ -157,7 +157,7 @@ The following are additional fields that are specific to web sessions:
 
 | Field | Class | Type | Description |
 | --- | --- | --- | --- |
-| <a name="url"></a>**Url** | Mandatory | String | The HTTP request URL, including parameters. For `HTTPSession` events, the URL should include the schema and server parts. For `WebServerSession` and for `ApiRequest` the URL would typically not include the schema and server, which can be found in the `NetworkApplicationProtocol` and `DstFQDN` fields respectively. <br><br>Example: `https://contoso.com/fo/?k=v&amp;q=u#f` |
+| <a name="url"></a>**Url** | Mandatory | String | The HTTP request URL, including parameters. For `HTTPSession` events, the URL may include the schema and should include the server name. For `WebServerSession` and for `ApiRequest` the URL would typically not include the schema and server, which can be found in the `NetworkApplicationProtocol` and `DstFQDN` fields respectively. <br><br>Example: `https://contoso.com/fo/?k=v&amp;q=u#f` |
 | **UrlCategory** | Optional | String | The defined grouping of a URL or the domain part of the URL. The category is commonly provided by web security gateways and is based on the content of the site the URL points to.<br><br>Example: search engines, adult, news, advertising, and parked domains. |
 | **UrlOriginal** | Optional | String | The original value of the URL, when the URL was modified by the reporting device and both values are provided. |
 | **HttpVersion** | Optional | String | The HTTP Request Version.<br><br>Example: `2.0` |