Merge pull request #225478 from oshezaf/asim/add-conditional-classes

asim/add-conditional-classes

Author: JamesJBarnett

.openpublishing.redirection.json

@@ -16074,6 +16074,11 @@
             "redirect_url": "/azure/sentinel/normalization-schema-web",
             "redirect_document_id": true
         },
+        {
+            "source_path_from_root": "/articles/sentinel/network-normalization-schema.md",
+            "redirect_url": "/azure/sentinel/normalization-schema-network",
+            "redirect_document_id": true
+        },
         {
             "source_path_from_root": "/articles/sentinel/dns-normalization-schema.md",
             "redirect_url": "/azure/sentinel/normalization-schema-dns",

articles/sentinel/TOC.yml

@@ -568,7 +568,7 @@
         - name: ASIM file event schema
           href: normalization-schema-file-event.md
         - name: ASIM network session schema
-          href: network-normalization-schema.md
+          href: normalization-schema-network.md
         - name: ASIM process event schema
           href: process-events-normalization-schema.md
         - name: ASIM registry event schema

articles/sentinel/index.yml

@@ -96,7 +96,7 @@ landingContent:
           - text: CEF log field mapping
             url: cef-name-mapping.md
           - text: Network normalization schema
-            url: ./network-normalization-schema.md
+            url: ./normalization-schema-network.md
 
  # Card (optional)
   - title: Kusto Query Language in Microsoft Sentinel

articles/sentinel/normalization-about-parsers.md

@@ -64,7 +64,7 @@ Each schema has a standard set of filtering parameters documented in the relevan
 - [Audit Event](normalization-schema-audit.md)
 - [Authentication](authentication-normalization-schema.md)
 - [DNS](normalization-schema-dns.md#filtering-parser-parameters)
-- [Network Session](network-normalization-schema.md#filtering-parser-parameters)
+- [Network Session](normalization-schema-network.md#filtering-parser-parameters)
 - [Web Session](normalization-schema-web.md#filtering-parser-parameters)
 
 Every schema that supports filtering parameters supports at least the `starttime` and `endtime` parameters and using them is often critical for optimizing performance.

articles/sentinel/normalization-about-schemas.md

@@ -41,10 +41,10 @@ The following concepts help to understand the schema reference documents and ext
 |---------|---------|
 |**Field names**     |   At the core of each schema are its field names. Field names belong to the following groups: <br><br>- Fields common to all schemas. <br>- Fields specific to a schema. <br>-	Fields that represent entities, such as users, which take part in the schema. Fields that represent entities [are similar across schemas](#entities). <br><br>When sources have fields that aren't presented in the documented schema, they're normalized to maintain consistency. If the extra fields represent an entity, they'll be normalized based on the entity field guidelines. Otherwise, the schemas strive to keep consistency across all schemas.<br><br> For example, while DNS server activity logs don't provide user information, DNS activity logs from an endpoint might include user information, which can be normalized according to the user entity guidelines.      |
 |[**Field types**](#logical-types)     |  Each schema field has a type. The Log Analytics workspace has a limited set of data types. For this reason, Microsoft Sentinel uses a logical type for many schema fields, which Log Analytics doesn't enforce but is required for schema compatibility. Logical field types ensure that both values and field names are consistent across sources.  <br><br>For more information, see [Logical types](#logical-types).     |
-|**Field class**     | Fields might have several classes, which define when the fields should be implemented by a parser: <br><br>-	**Mandatory** fields must appear in every parser. If your source doesn't provide information for this value, or the data can't be otherwise added, it won't support most content items that reference the normalized schema.<br>-	**Recommended** fields should be normalized if available. However, they might not be available in every source. Any content item that references that normalized schema should take availability into account. <br>-	**Optional** fields, if available, can be normalized or left in their original form. Typically, a minimal parser wouldn't normalize them for performance reasons.    |
+|**Field class**     | Fields might have several classes, which define when the fields should be implemented by a parser: <br><br> - **Mandatory** fields must appear in every parser. If your source doesn't provide information for this value, or the data can't be otherwise added, it won't support most content items that reference the normalized schema.<br> -	**Recommended** fields should be normalized if available. However, they might not be available in every source. Any content item that references that normalized schema should take availability into account.<br> - **Optional** fields, if available, can be normalized or left in their original form. Typically, a minimal parser wouldn't normalize them for performance reasons.<br> - **Conditional** fields are mandatory if the field they follow is populated. Conditional fields are typically used to describe the value in another field. For example, the common field [DvcIdType](normalization-common-fields.md#dvcidtype) describes the value int the common field [DvcId](normalization-common-fields.md#dvcid) and is therefore mandatory if the latter is populated.<br>- **Alias** is a special type of a conditional field, and is mandatory if the aliased field is populated.   |
 |[**Common fields**](normalization-common-fields.md) | Some fields are common to all ASIM schemas. Each schema might add guidelines for using some of the common fields in the context of the specific schema. For example, permitted values for the **EventType** field might vary per schema, as might the value of the **EventSchemaVersion** field. |
-|**Entities**     | Events evolve around entities, such as users, hosts, processes, or files. Each entity might require several fields to describe it. For example, a host might have a name and an IP address. <br><br>A single record might include multiple entities of the same type, such as both a source and destination host. <br><br>ASIM defines how to describe entities consistently, and entities allow for extending the schemas. <br><br>For example, while the Network Session schema doesn't include process information, some event sources do provide process information that can be added. For more information, see [Entities](#entities). |
-|**Aliases**     |  In some cases, different users expect a field to have different names. For example, in DNS terminology, you might expect a field named `query`, while more generally, it holds a domain name. Aliases solve this issue of ambiguity by allowing multiple names for a specified value. The alias class would be the same as the field that it aliases.<br><br>Log Analytics doesn't support aliasing. To implement aliases parsers, create a copy of the original value by using the `extend` operator.        |
+|**Entities**| Events evolve around entities, such as users, hosts, processes, or files. Each entity might require several fields to describe it. For example, a host might have a name and an IP address. <br><br>A single record might include multiple entities of the same type, such as both a source and destination host. <br><br>ASIM defines how to describe entities consistently, and entities allow for extending the schemas. <br><br>For example, while the Network Session schema doesn't include process information, some event sources do provide process information that can be added. For more information, see [Entities](#entities). |
+|**Aliases**| Aliases allow multiple names for a specified value. In some cases, different users expect a field to have different names. For example, in DNS terminology, you might expect a field named [DnsQuery](normalization-schema-dns.md#query), while more generally, it holds a domain name. The alias [Domain](normalization-schema-dns.md#domain) helps the user by allowing the use of both names. <br><br>In some cases, an alias can have the value of one of several fields, depending on which values are available in the event. For example, the [Dvc](normalization-common-fields.md#dvc) alias, aliases either the [DvcFQDN](normalization-common-fields.md#dvcfqdn), [DvcId](normalization-common-fields.md#dvcid), [DvcHostname](normalization-common-fields.md#dvchostname), or [DvcIpAddr](normalization-common-fields.md#dvcipaddr) , or [Event Product](normalization-common-fields.md#eventproduct) fields.<br><br>[Native normalized tables](normalization-ingest-time.md#ingest-time-parsing) do not include aliases, as those would imply duplicate data storage. Instead the [stub parsers](normalization-ingest-time.md#combining-ingest-time-and-query-time-normalization) add the aliases. To implement aliases in parsers, create a copy of the original value by using the `extend` operator.        |
 
 
 ## Logical types

articles/sentinel/normalization-common-fields.md

@@ -64,15 +64,15 @@ The role of the device fields is different for different schemas and event types
 
 | Field               | Class       | Type       |  Description        |
 |---------------------|-------------|------------|--------------------|
-| <a name="dvc"></a>**Dvc** | Mandatory       | String     | A unique identifier of the device on which the event occurred or which reported the event, depending on the schema. <br><br>This field might alias the [DvcFQDN](#dvcfqdn), [DvcId](#dvcid), [DvcHostname](#dvchostname), or [DvcIpAddr](#dvcipaddr) fields. For cloud sources, for which there is no apparent device, use the same value as the [Event Product](#eventproduct) field.            |
+| <a name="dvc"></a>**Dvc** | Alias       | String     | A unique identifier of the device on which the event occurred or which reported the event, depending on the schema. <br><br>This field might alias the [DvcFQDN](#dvcfqdn), [DvcId](#dvcid), [DvcHostname](#dvchostname), or [DvcIpAddr](#dvcipaddr) fields. For cloud sources, for which there is no apparent device, use the same value as the [Event Product](#eventproduct) field.            |
 | <a name ="dvcipaddr"></a>**DvcIpAddr**           | Recommended | IP address | The IP address of the device on which the event occurred or which reported the event, depending on the schema. <br><br>Example: `45.21.42.12`    |
 | <a name ="dvchostname"></a>**DvcHostname**         | Recommended | Hostname   | The hostname of the device on which the event occurred or which reported the event, depending on the schema. <br><br>Example: `ContosoDc`               |
 | <a name="dvcdomain"></a>**DvcDomain** | Recommended | String | The domain of the device on which the event occurred or which reported the event, depending on the schema.<br><br>Example: `Contoso` |
-| <a name="dvcdomaintype"></a>**DvcDomainType** | Recommended | Enumerated | The type of  [DvcDomain](#dvcdomain). For a list of allowed values and further information refer to [DomainType](normalization-about-schemas.md#domaintype).<br><br>**Note**: This field is required if the [DvcDomain](#dvcdomain) field is used. |
+| <a name="dvcdomaintype"></a>**DvcDomainType** | Conditional | Enumerated | The type of  [DvcDomain](#dvcdomain). For a list of allowed values and further information refer to [DomainType](normalization-about-schemas.md#domaintype).<br><br>**Note**: This field is required if the [DvcDomain](#dvcdomain) field is used. |
 | <a name="dvcfqdn"></a>**DvcFQDN** | Optional | String | The hostname of the device on which the event occurred or which reported the event, depending on the schema. <br><br> Example: `Contoso\DESKTOP-1282V4D`<br><br>**Note**: This field supports both traditional FQDN format and Windows domain\hostname format. The  [DvcDomainType](#dvcdomaintype) field reflects the format used.  |
 | <a name = "dvcdescription"></a>**DvcDescription** | Optional | String | A descriptive text associated with the device. For example: `Primary Domain Controller`. |
 | <a name ="dvcid"></a>**DvcId**               | Optional    | String     | The unique ID of the device on which the event occurred or which reported the event, depending on the schema. <br><br>Example: `41502da5-21b7-48ec-81c9-baeea8d7d669`   |
-| <a name="dvcidtype"></a>**DvcIdType** | Optional | Enumerated | The type of [DvcId](#dvcid). For a list of allowed values and further information refer to [DvcIdType](#dvcidtype).<br>- `MDEid`<br><br>If multiple IDs are available, use the first one from the list, and store the others by using the field names **DvcAzureResourceId** and **DvcMDEid**, respectively.<br><br>**Note**: This field is required if the [DvcId](#dvcid) field is used. |
+| <a name="dvcidtype"></a>**DvcIdType** | Conditional | Enumerated | The type of [DvcId](#dvcid). For a list of allowed values and further information refer to [DvcIdType](#dvcidtype).<br>- `MDEid`<br><br>If multiple IDs are available, use the first one from the list, and store the others by using the field names **DvcAzureResourceId** and **DvcMDEid**, respectively.<br><br>**Note**: This field is required if the [DvcId](#dvcid) field is used. |
 | <a name="dvcmacaddr"></a>**DvcMacAddr**          | Optional    | MAC        |   The MAC address of the device on which the event occurred or which reported the event.  <br><br>Example: `00:1B:44:11:3A:B7`       |
 | <a name="dvczone"></a>**DvcZone** | Optional | String | The network on which the event occurred or which reported the event, depending on the schema. The zone is defined by the reporting device.<br><br>Example: `Dmz` |
 | <a name="dvcos"></a>**DvcOs**               | Optional    | String     |         The operating system running on the device on which the event occurred or which reported the event.    <br><br>Example: `Windows`    |

articles/sentinel/normalization-ingest-time.md

@@ -15,7 +15,7 @@ As discussion in the [ASIM overview](normalization.md), Microsoft Sentinel uses
 
 To use query time normalization, use the [query time unifying parsers](normalization-about-parsers.md#unifying-parsers), such as `_Im_Dns` in your queries. Normalizing using query time parsing has several advantages:
 
-- **Preserving the original format**: Query time normalization does't require the data to be modified, thus preserving the original data format sent by the source.
+- **Preserving the original format**: Query time normalization doesn't require the data to be modified, thus preserving the original data format sent by the source.
 - **Avoiding potential duplicate storage**: Since the normalized data is only a view of the original data, there is no need to store both original and normalized data. 
 - **Easier development**: Since query time parsers present a view of the data and don't modify the data, they are easy to develop. Developing, testing and fixing a parser can all be done on existing data. Moreover, parsers can be fixed when an issue is discovered and the fix will apply to existing data.
 
@@ -29,7 +29,7 @@ Normalized data can be stored in Microsoft Sentinel's native normalized tables,
 
 Currently, ASIM supports the following native normalized tables as a destination for ingest time normalization:
 - [**ASimDnsActivityLogs**](/azure/azure-monitor/reference/tables/asimdnsactivitylogs) for the [DNS](normalization-schema-dns.md) schema.
-- [**ASimNetworkSessionLogs**](/azure/azure-monitor/reference/tables/asimnetworksessionlogs) for the [Network Session](network-normalization-schema.md) schema 
+- [**ASimNetworkSessionLogs**](/azure/azure-monitor/reference/tables/asimnetworksessionlogs) for the [Network Session](normalization-schema-network.md) schema 
 
 The advantage of native normalized tables is that they are included by default in the ASIM unifying parsers. Custom normalized tables can be included in the unifying parsers, as discussed in [Manage Parsers](normalization-manage-parsers.md).
 
@@ -39,7 +39,7 @@ Queries should always use the [query time unifying parsers](normalization-about-
 
 The stub parser is a query time parser that uses as input the normalized table. Since the normalized table doesn't require parsing, the stub parser is efficient.
 
-The stub parser presents to queries a view that adds to the ASIM native table:
+The stub parser presents a view to the calling query that adds to the ASIM native table:
 
 - **Aliases** - in order to not waste storage on repeating values, aliases are not stored in ASIM native tables and are added at query time by the stub parsers.
 - **Constant values** - Like aliases, and for the same reason, ASIM normalized tables also don't store constant values  such as [EventSchema](normalization-common-fields.md#eventschema). The stub parser adds those fields. ASIM normalized table is shared by many sources, and ingest time parsers can change their output version. Therefore, fields such as [EventProduct](normalization-common-fields.md#eventproduct), [EventVendor](normalization-common-fields.md#eventvendor), and [EventSchemaVersion](normalization-common-fields.md#eventschemaversion) are not constant and are not added by the stub parser.