You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/sentinel/normalization-about-schemas.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -41,7 +41,7 @@ The following concepts help to understand the schema reference documents and ext
41
41
|---------|---------|
42
42
|**Field names** | At the core of each schema are its field names. Field names belong to the following groups: <br><br>- Fields common to all schemas. <br>- Fields specific to a schema. <br>- Fields that represent entities, such as users, which take part in the schema. Fields that represent entities [are similar across schemas](#entities). <br><br>When sources have fields that aren't presented in the documented schema, they're normalized to maintain consistency. If the extra fields represent an entity, they'll be normalized based on the entity field guidelines. Otherwise, the schemas strive to keep consistency across all schemas.<br><br> For example, while DNS server activity logs don't provide user information, DNS activity logs from an endpoint might include user information, which can be normalized according to the user entity guidelines. |
43
43
|[**Field types**](#logical-types) | Each schema field has a type. The Log Analytics workspace has a limited set of data types. For this reason, Microsoft Sentinel uses a logical type for many schema fields, which Log Analytics doesn't enforce but is required for schema compatibility. Logical field types ensure that both values and field names are consistent across sources. <br><br>For more information, see [Logical types](#logical-types). |
44
-
|**Field class** | Fields might have several classes, which define when the fields should be implemented by a parser: <br><br> - **Mandatory** fields must appear in every parser. If your source doesn't provide information for this value, or the data can't be otherwise added, it won't support most content items that reference the normalized schema.<br> - **Recommended** fields should be normalized if available. However, they might not be available in every source. Any content item that references that normalized schema should take availability into account.<br> - **Optional** fields, if available, can be normalized or left in their original form. Typically, a minimal parser wouldn't normalize them for performance reasons.<br> - **Conditional** fields are mandatory if the field they follow is populated. Conditional fields are typically used to describe the value in another field. For example, the common field [DvcIdType](normalization-common-fields.md#dvcidtype) describes the value int the common field [DvcId](normalization-common-fields.md#dvcid) and is therefore mandatory if the latter is populated.<br>- **Alias** is a special type of a conditional field, and is mandatory is the aliased field is populated. |
44
+
|**Field class** | Fields might have several classes, which define when the fields should be implemented by a parser: <br><br> - **Mandatory** fields must appear in every parser. If your source doesn't provide information for this value, or the data can't be otherwise added, it won't support most content items that reference the normalized schema.<br> - **Recommended** fields should be normalized if available. However, they might not be available in every source. Any content item that references that normalized schema should take availability into account.<br> - **Optional** fields, if available, can be normalized or left in their original form. Typically, a minimal parser wouldn't normalize them for performance reasons.<br> - **Conditional** fields are mandatory if the field they follow is populated. Conditional fields are typically used to describe the value in another field. For example, the common field [DvcIdType](normalization-common-fields.md#dvcidtype) describes the value int the common field [DvcId](normalization-common-fields.md#dvcid) and is therefore mandatory if the latter is populated.<br>- **Alias** is a special type of a conditional field, and is mandatory if the aliased field is populated. |
45
45
|[**Common fields**](normalization-common-fields.md) | Some fields are common to all ASIM schemas. Each schema might add guidelines for using some of the common fields in the context of the specific schema. For example, permitted values for the **EventType** field might vary per schema, as might the value of the **EventSchemaVersion** field. |
46
46
|**Entities**| Events evolve around entities, such as users, hosts, processes, or files. Each entity might require several fields to describe it. For example, a host might have a name and an IP address. <br><br>A single record might include multiple entities of the same type, such as both a source and destination host. <br><br>ASIM defines how to describe entities consistently, and entities allow for extending the schemas. <br><br>For example, while the Network Session schema doesn't include process information, some event sources do provide process information that can be added. For more information, see [Entities](#entities). |
47
47
|**Aliases**| Aliases allow multiple names for a specified value. In some cases, different users expect a field to have different names. For example, in DNS terminology, you might expect a field named [DnsQuery](normalization-schema-dns.md#query), while more generally, it holds a domain name. The alias [Domain](normalization-schema-dns.md#domain) helps the user by allowing the use of both names. <br><br>In some cases, an alias can have the value of one of several fields, depending on which values are available in the event. For example, the [Dvc](normalization-common-fields.md#dvc) alias, aliases either the [DvcFQDN](normalization-common-fields.md#dvcfqdn), [DvcId](normalization-common-fields.md#dvcid), [DvcHostname](normalization-common-fields.md#dvchostname), or [DvcIpAddr](normalization-common-fields.md#dvcipaddr) , or [Event Product](normalization-common-fields.md#eventproduct) fields.<br><br>[Native normalized tables](normalization-ingest-time.md#ingest-time-parsing) do not include aliases, as those would imply duplicate data storage. Instead the [stub parsers](normalization-ingest-time.md#combining-ingest-time-and-query-time-normalization) add the aliases. To implement aliases in parsers, create a copy of the original value by using the `extend` operator. |
Copy file name to clipboardExpand all lines: articles/sentinel/normalization-ingest-time.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,7 +15,7 @@ As discussion in the [ASIM overview](normalization.md), Microsoft Sentinel uses
15
15
16
16
To use query time normalization, use the [query time unifying parsers](normalization-about-parsers.md#unifying-parsers), such as `_Im_Dns` in your queries. Normalizing using query time parsing has several advantages:
17
17
18
-
-**Preserving the original format**: Query time normalization does't require the data to be modified, thus preserving the original data format sent by the source.
18
+
-**Preserving the original format**: Query time normalization doesn't require the data to be modified, thus preserving the original data format sent by the source.
19
19
-**Avoiding potential duplicate storage**: Since the normalized data is only a view of the original data, there is no need to store both original and normalized data.
20
20
-**Easier development**: Since query time parsers present a view of the data and don't modify the data, they are easy to develop. Developing, testing and fixing a parser can all be done on existing data. Moreover, parsers can be fixed when an issue is discovered and the fix will apply to existing data.
21
21
@@ -39,7 +39,7 @@ Queries should always use the [query time unifying parsers](normalization-about-
39
39
40
40
The stub parser is a query time parser that uses as input the normalized table. Since the normalized table doesn't require parsing, the stub parser is efficient.
41
41
42
-
The stub parser presents to queries a view that adds to the ASIM native table:
42
+
The stub parser presents a view to the calling query that adds to the ASIM native table:
43
43
44
44
-**Aliases** - in order to not waste storage on repeating values, aliases are not stored in ASIM native tables and are added at query time by the stub parsers.
45
45
-**Constant values** - Like aliases, and for the same reason, ASIM normalized tables also don't store constant values such as [EventSchema](normalization-common-fields.md#eventschema). The stub parser adds those fields. ASIM normalized table is shared by many sources, and ingest time parsers can change their output version. Therefore, fields such as [EventProduct](normalization-common-fields.md#eventproduct), [EventVendor](normalization-common-fields.md#eventvendor), and [EventSchemaVersion](normalization-common-fields.md#eventschemaversion) are not constant and are not added by the stub parser.
Copy file name to clipboardExpand all lines: articles/sentinel/normalization-schema-web.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -157,7 +157,7 @@ The following are additional fields that are specific to web sessions:
157
157
158
158
| Field | Class | Type | Description |
159
159
| --- | --- | --- | --- |
160
-
| <aname="url"></a>**Url**| Mandatory | String | The HTTP request URL, including parameters. For `HTTPSession` events, the URL should include the schema and server parts. For `WebServerSession` and for `ApiRequest` the URL would typlicaly not include the schema and server, which can be found in the `NetworkApplicationProtocol` and `DstFQDN` fields respectively. <br><br>Example: `https://contoso.com/fo/?k=v&q=u#f`|
160
+
| <aname="url"></a>**Url**| Mandatory | String | The HTTP request URL, including parameters. For `HTTPSession` events, the URL should include the schema and server parts. For `WebServerSession` and for `ApiRequest` the URL would typically not include the schema and server, which can be found in the `NetworkApplicationProtocol` and `DstFQDN` fields respectively. <br><br>Example: `https://contoso.com/fo/?k=v&q=u#f`|
161
161
|**UrlCategory**| Optional | String | The defined grouping of a URL or the domain part of the URL. The category is commonly provided by web security gateways and is based on the content of the site the URL points to.<br><br>Example: search engines, adult, news, advertising, and parked domains. |
162
162
|**UrlOriginal**| Optional | String | The original value of the URL, when the URL was modified by the reporting device and both values are provided. |
163
163
|**HttpVersion**| Optional | String | The HTTP Request Version.<br><br>Example: `2.0`|
0 commit comments