Skip to content

Commit 39a6335

Browse files
committed
Continued work 5
1 parent 95911dc commit 39a6335

13 files changed

+37
-26
lines changed

articles/sentinel/monitor-data-connector-health.md

Lines changed: 22 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ There are three tabbed sections in this workbook:
5252

5353
- The **Overview** tab shows the general status of data ingestion in the selected workspace: volume measures, EPS rates, and time last log received.
5454

55-
- The **Data collection anomalies** tab will help you to detect anomalies in the data collection process, by table and data source. Each tab presents anomalies for a particular table (the **General** tab includes a collection of tables). The anomalies are calculated using the **series_decompose_anomalies()** function that returns an **anomaly score**. [Learn more about this function](/azure/data-explorer/kusto/query/series-decompose-anomaliesfunction?WT.mc_id=Portal-fx). Set the following parameters for the function to evaluate:
55+
- The **Data collection anomalies** tab will help you to detect anomalies in the data collection process, by table and data source. Each tab presents anomalies for a particular table (the **General** tab includes a collection of tables). The anomalies are calculated using the **series_decompose_anomalies()** function that returns an **anomaly score**. [Learn more about this function](/kusto/query/series-decompose-anomalies-function?view=microsoft-sentinel&preserve-view=true&WT.mc_id=Portal-fx). Set the following parameters for the function to evaluate:
5656

5757
- **AnomaliesTimeRange**: This time picker applies only to the data collection anomalies view.
5858
- **SampleInterval**: The time interval in which data is sampled in the given time range. The anomaly score is calculated only on the last interval's data.
@@ -117,45 +117,56 @@ SentinelHealth
117117
**Detect connectors with changes from fail to success state**:
118118

119119
```kusto
120-
let lastestStatus = SentinelHealth
120+
let latestStatus = SentinelHealth
121121
| where TimeGenerated > ago(12h)
122122
| where OperationName == 'Data fetch status change'
123123
| where Status in ('Success', 'Failure')
124124
| project TimeGenerated, SentinelResourceName, SentinelResourceId, LastStatus = Status
125125
| summarize TimeGenerated = arg_max(TimeGenerated,*) by SentinelResourceName, SentinelResourceId;
126-
let nextToLastestStatus = SentinelHealth
126+
let nextTolatestStatus = SentinelHealth
127127
| where TimeGenerated > ago(12h)
128128
| where OperationName == 'Data fetch status change'
129129
| where Status in ('Success', 'Failure')
130-
| join kind = leftanti (lastestStatus) on SentinelResourceName, SentinelResourceId, TimeGenerated
130+
| join kind = leftanti (latestStatus) on SentinelResourceName, SentinelResourceId, TimeGenerated
131131
| project TimeGenerated, SentinelResourceName, SentinelResourceId, NextToLastStatus = Status
132132
| summarize TimeGenerated = arg_max(TimeGenerated,*) by SentinelResourceName, SentinelResourceId;
133-
lastestStatus
134-
| join kind=inner (nextToLastestStatus) on SentinelResourceName, SentinelResourceId
133+
latestStatus
134+
| join kind=inner (nextTolatestStatus) on SentinelResourceName, SentinelResourceId
135135
| where NextToLastStatus == 'Failure' and LastStatus == 'Success'
136136
```
137137

138138
**Detect connectors with changes from success to fail state**:
139139

140140
```kusto
141-
let lastestStatus = SentinelHealth
141+
let latestStatus = SentinelHealth
142142
| where TimeGenerated > ago(12h)
143143
| where OperationName == 'Data fetch status change'
144144
| where Status in ('Success', 'Failure')
145145
| project TimeGenerated, SentinelResourceName, SentinelResourceId, LastStatus = Status
146146
| summarize TimeGenerated = arg_max(TimeGenerated,*) by SentinelResourceName, SentinelResourceId;
147-
let nextToLastestStatus = SentinelHealth
147+
let nextTolatestStatus = SentinelHealth
148148
| where TimeGenerated > ago(12h)
149149
| where OperationName == 'Data fetch status change'
150150
| where Status in ('Success', 'Failure')
151-
| join kind = leftanti (lastestStatus) on SentinelResourceName, SentinelResourceId, TimeGenerated
151+
| join kind = leftanti (latestStatus) on SentinelResourceName, SentinelResourceId, TimeGenerated
152152
| project TimeGenerated, SentinelResourceName, SentinelResourceId, NextToLastStatus = Status
153153
| summarize TimeGenerated = arg_max(TimeGenerated,*) by SentinelResourceName, SentinelResourceId;
154-
lastestStatus
155-
| join kind=inner (nextToLastestStatus) on SentinelResourceName, SentinelResourceId
154+
latestStatus
155+
| join kind=inner (nextTolatestStatus) on SentinelResourceName, SentinelResourceId
156156
| where NextToLastStatus == 'Success' and LastStatus == 'Failure'
157157
```
158158

159+
See more information on the following items used in the preceding examples, in the Kusto documentation:
160+
- [***let*** statement](/kusto/query/let-statement?view=microsoft-sentinel&preserve-view=true)
161+
- [***where*** operator](/kusto/query/where-operator?view=microsoft-sentinel&preserve-view=true)
162+
- [***project*** operator](/kusto/query/project-operator?view=microsoft-sentinel&preserve-view=true)
163+
- [***summarize*** operator](/kusto/query/summarize-operator?view=microsoft-sentinel&preserve-view=true)
164+
- [***join*** operator](/kusto/query/join-operator?view=microsoft-sentinel&preserve-view=true)
165+
- [***ago()*** function](/kusto/query/ago-function?view=microsoft-sentinel&preserve-view=true)
166+
- [***arg_max()*** aggregation function](/kusto/query/arg-max-aggregation-function?view=microsoft-sentinel&preserve-view=true)
167+
168+
[!INCLUDE [kusto-reference-general-no-alert](includes/kusto-reference-general-no-alert.md)]
169+
159170
### Configure alerts and automated actions for health issues
160171

161172
While you can use the Microsoft Sentinel [analytics rules](automate-incident-handling-with-automation-rules.md) to configure automation in Microsoft Sentinel logs, if you want to be notified and take immediate action for health drifts in your data connectors, we recommend that you use [Azure Monitor alert rules](/azure/azure-monitor/alerts/alerts-overview).

articles/sentinel/normalization-develop-parsers.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -410,7 +410,7 @@ Handle the results as follows:
410410
| Missing recommended alias [\<Field\>] aliasing existing column [\<Field\>] | Add the alias to your parser |
411411
| Missing optional alias [\<Field\>] aliasing existing column [\<Field\>] | Add the alias to your parser |
412412
| Missing mandatory alias [\<Field\>] aliasing missing column [\<Field\>] | This error accompanies a similar error for the aliased field. Correct the aliased field error and add this alias to your parser. |
413-
| Type mismatch for field [\<Field\>]. It is currently [\<Type\>] and should be [\<Type\>] | Make sure that the type of normalized field is correct, usually by using a [conversion function](/azure/data-explorer/kusto/query/scalarfunctions#conversion-functions) such as `tostring`. |
413+
| Type mismatch for field [\<Field\>]. It is currently [\<Type\>] and should be [\<Type\>] | Make sure that the type of normalized field is correct, usually by using a [conversion function](/kusto/query/scalar-functions?view=microsoft-sentinel&preserve-view=true#conversion-functions) such as `tostring`. |
414414

415415
| Info | Action |
416416
| ----- | ------ |
@@ -447,7 +447,7 @@ Handle the results as follows:
447447

448448
| Message | Action |
449449
| ------- | ------ |
450-
| **(0) Error: type mismatch for column [\<Field\>]. It is currently [\<Type\>] and should be [\<Type\>]** | Make sure that the type of normalized field is correct, usually by using a [conversion function](/azure/data-explorer/kusto/query/scalarfunctions#conversion-functions) such as `tostring`. |
450+
| **(0) Error: type mismatch for column [\<Field\>]. It is currently [\<Type\>] and should be [\<Type\>]** | Make sure that the type of normalized field is correct, usually by using a [conversion function](/kusto/query/scalar-functions?view=microsoft-sentinel&preserve-view=true#conversion-functions) such as `tostring`. |
451451
| **(0) Error: Invalid value(s) (up to 10 listed) for field [\<Field\>] of type [\<Logical Type\>]** | Make sure that the parser maps the correct source field to the output field. If mapped correctly, update the parser to transform the source value to the correct type, value or format. Refer to the [list of logical types](normalization-about-schemas.md#logical-types) for more information on the correct values and formats for each logical type. <br><br>Note that the testing tool lists only a sample of 10 invalid values. |
452452
| **(1) Warning: Empty value in mandatory field [\<Field\>]** | Mandatory fields should be populated, not just defined. Check whether the field can be populated from other sources for records for which the current source is empty. |
453453
| **(2) Info: Empty value in recommended field [\<Field\>]** | Recommended fields should usually be populated. Check whether the field can be populated from other sources for records for which the current source is empty. |

articles/sentinel/normalization-parsers-overview.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ ms.author: ofshezaf
1313

1414
# The Advanced Security Information Model (ASIM) parsers (Public preview)
1515

16-
In Microsoft Sentinel, parsing and [normalizing](normalization.md) happen at query time. Parsers are built as [KQL user-defined functions](/azure/data-explorer/kusto/query/functions/user-defined-functions) that transform data in existing tables, such as **CommonSecurityLog**, custom logs tables, or Syslog, into the normalized schema.
16+
In Microsoft Sentinel, parsing and [normalizing](normalization.md) happen at query time. Parsers are built as [KQL user-defined functions](/kusto/query/functions/user-defined-functions?view=microsoft-sentinel&preserve-view=true) that transform data in existing tables, such as **CommonSecurityLog**, custom logs tables, or Syslog, into the normalized schema.
1717

1818
Users [use Advanced Security Information Model (ASIM) parsers](normalization-about-parsers.md) instead of table names in their queries to view data in a normalized format, and to include all data relevant to the schema in your query.
1919

articles/sentinel/normalization-schema-audit.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ The following filtering parameters are available:
7676
| **object_has_any** | dynamic/string | Filter only events in which [Object](#object) field includes any of the terms provided. |
7777
| **newvalue_has_any** | dynamic/string | Filter only events in which [NewValue](#object) field includes any of the terms provided. |
7878

79-
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`
79+
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`
8080

8181
For example, to filter only audit events with the terms `install` or `update` in their [Operation](#operation) field, from the last day , use:
8282

articles/sentinel/normalization-schema-authentication.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ imAuthentication (targetusername_has = 'johndoe', starttime = ago(1d), endtime=n
6767

6868

6969
> [!TIP]
70-
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`.
70+
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`.
7171
>
7272
7373

articles/sentinel/normalization-schema-dns.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ let torProxies=dynamic(["tor2web.org", "tor2web.com", "torlink.co"]);
108108
_Im_Dns (domain_has_any = torProxies)
109109
```
110110

111-
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`
111+
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`
112112

113113
## Normalized content
114114

articles/sentinel/normalization-schema-network.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ The following filtering parameters are available:
7171
| **dvcaction** | dynamic/string | Filter only network sessions for which the [Device Action field](#dvcaction) is any of the values listed. |
7272
| **eventresult** | String | Filter only network sessions with a specific **EventResult** value. |
7373

74-
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`
74+
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`
7575

7676
For example, to filter only network sessions for a specified list of domain names, use:
7777

@@ -81,7 +81,7 @@ _Im_NetworkSession (hostname_has_any = torProxies)
8181
```
8282

8383
> [!TIP]
84-
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`.
84+
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`.
8585
>
8686
8787
## Normalized content

articles/sentinel/normalization-schema-process-event.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ imProcessCreate (targetusername_has = 'johndoe', starttime = ago(1d), endtime=no
7575
```
7676

7777
> [!TIP]
78-
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`.
78+
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`.
7979
>
8080
8181
## Normalized content

articles/sentinel/normalization-schema-web.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ The following filtering parameters are available:
8686
| **eventresultdetails_in** | dynamic | Filter only web sessions for which the HTTP status code, stored in the [EventResultDetails](#eventresultdetails) field, is any of the values listed. |
8787
| **eventresult** | string | Filter only network sessions with a specific **EventResult** value. |
8888

89-
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals). For example: `dynamic(['192.168.','10.'])`
89+
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`
9090

9191
For example, to filter only Web sessions for a specified list of domain names, use:
9292

articles/sentinel/normalization.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ For more information, see [ASIM schemas](normalization-about-schemas.md).
8080

8181
### Query time parsers
8282

83-
ASIM uses query time parsers to map existing data to the normalized schemas using [KQL functions](/azure/data-explorer/kusto/query/functions/user-defined-functions). Many ASIM parsers are available out of the box with Microsoft Sentinel. More parsers, and versions of the built-in parsers that can be modified can be deployed from the [Microsoft Sentinel GitHub repository](https://aka.ms/AzSentinelASim).
83+
ASIM uses query time parsers to map existing data to the normalized schemas using [KQL functions](/kusto/query/functions/user-defined-functions?view=microsoft-sentinel&preserve-view=true). Many ASIM parsers are available out of the box with Microsoft Sentinel. More parsers, and versions of the built-in parsers that can be modified can be deployed from the [Microsoft Sentinel GitHub repository](https://aka.ms/AzSentinelASim).
8484

8585
For more information, see [ASIM parsers](normalization-parsers-overview.md).
8686

0 commit comments

Comments
 (0)