You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/sentinel/monitor-data-connector-health.md
+22-11Lines changed: 22 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -52,7 +52,7 @@ There are three tabbed sections in this workbook:
52
52
53
53
- The **Overview** tab shows the general status of data ingestion in the selected workspace: volume measures, EPS rates, and time last log received.
54
54
55
-
- The **Data collection anomalies** tab will help you to detect anomalies in the data collection process, by table and data source. Each tab presents anomalies for a particular table (the **General** tab includes a collection of tables). The anomalies are calculated using the **series_decompose_anomalies()** function that returns an **anomaly score**. [Learn more about this function](/azure/data-explorer/kusto/query/series-decompose-anomaliesfunction?WT.mc_id=Portal-fx). Set the following parameters for the function to evaluate:
55
+
- The **Data collection anomalies** tab will help you to detect anomalies in the data collection process, by table and data source. Each tab presents anomalies for a particular table (the **General** tab includes a collection of tables). The anomalies are calculated using the **series_decompose_anomalies()** function that returns an **anomaly score**. [Learn more about this function](/kusto/query/series-decompose-anomalies-function?view=microsoft-sentinel&preserve-view=true&WT.mc_id=Portal-fx). Set the following parameters for the function to evaluate:
56
56
57
57
-**AnomaliesTimeRange**: This time picker applies only to the data collection anomalies view.
58
58
-**SampleInterval**: The time interval in which data is sampled in the given time range. The anomaly score is calculated only on the last interval's data.
@@ -117,45 +117,56 @@ SentinelHealth
117
117
**Detect connectors with changes from fail to success state**:
118
118
119
119
```kusto
120
-
let lastestStatus = SentinelHealth
120
+
let latestStatus = SentinelHealth
121
121
| where TimeGenerated > ago(12h)
122
122
| where OperationName == 'Data fetch status change'
123
123
| where Status in ('Success', 'Failure')
124
124
| project TimeGenerated, SentinelResourceName, SentinelResourceId, LastStatus = Status
125
125
| summarize TimeGenerated = arg_max(TimeGenerated,*) by SentinelResourceName, SentinelResourceId;
126
-
let nextToLastestStatus = SentinelHealth
126
+
let nextTolatestStatus = SentinelHealth
127
127
| where TimeGenerated > ago(12h)
128
128
| where OperationName == 'Data fetch status change'
129
129
| where Status in ('Success', 'Failure')
130
-
| join kind = leftanti (lastestStatus) on SentinelResourceName, SentinelResourceId, TimeGenerated
130
+
| join kind = leftanti (latestStatus) on SentinelResourceName, SentinelResourceId, TimeGenerated
131
131
| project TimeGenerated, SentinelResourceName, SentinelResourceId, NextToLastStatus = Status
132
132
| summarize TimeGenerated = arg_max(TimeGenerated,*) by SentinelResourceName, SentinelResourceId;
133
-
lastestStatus
134
-
| join kind=inner (nextToLastestStatus) on SentinelResourceName, SentinelResourceId
133
+
latestStatus
134
+
| join kind=inner (nextTolatestStatus) on SentinelResourceName, SentinelResourceId
135
135
| where NextToLastStatus == 'Failure' and LastStatus == 'Success'
136
136
```
137
137
138
138
**Detect connectors with changes from success to fail state**:
139
139
140
140
```kusto
141
-
let lastestStatus = SentinelHealth
141
+
let latestStatus = SentinelHealth
142
142
| where TimeGenerated > ago(12h)
143
143
| where OperationName == 'Data fetch status change'
144
144
| where Status in ('Success', 'Failure')
145
145
| project TimeGenerated, SentinelResourceName, SentinelResourceId, LastStatus = Status
146
146
| summarize TimeGenerated = arg_max(TimeGenerated,*) by SentinelResourceName, SentinelResourceId;
147
-
let nextToLastestStatus = SentinelHealth
147
+
let nextTolatestStatus = SentinelHealth
148
148
| where TimeGenerated > ago(12h)
149
149
| where OperationName == 'Data fetch status change'
150
150
| where Status in ('Success', 'Failure')
151
-
| join kind = leftanti (lastestStatus) on SentinelResourceName, SentinelResourceId, TimeGenerated
151
+
| join kind = leftanti (latestStatus) on SentinelResourceName, SentinelResourceId, TimeGenerated
152
152
| project TimeGenerated, SentinelResourceName, SentinelResourceId, NextToLastStatus = Status
153
153
| summarize TimeGenerated = arg_max(TimeGenerated,*) by SentinelResourceName, SentinelResourceId;
154
-
lastestStatus
155
-
| join kind=inner (nextToLastestStatus) on SentinelResourceName, SentinelResourceId
154
+
latestStatus
155
+
| join kind=inner (nextTolatestStatus) on SentinelResourceName, SentinelResourceId
156
156
| where NextToLastStatus == 'Success' and LastStatus == 'Failure'
157
157
```
158
158
159
+
See more information on the following items used in the preceding examples, in the Kusto documentation:
### Configure alerts and automated actions for health issues
160
171
161
172
While you can use the Microsoft Sentinel [analytics rules](automate-incident-handling-with-automation-rules.md) to configure automation in Microsoft Sentinel logs, if you want to be notified and take immediate action for health drifts in your data connectors, we recommend that you use [Azure Monitor alert rules](/azure/azure-monitor/alerts/alerts-overview).
Copy file name to clipboardExpand all lines: articles/sentinel/normalization-develop-parsers.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -410,7 +410,7 @@ Handle the results as follows:
410
410
| Missing recommended alias [\<Field\>] aliasing existing column [\<Field\>] | Add the alias to your parser |
411
411
| Missing optional alias [\<Field\>] aliasing existing column [\<Field\>] | Add the alias to your parser |
412
412
| Missing mandatory alias [\<Field\>] aliasing missing column [\<Field\>] | This error accompanies a similar error for the aliased field. Correct the aliased field error and add this alias to your parser. |
413
-
| Type mismatch for field [\<Field\>]. It is currently [\<Type\>] and should be [\<Type\>] | Make sure that the type of normalized field is correct, usually by using a [conversion function](/azure/data-explorer/kusto/query/scalarfunctions#conversion-functions) such as `tostring`. |
413
+
| Type mismatch for field [\<Field\>]. It is currently [\<Type\>] and should be [\<Type\>] | Make sure that the type of normalized field is correct, usually by using a [conversion function](/kusto/query/scalar-functions?view=microsoft-sentinel&preserve-view=true#conversion-functions) such as `tostring`. |
414
414
415
415
| Info | Action |
416
416
| ----- | ------ |
@@ -447,7 +447,7 @@ Handle the results as follows:
447
447
448
448
| Message | Action |
449
449
| ------- | ------ |
450
-
| **(0) Error: type mismatch for column [\<Field\>]. It is currently [\<Type\>] and should be [\<Type\>]** | Make sure that the type of normalized field is correct, usually by using a [conversion function](/azure/data-explorer/kusto/query/scalarfunctions#conversion-functions) such as `tostring`. |
450
+
| **(0) Error: type mismatch for column [\<Field\>]. It is currently [\<Type\>] and should be [\<Type\>]** | Make sure that the type of normalized field is correct, usually by using a [conversion function](/kusto/query/scalar-functions?view=microsoft-sentinel&preserve-view=true#conversion-functions) such as `tostring`. |
451
451
| **(0) Error: Invalid value(s) (up to 10 listed) for field [\<Field\>] of type [\<Logical Type\>]** | Make sure that the parser maps the correct source field to the output field. If mapped correctly, update the parser to transform the source value to the correct type, value or format. Refer to the [list of logical types](normalization-about-schemas.md#logical-types) for more information on the correct values and formats for each logical type. <br><br>Note that the testing tool lists only a sample of 10 invalid values. |
452
452
| **(1) Warning: Empty value in mandatory field [\<Field\>]** | Mandatory fields should be populated, not just defined. Check whether the field can be populated from other sources for records for which the current source is empty. |
453
453
| **(2) Info: Empty value in recommended field [\<Field\>]** | Recommended fields should usually be populated. Check whether the field can be populated from other sources for records for which the current source is empty. |
Copy file name to clipboardExpand all lines: articles/sentinel/normalization-parsers-overview.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,7 +13,7 @@ ms.author: ofshezaf
13
13
14
14
# The Advanced Security Information Model (ASIM) parsers (Public preview)
15
15
16
-
In Microsoft Sentinel, parsing and [normalizing](normalization.md) happen at query time. Parsers are built as [KQL user-defined functions](/azure/data-explorer/kusto/query/functions/user-defined-functions) that transform data in existing tables, such as **CommonSecurityLog**, custom logs tables, or Syslog, into the normalized schema.
16
+
In Microsoft Sentinel, parsing and [normalizing](normalization.md) happen at query time. Parsers are built as [KQL user-defined functions](/kusto/query/functions/user-defined-functions?view=microsoft-sentinel&preserve-view=true) that transform data in existing tables, such as **CommonSecurityLog**, custom logs tables, or Syslog, into the normalized schema.
17
17
18
18
Users [use Advanced Security Information Model (ASIM) parsers](normalization-about-parsers.md) instead of table names in their queries to view data in a normalized format, and to include all data relevant to the schema in your query.
Copy file name to clipboardExpand all lines: articles/sentinel/normalization-schema-audit.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -76,7 +76,7 @@ The following filtering parameters are available:
76
76
|**object_has_any**| dynamic/string | Filter only events in which [Object](#object) field includes any of the terms provided. |
77
77
|**newvalue_has_any**| dynamic/string | Filter only events in which [NewValue](#object) field includes any of the terms provided. |
78
78
79
-
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`
79
+
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`
80
80
81
81
For example, to filter only audit events with the terms `install` or `update` in their [Operation](#operation) field, from the last day , use:
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`.
70
+
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`.
Copy file name to clipboardExpand all lines: articles/sentinel/normalization-schema-dns.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -108,7 +108,7 @@ let torProxies=dynamic(["tor2web.org", "tor2web.com", "torlink.co"]);
108
108
_Im_Dns (domain_has_any = torProxies)
109
109
```
110
110
111
-
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`
111
+
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`
Copy file name to clipboardExpand all lines: articles/sentinel/normalization-schema-network.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -71,7 +71,7 @@ The following filtering parameters are available:
71
71
|**dvcaction**| dynamic/string | Filter only network sessions for which the [Device Action field](#dvcaction) is any of the values listed. |
72
72
|**eventresult**| String | Filter only network sessions with a specific **EventResult** value. |
73
73
74
-
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`
74
+
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`
75
75
76
76
For example, to filter only network sessions for a specified list of domain names, use:
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`.
84
+
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`.
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals.md). For example: `dynamic(['192.168.','10.'])`.
78
+
> To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`.
Copy file name to clipboardExpand all lines: articles/sentinel/normalization-schema-web.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -86,7 +86,7 @@ The following filtering parameters are available:
86
86
|**eventresultdetails_in**| dynamic | Filter only web sessions for which the HTTP status code, stored in the [EventResultDetails](#eventresultdetails) field, is any of the values listed. |
87
87
|**eventresult**| string | Filter only network sessions with a specific **EventResult** value. |
88
88
89
-
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-literals). For example: `dynamic(['192.168.','10.'])`
89
+
Some parameter can accept both list of values of type `dynamic` or a single string value. To pass a literal list to parameters that expect a dynamic value, explicitly use a [dynamic literal](/kusto/query/scalar-data-types/dynamic?view=microsoft-sentinel&preserve-view=true#dynamic-literals). For example: `dynamic(['192.168.','10.'])`
90
90
91
91
For example, to filter only Web sessions for a specified list of domain names, use:
Copy file name to clipboardExpand all lines: articles/sentinel/normalization.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -80,7 +80,7 @@ For more information, see [ASIM schemas](normalization-about-schemas.md).
80
80
81
81
### Query time parsers
82
82
83
-
ASIM uses query time parsers to map existing data to the normalized schemas using [KQL functions](/azure/data-explorer/kusto/query/functions/user-defined-functions). Many ASIM parsers are available out of the box with Microsoft Sentinel. More parsers, and versions of the built-in parsers that can be modified can be deployed from the [Microsoft Sentinel GitHub repository](https://aka.ms/AzSentinelASim).
83
+
ASIM uses query time parsers to map existing data to the normalized schemas using [KQL functions](/kusto/query/functions/user-defined-functions?view=microsoft-sentinel&preserve-view=true). Many ASIM parsers are available out of the box with Microsoft Sentinel. More parsers, and versions of the built-in parsers that can be modified can be deployed from the [Microsoft Sentinel GitHub repository](https://aka.ms/AzSentinelASim).
84
84
85
85
For more information, see [ASIM parsers](normalization-parsers-overview.md).
0 commit comments