You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/cse/troubleshoot/troubleshoot-rules.md
+46-42Lines changed: 46 additions & 42 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,13 +12,13 @@ This article provides guidance for administrators to diagnose, troubleshoot, and
12
12
Rules in Cloud SIEM serve to provide security detection signals from log data that has previously been ingested, parsed, normalized, and enriched into Cloud SIEM records. Signals produced by rules form the core alerting capability in Cloud SIEM and are themselves correlated into insights by the entities elevated from the records that generate them.
13
13
14
14
Rule issues can manifest in several ways:
15
-
* No signals
16
-
<br/>A rule that should be generating signals is not, or a "Test Rule Expression" returns results but no signals are generated.
17
-
* Too many signals
15
+
*[No signals](#no-signals-being-generated)
16
+
<br/>A rule that should be generating signals is not, or clicking the **Test Rule Expression** button on a rule returns results but no signals are generated. This can also result when the rule status is either in failed or degraded and is either permanently or temporarily disabled due to rule logic or throughput issues.
17
+
*[Too many signals](#too-many-signals-are-generated)
18
18
<br/>False positive signals or signals that aren’t useful are being triggered or are unsuppressed.
19
-
*Rule degradations or failures
20
-
<br/>The rule status is either in failed or degraded and is either permanently or temporarily disabled due to rule logic or throughput issues.
21
-
* Signal time out of sync
19
+
*[Unexpected signal suppression](#signal-suppression-problems)
20
+
<br/>Signals are not suppressed that you want to suppress, or signals are suppressed that you don't want to be suppressed.
21
+
*[Signal time out of sync](#signal-time-offset)
22
22
<br/>Signals being generated out of sync with the actual event time either in the past or future.
23
23
24
24
## Common rule components
@@ -31,7 +31,7 @@ The [expression on a rule](/docs/cse/rules/about-cse-rules/#about-rule-expressio
31
31
32
32
<img src={useBaseUrl('img/cse/troubleshoot-tuning-expression.png')} alt="Example match expression on a rule" style={{border: '1px solid gray'}} width="500" />
33
33
34
-
Rules with a [tuning expression](/docs/cse/rules/rule-tuning-expressions/) added will require whatever additional criteria is present in the tuning expression as well as the base expression to match as they are evaluated as a single expression at processing time.
34
+
Rules with a [tuning expression](/docs/cse/rules/rule-tuning-expressions/) added will require whatever additional criteria is present in the tuning expression as well as the base expression to match because they are evaluated as a single expression at processing time.
35
35
36
36
### Entity selector
37
37
@@ -48,12 +48,12 @@ The signal name on a rule controls what the name of any generated signal will be
48
48
### Suppression
49
49
50
50
[Signals can be suppressed](/docs/cse/records-signals-entities-insights/about-signal-suppression) a number of ways, most commonly:
51
-
*[Redundant signal suppression](/docs/cse/get-started-with-cloud-siem/insight-generation-process/#redundant-signal-suppression)
51
+
* Redundant signal suppression
52
52
<br/>A signal with the same name and entity was previously triggered. The default window for redundant signal suppression is 72 hours, but can be overridden per-rule or globally.
<br/>For IP address entities contained in a defined network block that has been suppressed.
55
+
* Network block suppression
56
+
<br/>IP address entities contained in a defined network block are suppressed.
57
57
58
58
Suppressed signals are collapsed within the first signal generated that was unsuppressed and are not considered in insights.
59
59
@@ -63,49 +63,55 @@ Following is an example of a per-rule entity suppression override:
63
63
64
64
## Rule type dependent components
65
65
66
-
Each rule type other than the match rule will perform some form of aggregation as they consider multiple records in order to trigger a signal.
66
+
Each rule type other than the match rule will perform some form of aggregation as they consider multiple records in order to trigger a signal. (For rules triggered by multiple records, each of the defined criteria for each record must match in addition to the common rule components.)
67
67
68
68
### Aggregation components
69
69
70
-
***Grouped by**
71
-
<br/>Groups together values from one or more parsed fields from a record. Any entity field selected will be included implicitly in rules using a "grouped by". It appears on these rule types: threshold, aggregation, chain, outlier, and first seen.
72
-
<br/><br/>"Grouped by" fields appear differently for different rule types:
70
+
#### Grouped by
71
+
72
+
This component groups together values from one or more parsed fields from a record. Any entity field selected will be included implicitly in rules. "Grouped by" appears on these rule types: threshold, aggregation, chain, outlier, and first seen.
73
+
74
+
"Grouped by" fields appear differently for different rule types:
73
75
* In first seen rules, it appears as "**has a new value for the field(s)**", and for per-entity baselines as "**for the following (entities)**".
74
76
* In outlier rules, it appears within the **Outlier Model Configuration** section within "**of the record field**".
75
77
* In threshold rules, the **Group by one or more fields** option appears after clicking **Show advanced**.
76
78
* In chain and aggregation rules, it appears as "**grouped by**" after the rule expressions.
77
-
***Count**
78
-
<br/>A basic count threshold of logs that meet the other rule criteria. It appears on these rule types: threshold, aggregation (as a math function), chain, and outlier.
79
-
***Count Distinct**
80
-
<br/>A count of unique values found in the field or fields (distinct groups of values). It appears on rule types: threshold, aggregation, and outlier.
81
-
***Other math functions**
82
-
<br/>These math functions are available:
79
+
80
+
#### Count
81
+
82
+
This component is a basic count threshold of logs that meet the other rule criteria. It appears on these rule types: threshold, aggregation (as a math function), chain, and outlier.
83
+
84
+
#### Count distinct
85
+
86
+
This component is a count of unique values found in the field or fields (distinct groups of values). It appears on rule types: threshold, aggregation, and outlier.
87
+
88
+
#### Other math functions
89
+
90
+
These math functions are available:
83
91
`avg`, `first`, `last`, `max`, `min`, and `sum`. (`first` and `last` are available in the aggregation rule type.)
84
-
***Math expression**
85
-
<br/>Used in conjunction with the built-in math functions to do additional calculations on values from the evaluated records. Available in outlier and aggregation rule types
86
92
87
-
For rules triggered by multiple records, each of the defined criteria for each record must match in addition to the common rule components.
93
+
#### Math expression
94
+
95
+
Math expressions are used in conjunction with the built-in math functions to do additional calculations on values from the evaluated records. Available in outlier and aggregation rule types.
88
96
89
97
### Anomaly detection rule components
90
98
91
99
#### First seen rules
92
100
93
-
Uses a baseline of activity particular to an organization or specific entity and signals on the occurrence of a new activity. Baseline settings:
94
-
* Baseline types
101
+
[First seen rules](/docs/cse/rules/write-first-seen-rule/) use a baseline of activity particular to an organization or specific entity, and generate signals on the occurrence of a new activity. Following are the baseline settings:
102
+
* Baseline types:
95
103
* Global
96
-
<br/>Tracks activity across an organization. Signals on the first observance of a given activity after the baseline is built.
104
+
<br/>Tracks activity across an organization. It generates signals on the first observance of a given activity after the baseline is built.
97
105
* Per-entity
98
-
<br/>Tracks activity per-entity in an environment. Signals on the first observance of a given activity for an entity after the baseline is built.
106
+
<br/>Tracks activity per-entity in an environment. It generates signals on the first observance of a given activity for an entity after the baseline is built.
99
107
* Retention period
100
108
<br/>Rolling window for retaining records that pertain to the activities tracked in the baseline.
101
109
* Baseline period
102
110
<br/>Amount of time to look-back for activity before allowing a signal to be triggered.
103
111
104
-
For more on first seen rules, see [Write a First Seen Rule](/docs/cse/rules/write-first-seen-rule/).
105
-
106
112
#### Outlier rules
107
113
108
-
Uses a baseline of activity and signals on an outlier in the tracked activity. Baseline settings:
114
+
[Outlier rules](/docs/cse/rules/write-outlier-rule/) use a baseline of activity, and generates signals on an outlier in the tracked activity. Following are the baseline settings:
109
115
* Bucketing of behavior either on an hourly or daily granularity.
110
116
* Retention period
111
117
<br/>Rolling window for retaining records that pertain to the activities tracked in the baseline.
@@ -116,19 +122,17 @@ Uses a baseline of activity and signals on an outlier in the tracked activity. B
116
122
117
123
Rules that require a baseline are dependent on the baseline criteria being met before triggering a signal. For first seen rules, this means that an activity has not been observed in the baseline period and subsequently is observed. For outlier rules, this means that the tracked activity has exceeded both the floor value and model sensitivity threshold.
118
124
119
-
For more on outlier rules, see [Write an Outlier Rule](/docs/cse/rules/write-outlier-rule/).
120
-
121
-
## Test Rule Expression
125
+
## Test rule expression
122
126
123
127
### Limitations
124
128
125
-
Each rule expression text box (chain rules have two or more expressions) has a corresponding **Test Rule Expression** button that allows the user constructing a rule to validate that the expression logic matches records. While this is a helpful component in rule validation, it only tests the criteria contained in the expression using a Sumo Logic log search against Cloud SIEM records, other rule criteria required for signal generation are not validated when running the test expression, such as the entity, groupings, or baselines. As such, records returned when running an expression test, even for simple match rules, are not a reliable indicator that the rule will trigger signals when active.
129
+
Each rule expression text box (chain rules have two or more expressions) has a corresponding **Test Rule Expression** button that allows the user constructing a rule to validate that the expression logic matches records. While this is a helpful component in rule validation, it only tests the criteria contained in the expression using a Sumo Logic log search against Cloud SIEM records. Other rule criteria required for signal generation are not validated when running the test expression, such as the entity, groupings, or baselines. As such, records returned when running an expression test, even for simple match rules, are not a reliable indicator that the rule will trigger signals when active.
126
130
127
131
:::note
128
-
There are limited circumstances where test rule expression can return misleading results due to minor differences in Cloud SIEM expression syntax and log search syntax. See [Cloud SIEM Rules Syntax](/docs/cse/rules/cse-rules-syntax/) for more information.
132
+
There are limited circumstances where testing the rule expression can return misleading results due to minor differences in Cloud SIEM expression syntax and log search syntax. See [Cloud SIEM Rules Syntax](/docs/cse/rules/cse-rules-syntax/) for more information.
129
133
:::
130
134
131
-
### Using Test Rule Expression in rule troubleshooting
135
+
### Using the Test Rule Expression feature in rule troubleshooting
132
136
133
137
Despite limitations, the Test Rule Expression feature is still a crucial component in rule troubleshooting. Records returned validate that the tested expression logic is valid (see above note on syntax differences). While records returned won’t necessarily meet other rule criteria, they can be inspected for other rule signal criteria:
134
138
* Entity field is present.
@@ -162,11 +166,11 @@ Despite limitations, the Test Rule Expression feature is still a crucial compone
162
166
1. If yes, check other criteria outside of entity, match expression, and grouping criteria.
163
167
1. If no:
164
168
* Validate that matching records contain the grouping criteria.
165
-
* Review contributing mappers are mapping the fields, or if using "fields" for parsed but not mapped attributes, that the parser is creating parsing the desired field for the record. For specific steps on mapping and parsing issues, see [Troubleshoot Mappers](/docs/cse/troubleshoot/troubleshoot-mappers/) and [Troubleshoot Parsers](/docs/cse/troubleshoot/troubleshoot-parsers/).
169
+
* Review contributing mappers are mapping the fields, or if using "fields" for parsed but not mapped attributes, that the parser is parsing the desired field for the record. For specific steps on mapping and parsing issues, see [Troubleshoot Mappers](/docs/cse/troubleshoot/troubleshoot-mappers/) and [Troubleshoot Parsers](/docs/cse/troubleshoot/troubleshoot-parsers/).
166
170
1. Are other aggregation criteria being met (counts, math functions, distinct values, etc.)?
167
171
1. If yes, see [Escalate rule issues](#escalate-rule-issues).
168
172
1. If no, interrogate math functions, count, or count distinct criteria are met. [Searches against `sec_record` indexes](#records-index) are best suited for evaluating these criteria.
169
-
1. If all rule criteria are met and signals are not being generated see [Escalating rule issues](#escalating-rule-issues).
173
+
1. If all rule criteria are met and signals are not being generated see [Escalate rule issues](#escalate-rule-issues).
170
174
171
175
### Too many signals are generated
172
176
@@ -181,7 +185,7 @@ Despite limitations, the Test Rule Expression feature is still a crucial compone
181
185
1. Insufficient filtering criteria.
182
186
<br/>Expression matches targeting specific patterns, such as in a command line, have potential to come from myriad data sources that aren’t pertinent to the desired detection. Add additional filtering criteria that may not be critical to the detection, but reduces the volume of records considered for a detection without compromising the fidelity of the rule. Command lines only relevant to Windows could include additional criteria to narrow the detection to Windows sources. The inverse would be true for detections that would not be relevant to Windows, such as commands only pertinent to Unix based systems.
183
187
1. Erroneous entity selection.
184
-
<br/>There is no limit on the number of entities that can be selected for a rule, but selecting too many entities may result in undesired signal volume if the entity for which a signal is created is not pertinent to the intent of the detection. For example, RDP logon from localhost will only contain localhost (`127.0.0.1`) IP addresses which will not be useful in the detection
188
+
<br/>There is no limit on the number of entities that can be selected for a rule, but selecting too many entities may result in undesired signal volume if the entity for which a signal is created is not pertinent to the intent of the detection. For example, RDP logon from localhost will only contain localhost (`127.0.0.1`) IP addresses which will not be useful in the detection.
185
189
186
190
### Signal suppression problems
187
191
@@ -200,7 +204,7 @@ If signals are suppressed that you don't want to be suppressed:
200
204
201
205
### Signal time offset
202
206
203
-
When signal or signals timestamps differ from their constituent records, record timestamps typically account for time differences in signals. Records will default to UTC for timestamp if there is missing, misconfigured, or unparsed timezone information.
207
+
When signal timestamps differ from their constituent records, record timestamps typically account for time differences in signals. Records will default to UTC for timestamp if there is missing, misconfigured, or unparsed timezone information.
204
208
205
209
Time issues can be introduced at several places in the processing pipeline:
206
210
* Collection
@@ -271,6 +275,6 @@ Use this search query to return the top entities creating signals to identify pa
271
275
272
276
### Records index
273
277
274
-
Using the records index `_index=sec_record*` is much more flexible than using **Test Rule Expression** within the rules editor, as you can quickly enter search terms and get records that can match the desired rule criteria to preemptively identify patterns in records that could be useful in a rule or tuning expression.
278
+
Using the records index `_index=sec_record*` is much more flexible than using **Test Rule Expression**button within the rules editor, as you can quickly enter search terms and get records that can match the desired rule criteria to preemptively identify patterns in records that could be useful in a rule or tuning expression.
275
279
276
280
For more information about searching using the records index, see [Searching for Cloud SIEM Records in Sumo Logic](/docs/cse/records-signals-entities-insights/search-cse-records-in-sumo/).
0 commit comments