Skip to content

Commit 45dce70

Browse files
committed
Final cleanup
1 parent 6324851 commit 45dce70

File tree

1 file changed

+46
-42
lines changed

1 file changed

+46
-42
lines changed

docs/cse/troubleshoot/troubleshoot-rules.md

Lines changed: 46 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,13 @@ This article provides guidance for administrators to diagnose, troubleshoot, and
1212
Rules in Cloud SIEM serve to provide security detection signals from log data that has previously been ingested, parsed, normalized, and enriched into Cloud SIEM records. Signals produced by rules form the core alerting capability in Cloud SIEM and are themselves correlated into insights by the entities elevated from the records that generate them.
1313

1414
Rule issues can manifest in several ways:
15-
* No signals
16-
<br/>A rule that should be generating signals is not, or a "Test Rule Expression" returns results but no signals are generated.
17-
* Too many signals
15+
* [No signals](#no-signals-being-generated)
16+
<br/>A rule that should be generating signals is not, or clicking the **Test Rule Expression** button on a rule returns results but no signals are generated. This can also result when the rule status is either in failed or degraded and is either permanently or temporarily disabled due to rule logic or throughput issues.
17+
* [Too many signals](#too-many-signals-are-generated)
1818
<br/>False positive signals or signals that aren’t useful are being triggered or are unsuppressed.
19-
* Rule degradations or failures
20-
<br/>The rule status is either in failed or degraded and is either permanently or temporarily disabled due to rule logic or throughput issues.
21-
* Signal time out of sync
19+
* [Unexpected signal suppression](#signal-suppression-problems)
20+
<br/>Signals are not suppressed that you want to suppress, or signals are suppressed that you don't want to be suppressed.
21+
* [Signal time out of sync](#signal-time-offset)
2222
<br/>Signals being generated out of sync with the actual event time either in the past or future.
2323

2424
## Common rule components
@@ -31,7 +31,7 @@ The [expression on a rule](/docs/cse/rules/about-cse-rules/#about-rule-expressio
3131

3232
<img src={useBaseUrl('img/cse/troubleshoot-tuning-expression.png')} alt="Example match expression on a rule" style={{border: '1px solid gray'}} width="500" />
3333

34-
Rules with a [tuning expression](/docs/cse/rules/rule-tuning-expressions/) added will require whatever additional criteria is present in the tuning expression as well as the base expression to match as they are evaluated as a single expression at processing time.
34+
Rules with a [tuning expression](/docs/cse/rules/rule-tuning-expressions/) added will require whatever additional criteria is present in the tuning expression as well as the base expression to match because they are evaluated as a single expression at processing time.
3535

3636
### Entity selector
3737

@@ -48,12 +48,12 @@ The signal name on a rule controls what the name of any generated signal will be
4848
### Suppression
4949

5050
[Signals can be suppressed](/docs/cse/records-signals-entities-insights/about-signal-suppression) a number of ways, most commonly:
51-
* [Redundant signal suppression](/docs/cse/get-started-with-cloud-siem/insight-generation-process/#redundant-signal-suppression)
51+
* Redundant signal suppression
5252
<br/>A signal with the same name and entity was previously triggered. The default window for redundant signal suppression is 72 hours, but can be overridden per-rule or globally.
53-
* [Entity suppression](/docs/cse/records-signals-entities-insights/about-signal-suppression/#suppress-by-entity)
53+
* Entity suppression
5454
<br/>The entity for which a signal was triggered has been suppressed.
55-
* [Network block suppression](/docs/cse/records-signals-entities-insights/about-signal-suppression/#suppress-by-network-block)
56-
<br/>For IP address entities contained in a defined network block that has been suppressed.
55+
* Network block suppression
56+
<br/>IP address entities contained in a defined network block are suppressed.
5757

5858
Suppressed signals are collapsed within the first signal generated that was unsuppressed and are not considered in insights.
5959

@@ -63,49 +63,55 @@ Following is an example of a per-rule entity suppression override:
6363

6464
## Rule type dependent components
6565

66-
Each rule type other than the match rule will perform some form of aggregation as they consider multiple records in order to trigger a signal.
66+
Each rule type other than the match rule will perform some form of aggregation as they consider multiple records in order to trigger a signal. (For rules triggered by multiple records, each of the defined criteria for each record must match in addition to the common rule components.)
6767

6868
### Aggregation components
6969

70-
* **Grouped by**
71-
<br/>Groups together values from one or more parsed fields from a record. Any entity field selected will be included implicitly in rules using a "grouped by". It appears on these rule types: threshold, aggregation, chain, outlier, and first seen.
72-
<br/><br/>"Grouped by" fields appear differently for different rule types:
70+
#### Grouped by
71+
72+
This component groups together values from one or more parsed fields from a record. Any entity field selected will be included implicitly in rules. "Grouped by" appears on these rule types: threshold, aggregation, chain, outlier, and first seen.
73+
74+
"Grouped by" fields appear differently for different rule types:
7375
* In first seen rules, it appears as "**has a new value for the field(s)**", and for per-entity baselines as "**for the following (entities)**".
7476
* In outlier rules, it appears within the **Outlier Model Configuration** section within "**of the record field**".
7577
* In threshold rules, the **Group by one or more fields** option appears after clicking **Show advanced**.
7678
* In chain and aggregation rules, it appears as "**grouped by**" after the rule expressions.
77-
* **Count**
78-
<br/>A basic count threshold of logs that meet the other rule criteria. It appears on these rule types: threshold, aggregation (as a math function), chain, and outlier.
79-
* **Count Distinct**
80-
<br/>A count of unique values found in the field or fields (distinct groups of values). It appears on rule types: threshold, aggregation, and outlier.
81-
* **Other math functions**
82-
<br/>These math functions are available:
79+
80+
#### Count
81+
82+
This component is a basic count threshold of logs that meet the other rule criteria. It appears on these rule types: threshold, aggregation (as a math function), chain, and outlier.
83+
84+
#### Count distinct
85+
86+
This component is a count of unique values found in the field or fields (distinct groups of values). It appears on rule types: threshold, aggregation, and outlier.
87+
88+
#### Other math functions
89+
90+
These math functions are available:
8391
`avg`, `first`, `last`, `max`, `min`, and `sum`. (`first` and `last` are available in the aggregation rule type.)
84-
* **Math expression**
85-
<br/>Used in conjunction with the built-in math functions to do additional calculations on values from the evaluated records. Available in outlier and aggregation rule types
8692

87-
For rules triggered by multiple records, each of the defined criteria for each record must match in addition to the common rule components.
93+
#### Math expression
94+
95+
Math expressions are used in conjunction with the built-in math functions to do additional calculations on values from the evaluated records. Available in outlier and aggregation rule types.
8896

8997
### Anomaly detection rule components
9098

9199
#### First seen rules
92100

93-
Uses a baseline of activity particular to an organization or specific entity and signals on the occurrence of a new activity. Baseline settings:
94-
* Baseline types
101+
[First seen rules](/docs/cse/rules/write-first-seen-rule/) use a baseline of activity particular to an organization or specific entity, and generate signals on the occurrence of a new activity. Following are the baseline settings:
102+
* Baseline types:
95103
* Global
96-
<br/>Tracks activity across an organization. Signals on the first observance of a given activity after the baseline is built.
104+
<br/>Tracks activity across an organization. It generates signals on the first observance of a given activity after the baseline is built.
97105
* Per-entity
98-
<br/>Tracks activity per-entity in an environment. Signals on the first observance of a given activity for an entity after the baseline is built.
106+
<br/>Tracks activity per-entity in an environment. It generates signals on the first observance of a given activity for an entity after the baseline is built.
99107
* Retention period
100108
<br/>Rolling window for retaining records that pertain to the activities tracked in the baseline.
101109
* Baseline period
102110
<br/>Amount of time to look-back for activity before allowing a signal to be triggered.
103111

104-
For more on first seen rules, see [Write a First Seen Rule](/docs/cse/rules/write-first-seen-rule/).
105-
106112
#### Outlier rules
107113

108-
Uses a baseline of activity and signals on an outlier in the tracked activity. Baseline settings:
114+
[Outlier rules](/docs/cse/rules/write-outlier-rule/) use a baseline of activity, and generates signals on an outlier in the tracked activity. Following are the baseline settings:
109115
* Bucketing of behavior either on an hourly or daily granularity.
110116
* Retention period
111117
<br/>Rolling window for retaining records that pertain to the activities tracked in the baseline.
@@ -116,19 +122,17 @@ Uses a baseline of activity and signals on an outlier in the tracked activity. B
116122

117123
Rules that require a baseline are dependent on the baseline criteria being met before triggering a signal. For first seen rules, this means that an activity has not been observed in the baseline period and subsequently is observed. For outlier rules, this means that the tracked activity has exceeded both the floor value and model sensitivity threshold.
118124

119-
For more on outlier rules, see [Write an Outlier Rule](/docs/cse/rules/write-outlier-rule/).
120-
121-
## Test Rule Expression
125+
## Test rule expression
122126

123127
### Limitations
124128

125-
Each rule expression text box (chain rules have two or more expressions) has a corresponding **Test Rule Expression** button that allows the user constructing a rule to validate that the expression logic matches records. While this is a helpful component in rule validation, it only tests the criteria contained in the expression using a Sumo Logic log search against Cloud SIEM records, other rule criteria required for signal generation are not validated when running the test expression, such as the entity, groupings, or baselines. As such, records returned when running an expression test, even for simple match rules, are not a reliable indicator that the rule will trigger signals when active.
129+
Each rule expression text box (chain rules have two or more expressions) has a corresponding **Test Rule Expression** button that allows the user constructing a rule to validate that the expression logic matches records. While this is a helpful component in rule validation, it only tests the criteria contained in the expression using a Sumo Logic log search against Cloud SIEM records. Other rule criteria required for signal generation are not validated when running the test expression, such as the entity, groupings, or baselines. As such, records returned when running an expression test, even for simple match rules, are not a reliable indicator that the rule will trigger signals when active.
126130

127131
:::note
128-
There are limited circumstances where test rule expression can return misleading results due to minor differences in Cloud SIEM expression syntax and log search syntax. See [Cloud SIEM Rules Syntax](/docs/cse/rules/cse-rules-syntax/) for more information.
132+
There are limited circumstances where testing the rule expression can return misleading results due to minor differences in Cloud SIEM expression syntax and log search syntax. See [Cloud SIEM Rules Syntax](/docs/cse/rules/cse-rules-syntax/) for more information.
129133
:::
130134

131-
### Using Test Rule Expression in rule troubleshooting
135+
### Using the Test Rule Expression feature in rule troubleshooting
132136

133137
Despite limitations, the Test Rule Expression feature is still a crucial component in rule troubleshooting. Records returned validate that the tested expression logic is valid (see above note on syntax differences). While records returned won’t necessarily meet other rule criteria, they can be inspected for other rule signal criteria:
134138
* Entity field is present.
@@ -162,11 +166,11 @@ Despite limitations, the Test Rule Expression feature is still a crucial compone
162166
1. If yes, check other criteria outside of entity, match expression, and grouping criteria.
163167
1. If no:
164168
* Validate that matching records contain the grouping criteria.
165-
* Review contributing mappers are mapping the fields, or if using "fields" for parsed but not mapped attributes, that the parser is creating parsing the desired field for the record. For specific steps on mapping and parsing issues, see [Troubleshoot Mappers](/docs/cse/troubleshoot/troubleshoot-mappers/) and [Troubleshoot Parsers](/docs/cse/troubleshoot/troubleshoot-parsers/).
169+
* Review contributing mappers are mapping the fields, or if using "fields" for parsed but not mapped attributes, that the parser is parsing the desired field for the record. For specific steps on mapping and parsing issues, see [Troubleshoot Mappers](/docs/cse/troubleshoot/troubleshoot-mappers/) and [Troubleshoot Parsers](/docs/cse/troubleshoot/troubleshoot-parsers/).
166170
1. Are other aggregation criteria being met (counts, math functions, distinct values, etc.)?
167171
1. If yes, see [Escalate rule issues](#escalate-rule-issues).
168172
1. If no, interrogate math functions, count, or count distinct criteria are met. [Searches against `sec_record` indexes](#records-index) are best suited for evaluating these criteria.
169-
1. If all rule criteria are met and signals are not being generated see [Escalating rule issues](#escalating-rule-issues).
173+
1. If all rule criteria are met and signals are not being generated see [Escalate rule issues](#escalate-rule-issues).
170174

171175
### Too many signals are generated
172176

@@ -181,7 +185,7 @@ Despite limitations, the Test Rule Expression feature is still a crucial compone
181185
1. Insufficient filtering criteria.
182186
<br/>Expression matches targeting specific patterns, such as in a command line, have potential to come from myriad data sources that aren’t pertinent to the desired detection. Add additional filtering criteria that may not be critical to the detection, but reduces the volume of records considered for a detection without compromising the fidelity of the rule. Command lines only relevant to Windows could include additional criteria to narrow the detection to Windows sources. The inverse would be true for detections that would not be relevant to Windows, such as commands only pertinent to Unix based systems.
183187
1. Erroneous entity selection.
184-
<br/>There is no limit on the number of entities that can be selected for a rule, but selecting too many entities may result in undesired signal volume if the entity for which a signal is created is not pertinent to the intent of the detection. For example, RDP logon from localhost will only contain localhost (`127.0.0.1`) IP addresses which will not be useful in the detection
188+
<br/>There is no limit on the number of entities that can be selected for a rule, but selecting too many entities may result in undesired signal volume if the entity for which a signal is created is not pertinent to the intent of the detection. For example, RDP logon from localhost will only contain localhost (`127.0.0.1`) IP addresses which will not be useful in the detection.
185189

186190
### Signal suppression problems
187191

@@ -200,7 +204,7 @@ If signals are suppressed that you don't want to be suppressed:
200204

201205
### Signal time offset
202206

203-
When signal or signals timestamps differ from their constituent records, record timestamps typically account for time differences in signals. Records will default to UTC for timestamp if there is missing, misconfigured, or unparsed timezone information.
207+
When signal timestamps differ from their constituent records, record timestamps typically account for time differences in signals. Records will default to UTC for timestamp if there is missing, misconfigured, or unparsed timezone information.
204208

205209
Time issues can be introduced at several places in the processing pipeline:
206210
* Collection
@@ -271,6 +275,6 @@ Use this search query to return the top entities creating signals to identify pa
271275

272276
### Records index
273277

274-
Using the records index `_index=sec_record*` is much more flexible than using **Test Rule Expression** within the rules editor, as you can quickly enter search terms and get records that can match the desired rule criteria to preemptively identify patterns in records that could be useful in a rule or tuning expression.
278+
Using the records index `_index=sec_record*` is much more flexible than using **Test Rule Expression** button within the rules editor, as you can quickly enter search terms and get records that can match the desired rule criteria to preemptively identify patterns in records that could be useful in a rule or tuning expression.
275279

276280
For more information about searching using the records index, see [Searching for Cloud SIEM Records in Sumo Logic](/docs/cse/records-signals-entities-insights/search-cse-records-in-sumo/).

0 commit comments

Comments
 (0)