You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We're happy to announce that now when you create or update a first seen or outlier rule, the baseline starts building immediately using data already in the system. Now you don't have to wait days for a baseline learning period to end before a baseline is built and ready to use. Now baselines are typically generated in minutes, allow you to get value quickly from your first seen and outlier rules.
16
+
17
+
To learn more, see our information about baselines for [first seen rules](/docs/cse/rules/write-outlier-rule/#baselines-for-outlier-rules) and [outlier rules](/docs/cse/rules/write-outlier-rule/#baselines-for-outlier-rules).
Copy file name to clipboardExpand all lines: docs/cse/rules/rules-status.md
+16Lines changed: 16 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -36,6 +36,7 @@ Following are the different kinds of rule status. A rule's status can change dep
36
36
|**Degraded**| The rule encountered a problem during processing and is removed from execution until the problem is resolved. | Click the information button <img src={useBaseUrl('img/cse/rule-status-information-button.png')} alt="Rule status information button" width="20"/> on the **Degraded** label for details. Depending on the information provided, you may need to edit the rule to reduce the chance it will become degraded again later. See [Degraded rules](#degraded-rules) below for more information. |
37
37
|**Disabled**| The rule was manually disabled using the toggle in the UI, or was disabled with the API. | Enable the rule with the toggle in the UI, or enable the rule with the [API](https://api.sumologic.com/docs/sec/#operation/UpdateRuleEnabled). |
38
38
|**Failed**| The rule encountered a problem that resulted in its being automatically disabled. For example, processing the rule caused the system to exceed a rule limit. | Click the information button <img src={useBaseUrl('img/cse/rule-status-information-button.png')} alt="Rule status information button" width="20"/> on the **Failed** label for details about the failure. Depending on the reason provided in the details, you may need to edit the rule to prevent it from failing again in the future. <br/><br/>After addressing the reason for the failure, enable the rule with the toggle in the UI, or enable the rule with the [API](https://api.sumologic.com/docs/sec/#operation/UpdateRuleEnabled). |
39
+
|**Pending Baseline**| The baseline for the [first seen rule](/docs/cse/rules/write-first-seen-rule/#baselines-for-first-seen-rules) or [outlier rule](/docs/cse/rules/write-first-seen-rule/#baselines-for-first-seen-rules) is being generated. | Click the information button <img src={useBaseUrl('img/cse/rule-status-information-button.png')} alt="Rule status information button" width="20"/> on the **Pending Baseline** label for details. If data exists in the system to build the baseline, baseline generation typically takes only minutes to complete, and then the rule's status changes to "Active". However, if there is not enough data in the system, the pending status can last longer. See [Troubleshoot baseline problems](#troubleshoot-baseline-problems) below. |
39
40
40
41
<!-- For DOCS-72 - Rule limits
41
42
| **Warning** | The rule is approaching a rule limit and risks being disabled. | Click the information button <img src={useBaseUrl('img/cse/rule-warning-info-button.png')} alt="Rule warning information button" width="20"/> on the **Warning** label for details about the warning. Depending on the reasons provided in the details, you may need to edit the rule to prevent it from being disabled. |
@@ -55,6 +56,21 @@ Following are some situations when a rule can be become degraded:
55
56
* Rules have a limit on the number of records per second they can evaluate. If there is a value used in the "group by" field that causes the rule to exceed that threshold, Cloud SIEM might display a message like this when you click the information button on the "Degraded" rule status:
56
57
<br/>`The aggregation on the group key '[email protected]' has a record volume exceeding the supported limit, and has been disabled. Consider tuning the rule to exclude records producing this group key.`
57
58
59
+
### Troubleshoot baseline problems
60
+
61
+
Sometimes there may be a problem creating a baseline for a [first seen rule](/docs/cse/rules/write-first-seen-rule/#baselines-for-first-seen-rules) or [outlier rule](/docs/cse/rules/write-outlier-rule/#baselines-for-outlier-rules). In these cases, the rule might enter a Degraded, Failed, or Pending Baseline state. Clicking the information button <img src={useBaseUrl('img/cse/rule-status-information-button.png')} alt="Rule status information button" width="20"/> on the status label may provide enough information to resolve the problem. But if not, you can do additional troubleshooting:
62
+
* Check the [Sumo Logic status](https://status.sumologic.com/) page to see if there’s an outage in your deployment. If the system is down, it cannot generate the baseline.
63
+
* If the rule has a Degraded status because it failed to parse, fix the rule so that it parses correctly. A baseline cannot be built if the rule does not successfully parse. One thing you can do is ensure the matching expression for the rule is using the compatible [core platform literals](/docs/cse/rules/cse-rules-syntax/#sumo-logic-core-platform-literals-supported-in-cloud-siem).
64
+
* If the rule has a Failed status, clicking the information button might show that the amount of data requested is too large to return (see [Rule limits](#rule-limits)). In this case, create a more filtered baseline focusing on the exact activity you want to capture.
65
+
* If the rule has a persistent Pending Baseline status, there might not be enough data in the system to build the baseline:
66
+
* Check the ingest configuration of your Cloud SIEM data sources and confirm the appropriate records are being added to the system.
67
+
* The matching expression may not be using the right fields. Cloud SIEM records are normalized to a defined [schema](/docs/cse/schema/schema-attributes/). The matching expression and all other fields should use that schema and not the raw log field names.
68
+
* There may not be enough activity to build a baseline. In these cases, there are not enough data elements to build an accurate baseline. Expand the baseline learning period to gather more activity.
69
+
* Make sure that the Sumo Logic system has been active and ingesting data for the full baseline learning period. For example, if the rule has a default baseline learning period of 30 days, but the user who created the rule has only been using Sumo Logic for a few days, then the rule will remain in the Pending Baseline state until 30 days have passed. To resolve the issue, change the baseline learning period window.
70
+
71
+
72
+
73
+
58
74
## Rule limits
59
75
60
76
Limits are set on how often a rule fires so that the system is not overloaded. For example, if a rule fires too many signals in an hour, it can cause performance problems for all rule processes. If a rule exceeds a limit, its rule status changes from Active to Failed and the rule is disabled.
Copy file name to clipboardExpand all lines: docs/cse/rules/write-first-seen-rule.md
+26-16Lines changed: 26 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,23 +20,14 @@ If you are new to writing rules, see [About Cloud SIEM Rules](/docs/cse/rules/ab
20
20
:::
21
21
22
22
## About first seen rules
23
+
23
24
First seen rules allow you to generate a signal when behavior by an entity (such as a user) is encountered that hasn't been seen before. For example, a first seen rule might look for the events like the following:
24
25
25
26
* First time a user logged in from a new geographic location (geolocation)
26
27
* Newly created or added admin accounts
27
28
* High severity EDR alert seen for the first time
28
29
* MFA acceptance from first seen device
29
30
30
-
A first seen rule is different from other Cloud SIEM rule types in that you don’t define the criteria for firing a signal. Instead, the rule expression in a first seen rule is simply a filter condition that defines what incoming records the rule will apply to. For each first seen rule, Cloud SIEM automatically creates a baseline model of normal behavior evidenced by records that match the Rule Expression. After the baseline learning period is completed, when an incoming record includes matching activity not seen during the baseline learning period, the rule creates a signal.
31
-
32
-
For example, for the “First time a user logged in from a new geographic location” use case, Cloud SIEM will build a baseline model of all the geolocations from where a logon event is seen for the entity (user). Once the baselining period is complete, Cloud SIEM will create a signal for every new geolocation detected and incrementally add to the baseline.
33
-
34
-
:::tip
35
-
Sumo Logic ensures that rule processing does not impact the reliability of production environments through the implementation of "circuit breakers." If a rule matches too many records in too short a period of time, the circuit breaker will trip and the rule will move to a degraded state, and first seen rules are no exception.
36
-
37
-
On the rule detail page, if you hover over the degraded message, you will usually see more details about what tripped the circuit breaker and how to resolve the problem. Generally speaking, a rule that is degraded probably needs to be tuned for your specific environment.
38
-
:::
39
-
40
31
:::sumo Micro Lesson
41
32
42
33
Watch this micro lesson to learn more about first seen rules.
@@ -68,10 +59,27 @@ Watch this micro lesson to learn more about first seen rules.
68
59
69
60
:::
70
61
62
+
## Baselines for first seen rules
63
+
64
+
A first seen rule is different from other Cloud SIEM rule types in that you don’t define the criteria for firing a signal. Instead, the rule expression in a first seen rule is simply a filter condition that defines what incoming records the rule will apply to. For each first seen rule, Cloud SIEM automatically creates a baseline model of normal behavior for a defined learning period (by default for the last 30 days) evidenced by records that match the Rule Expression. Once the baseline is created, when an incoming record includes matching activity not seen during the baseline learning period, the rule creates a signal.
65
+
66
+
For example, for the “First time a user logged in from a new geographic location” use case, Cloud SIEM will build a baseline model of all the geolocations from where a logon event is seen for the entity (user). Once the baseline is created, Cloud SIEM will create a signal for every new geolocation detected and incrementally add to the baseline.
67
+
68
+
As soon as you save or update a first seen rule, the baseline is built using existing data collected. So if your baseline learning period is for the last 30 days (the default), the system uses data from the previous 30 days to build the baseline. If data exists in the system to build the baseline, baseline creation typically takes only minutes to complete.
69
+
70
+
:::tip
71
+
Sumo Logic ensures that rule processing does not impact the reliability of production environments through the implementation of "circuit breakers." If a rule matches too many records in too short a period of time, the circuit breaker will trip and the rule will move to a degraded state, and first seen rules are no exception.
72
+
73
+
On the rule detail page, if you hover over the degraded message, you will usually see more details about what tripped the circuit breaker and how to resolve the problem. Generally speaking, a rule that is degraded probably needs to be tuned for your specific environment.
74
+
75
+
For more information, see [Troubleshoot baseline problems](/docs/cse/rules/rules-status/#troubleshoot-baseline-problems).
76
+
:::
77
+
71
78
## Example rule
79
+
72
80
The screenshot below shows a first seen rule in the Cloud SIEM rules editor. For an explanation of the configuration options, see [Create a first seen rule](#create-a-first-seen-rule), below.
73
-
<img src={useBaseUrl('img/cse/first-seen-rule.png')} alt="Example first seen rule definition" style={{border: '1px solid gray'}} width="700"/>
74
81
82
+
<img src={useBaseUrl('img/cse/first-seen-rule.png')} alt="Example first seen rule definition" style={{border: '1px solid gray'}} width="700"/>
75
83
76
84
## Create a first seen rule
77
85
@@ -100,7 +108,7 @@ The settings in the **If Triggered** section determine what records the rule wil
100
108
:::
101
109
1. Set the baseline and retention settings:
102
110
1.**Baseline Retention Period (days)**. The number of days after which the data points in the baseline will expire (be dropped from the baseline). The default is 90 days. You can decrease this period, but not increase it.
103
-
1.**Baseline Learning Period (days)**. The minimum amount of time for which data points should be collected before firing a signal. The default is 30 days.
111
+
1.**Baseline Learning Period (days)**. The minimum amount of time for which data points should be collected before firing a signal. The default is for the last 30 days.
104
112
:::note
105
113
The **Baseline Learning Period** must be shorter than the **Baseline Retention Period**. Also be aware that short baseline learning periods can potentially generate false positive signals.
106
114
:::
@@ -123,13 +131,15 @@ The settings in the **If Triggered** section determine what records the rule wil
123
131
124
132
## When the baseline is reset for a first seen rule
125
133
126
-
The baseline learning period begins again when the following fields on the rule are updated or overridden:
134
+
Baseline creation begins again when the following fields on the rule are updated or overridden:
127
135
***If Triggered**:
128
136
***When a Record matching the expression**
129
137
***Has a new value for the field(s)**
130
138
***Then Create a Signal**:
131
139
***On Entity**
132
140
141
+
If data exists in the system to build the baseline, baseline creation typically takes only minutes to complete.
142
+
133
143
## Use case: Monitor login from first seen geolocation
134
144
135
145
This section shows how the same first seen rule would function with each of the two baselining strategies.
@@ -142,12 +152,12 @@ with **has a new value for the field(s)** set to `srcDeviceIP_countryName`
142
152
143
153
### With a global baseline
144
154
145
-
With a global baseline, and the default baseline learning period of 30 days, the rule will baseline all geolocations that users are logging in for a period of 30 days. After the 30 day baseline is completed, if a new geolocation is detected, a signal will be created. Then, if a new hire (that wasn’t part of the 30 day baseline) logs in from any geolocation, a signal
146
-
will be created. As a global baseline, the 30 day baseline is shared across all entity.
155
+
With a global baseline, and the default baseline learning period of the last 30 days, the rule creates a baseline of all geolocations that users logged in from for the previous 30 days. If a new geolocation is detected, a signal will be created. Then, if a new hire (that wasn’t part of the 30 day baseline) logs in from any geolocation, a signal
156
+
will be created. As a global baseline, the 30 day baseline is shared across all entities.
147
157
148
158
### With per-entity baselines
149
159
150
-
With a per-entity baseline, and the default baseline learning period of 30 days, the rule will baseline all geolocations on a per-entity basis for 30 days. It will generate a signal when a new geolocation is not part of a user’s historic baseline. On a new hire’s first login, a 30 day baseline will begin building. After the 30 day baseline is created, if that user logs on from a new geolocation, the rule will create a signal.
160
+
With a per-entity baseline, and the default baseline learning period of the last 30 days, the rule creates a baseline of all geolocations on a per-entity basis for the last 30 days. It will generate a signal when a new geolocation is not part of a user’s historic baseline. On a new hire’s first login, a baseline for the last 30 days will begin rebuilding. If that user logs on from a new geolocation, the rule will create a signal.
151
161
152
162
:::tip
153
163
If you are unsure whether to use a per-entity or a global baseline, consider your use case. If you’re inclined to select `user_username` in the **Has a new value for the field(s)** prompt, you’re better off creating a global baseline for that behavior. Alternatively, if you want to track a new value for a non-entity record field, a per-entity baseline is appropriate.
0 commit comments