Skip to content

Commit 8fdd25d

Browse files
[8.19] [Security Solution] Actualize prebuilt rule upgrade test plans (#222606) (#225040)
# Backport This will backport the following commits from `main` to `8.19`: - [[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)](#222606) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-06-24T10:09:52Z","message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v9.1.0","v8.19.0"],"title":"[Security Solution] Actualize prebuilt rule upgrade test plans","number":222606,"url":"https://github.com/elastic/kibana/pull/222606","mergeCommit":{"message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/222606","number":222606,"mergeCommit":{"message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Maxim Palenov <[email protected]>
1 parent 1da892f commit 8fdd25d

File tree

5 files changed

+944
-983
lines changed

5 files changed

+944
-983
lines changed

x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_installation.md

Lines changed: 75 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,10 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one
4646
- [**Scenario: User can see correct rule information in preview before installing**](#scenario-user-can-see-correct-rule-information-in-preview-before-installing)
4747
- [**Scenario: Optional tabs and sections without content should be hidden in preview before installing**](#scenario-optional-tabs-and-sections-without-content-should-be-hidden-in-preview-before-installing)
4848
- [Rule installation workflow: filtering, sorting, pagination](#rule-installation-workflow-filtering-sorting-pagination)
49+
- [**Scenario: User can search prebuilt rules by rule name, index pattern or MITRE ATT\&CK™ tactic or technique on the Prebuilt Rules installation page**](#scenario-user-can-search-prebuilt-rules-by-rule-name-index-pattern-or-mitre-attck-tactic-or-technique-on-the-prebuilt-rules-installation-page)
50+
- [**Scenario: User can filter prebuilt rules by tags on the Prebuilt Rules installation page**](#scenario-user-can-filter-prebuilt-rules-by-tags-on-the-prebuilt-rules-installation-page)
51+
- [**Scenario: User can sort prebuilt rules on Prebuilt Rules installation page**](#scenario-user-can-sort-prebuilt-rules-on-prebuilt-rules-installation-page)
52+
- [**Scenario: User can paginate over prebuilt rules on Prebuilt Rules installation page**](#scenario-user-can-paginate-over-prebuilt-rules-on-prebuilt-rules-installation-page)
4953
- [Rule installation workflow: misc cases](#rule-installation-workflow-misc-cases)
5054
- [**Scenario: User opening the Add Rules page sees a loading skeleton until the package installation is completed**](#scenario-user-opening-the-add-rules-page-sees-a-loading-skeleton-until-the-package-installation-is-completed)
5155
- [**Scenario: User can navigate from the Add Rules page to the Rule Management page via breadcrumbs**](#scenario-user-can-navigate-from-the-add-rules-page-to-the-rule-management-page-via-breadcrumbs)
@@ -108,79 +112,6 @@ Previewing properties of a prebuilt rule before installing it:
108112
- If user chooses to preview a prebuilt rule to be installed, we currently show this preview in a flyout.
109113
- In the prebuilt rule preview a tab that doesn't have any sections should not be displayed and a section that doesn't have any properties also should not be displayed.
110114

111-
Examples of rule properties we show in the prebuilt rule preview flyout:
112-
113-
```Gherkin
114-
Examples:
115-
| rule_type | property | tab | section |
116-
│ All rule types │ Author │ Overview │ About │
117-
│ All rule types │ Building block │ Overview │ About │
118-
│ All rule types │ Severity │ Overview │ About │
119-
│ All rule types │ Severity override │ Overview │ About │
120-
│ All rule types │ Risk score │ Overview │ About │
121-
│ All rule types │ Risk score override │ Overview │ About │
122-
│ All rule types │ Reference URLs │ Overview │ About │
123-
│ All rule types │ False positive examples │ Overview │ About │
124-
│ All rule types │ Custom highlighted fields │ Overview │ About │
125-
│ All rule types │ License │ Overview │ About │
126-
│ All rule types │ Rule name override │ Overview │ About │
127-
│ All rule types │ MITRE ATT&CK™ │ Overview │ About │
128-
│ All rule types │ Timestamp override │ Overview │ About │
129-
│ All rule types │ Tags │ Overview │ About │
130-
│ All rule types │ Type │ Overview │ Definition │
131-
│ All rule types │ Related integrations │ Overview │ Definition │
132-
│ All rule types │ Required fields │ Overview │ Definition │
133-
│ All rule types │ Timeline template │ Overview │ Definition │
134-
│ All rule types │ Runs every │ Overview │ Schedule │
135-
│ All rule types │ Additional look-back time │ Overview │ Schedule │
136-
│ All rule types │ Setup guide │ Overview │ Setup guide │
137-
│ All rule types │ Investigation guide │ Investigation guide │ Investigation guide │
138-
│ Custom Query │ Index patterns │ Overview │ Definition │
139-
│ Custom Query │ Data view ID │ Overview │ Definition │
140-
│ Custom Query │ Data view index pattern │ Overview │ Definition │
141-
│ Custom Query │ Custom query │ Overview │ Definition │
142-
│ Custom Query │ Filters │ Overview │ Definition │
143-
│ Custom Query │ Saved query name │ Overview │ Definition │
144-
│ Custom Query │ Saved query filters │ Overview │ Definition │
145-
│ Custom Query │ Saved query │ Overview │ Definition │
146-
│ Custom Query │ Suppress alerts by │ Overview │ Definition │
147-
│ Custom Query │ Suppress alerts for │ Overview │ Definition │
148-
│ Custom Query │ If a suppression field is missing │ Overview │ Definition │
149-
│ Machine Learning │ Anomaly score threshold │ Overview │ Definition │
150-
│ Machine Learning │ Machine Learning job │ Overview │ Definition │
151-
│ Threshold │ Threshold │ Overview │ Definition │
152-
│ Threshold │ Index patterns │ Overview │ Definition │
153-
│ Threshold │ Data view ID │ Overview │ Definition │
154-
│ Threshold │ Data view index pattern │ Overview │ Definition │
155-
│ Threshold │ Custom query │ Overview │ Definition │
156-
│ Threshold │ Filters │ Overview │ Definition │
157-
│ Event Correlation │ EQL query │ Overview │ Definition │
158-
│ Event Correlation │ Filters │ Overview │ Definition │
159-
│ Event Correlation │ Index patterns │ Overview │ Definition │
160-
│ Event Correlation │ Data view ID │ Overview │ Definition │
161-
│ Event Correlation │ Data view index pattern │ Overview │ Definition │
162-
│ Indicator Match │ Indicator index patterns │ Overview │ Definition │
163-
│ Indicator Match │ Indicator mapping │ Overview │ Definition │
164-
│ Indicator Match │ Indicator filters │ Overview │ Definition │
165-
│ Indicator Match │ Indicator index query │ Overview │ Definition │
166-
│ Indicator Match │ Index patterns │ Overview │ Definition │
167-
│ Indicator Match │ Data view ID │ Overview │ Definition │
168-
│ Indicator Match │ Data view index pattern │ Overview │ Definition │
169-
│ Indicator Match │ Custom query │ Overview │ Definition │
170-
│ Indicator Match │ Filters │ Overview │ Definition │
171-
│ New Terms │ Fields │ Overview │ Definition │
172-
│ New Terms │ History Window Size │ Overview │ Definition │
173-
│ New Terms │ Index patterns │ Overview │ Definition │
174-
│ New Terms │ Data view ID │ Overview │ Definition │
175-
│ New Terms │ Data view index pattern │ Overview │ Definition │
176-
│ New Terms │ Custom query │ Overview │ Definition │
177-
│ New Terms │ Filters │ Overview │ Definition │
178-
│ ESQL │ ESQL query │ Overview │ Definition │
179-
│ ESQL │ Suppress alerts by │ Overview │ Definition │
180-
│ ESQL │ Suppress alerts for │ Overview │ Definition │
181-
│ ESQL │ If a suppression field is missing │ Overview │ Definition │
182-
```
183-
184115
## Scenarios
185116

186117
### Rule installation notifications on the Rule Management page
@@ -197,7 +128,7 @@ Then user should NOT see a CTA to install prebuilt rules
197128
And user should NOT see a number of rules available to install
198129
And user should NOT see a CTA to upgrade prebuilt rules
199130
And user should NOT see a number of rules available to upgrade
200-
And user should NOT see the Rule Updates table
131+
And user should NOT see the Prebuilt Rules Upgrades page
201132
```
202133

203134
#### **Scenario: User is NOT notified when all prebuilt rules are installed and up to date**
@@ -212,7 +143,7 @@ Then user should NOT see a CTA to install prebuilt rules
212143
And user should NOT see a number of rules available to install
213144
And user should NOT see a CTA to upgrade prebuilt rules
214145
And user should NOT see a number of rules available to upgrade
215-
And user should NOT see the Rule Updates table
146+
And user should NOT see the Prebuilt Rules Upgrades page
216147
```
217148

218149
#### **Scenario: User is notified when no prebuilt rules are installed and there are rules available to install**
@@ -228,7 +159,7 @@ Then user should see a CTA to install prebuilt rules
228159
And user should see a number of rules available to install (X)
229160
And user should NOT see a CTA to upgrade prebuilt rules
230161
And user should NOT see a number of rules available to upgrade
231-
And user should NOT see the Rule Updates table
162+
And user should NOT see the Prebuilt Rules Upgrades page
232163
```
233164

234165
#### **Scenario: User is notified when some prebuilt rules can be installed**
@@ -245,7 +176,7 @@ Then user should see a CTA to install prebuilt rules
245176
And user should see the number of rules available to install (Y)
246177
And user should NOT see a CTA to upgrade prebuilt rules
247178
And user should NOT see a number of rules available to upgrade
248-
And user should NOT see the Rule Updates table
179+
And user should NOT see the Prebuilt Rules Upgrades page
249180
```
250181

251182
#### **Scenario: User is notified when both rules to install and upgrade are available**
@@ -418,7 +349,73 @@ And the Investigation Guide tab should NOT be displayed
418349

419350
### Rule installation workflow: filtering, sorting, pagination
420351

421-
TODO: add scenarios https://github.com/elastic/kibana/issues/166215
352+
#### **Scenario: User can search prebuilt rules by rule name, index pattern or MITRE ATT&CK™ tactic or technique on the Prebuilt Rules installation page**
353+
354+
**Automation**: 1 e2e test with mock rules
355+
356+
```Gherkin
357+
Given multiple prebuilt rules available for installation
358+
When user opens the Prebuilt Rules installation page
359+
Then the available prebuilt rules should be shown
360+
When user enters <text> in the search field
361+
Then only the available prebuilt rules matching the <text> should be shown
362+
```
363+
364+
**Examples:**
365+
366+
- `<text>`
367+
- rule name or its part
368+
- index pattern
369+
- MITRE ATT&CK™ tactic or technique
370+
371+
#### **Scenario: User can filter prebuilt rules by tags on the Prebuilt Rules installation page**
372+
373+
**Automation**: 1 e2e test with mock rules
374+
375+
```Gherkin
376+
Given multiple prebuilt rules available for installation
377+
When user opens the Prebuilt Rules installation page
378+
Then the available prebuilt rules should be shown
379+
When user filters the available prebuilt rules by one or more tags
380+
Then only the available prebuilt rules having these tags should be shown
381+
```
382+
383+
#### **Scenario: User can sort prebuilt rules on Prebuilt Rules installation page**
384+
385+
**Automation**: 1 e2e test with mock rules
386+
387+
```Gherkin
388+
Given multiple prebuilt rules available for installation
389+
When user opens the Prebuilt Rules installation page
390+
Then the available prebuilt rules should be shown
391+
When user clicks on <field> header by picking the sorting direction
392+
Then the available prebuilt rules should be sorted by <field> in the expected order
393+
```
394+
395+
**Examples:**
396+
397+
- `<field>`
398+
- rule name
399+
- risk score
400+
- severity
401+
402+
#### **Scenario: User can paginate over prebuilt rules on Prebuilt Rules installation page**
403+
404+
**Automation**: 1 e2e test with mock rules
405+
406+
```Gherkin
407+
Given multiple prebuilt rules available for installation
408+
When user opens the Prebuilt Rules installation page
409+
Then the available prebuilt rules should be shown
410+
When user picks the desired number of <rows_per_page>
411+
Then the <rows_per_page> of the available prebuilt rules should be shown on the page
412+
When user navigates to the next pages
413+
Then the next page of the available prebuilt rules should be shown
414+
```
415+
416+
**Examples:**
417+
418+
`<rows_per_page>` = 5 | 10 | 20 | 50 | 100
422419

423420
### Rule installation workflow: misc cases
424421

@@ -504,7 +501,6 @@ Notes:
504501
- install: `POST /internal/detection_engine/prebuilt_rules/installation/_perform`
505502
- status: `GET /internal/detection_engine/prebuilt_rules/status`
506503

507-
508504
#### **Scenario: API does not install prebuilt rules if they are up to date**
509505

510506
**Automation**: 4 integration tests with mock rules.

0 commit comments

Comments
 (0)