You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: solutions/security/get-started/automatic-migration.md
+63-37Lines changed: 63 additions & 37 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,9 +7,14 @@ applies_to:
7
7
8
8
# Automatic migration
9
9
10
-
Automatic Migration for detection rules helps you quickly convert SIEM rules from the Splunk's Search Processing Language (SPL) to the Elasticsearch Query Language ({{esql}}). If comparable Elastic-authored rules exist, it simplifies onboarding by mapping your rules to them. Otherwise, it creates custom rules on the fly so you can verify and edit them instead of writing them from scratch.
10
+
Automatic Migration helps you quickly migrate Splunk assets to {{elastic-sec}}. The following asset types are supported:
11
11
12
-
You can ingest your data before migrating your rules, or migrate your rules first in which case the tool will recommend which data sources you need to power your migrated rules.
12
+
* {applies_to}`stack: preview 9.0, ga 9.1` {applies_to}`serverless: ga` Splunk rules
For rule migrations, if comparable Elastic-authored rules exist, Automatic Migration simplifies onboarding by mapping your rules to them. Otherwise, it creates custom rules and dashboards on the fly so you can verify and edit them instead of writing them from scratch.
16
+
17
+
You can ingest your data before migrating your assets, or migrate your assets first in which case the tool will recommend which data sources you need to power your migrated rules.
13
18
14
19
::::{admonition} Requirements
15
20
* The `SIEM migrations: All` Security sub-feature privilege.
@@ -18,15 +23,14 @@ You can ingest your data before migrating your rules, or migrate your rules firs
18
23
* {{Stack}} users: {{ml}} must be enabled.
19
24
* {{serverless-short}} users: a [Security Complete](/deploy-manage/deploy/elastic-cloud/project-settings.md) subscription.
20
25
* {{ecloud}} users: {{ml}} must be enabled. We recommend a minimum size of 4GB of RAM per {{ml}} zone.
21
-
22
26
::::
23
27
24
28
## Get started with Automatic Migration
25
29
26
30
1. Find **Get started** in the navigation menu or use the [global search bar](/explore-analyze/find-and-organize/find-apps-and-objects.md).
27
31
2. Under **Configure AI provider**, select a configured model or [add a new one](/solutions/security/ai/set-up-connectors-for-large-language-models-llm.md). For information on how different models perform, refer to the [LLM performance matrix](/solutions/security/ai/large-language-model-performance-matrix.md).
28
-
3. Next, under **Migrate rules & add data**, click **Translate your existing SIEM rules to Elastic**, then **Upload rules**.
29
-
4. Follow the instructions on the **Upload Splunk SIEM rules**flyout to export your rules from Splunk as JSON.
32
+
3. Next, under **Migrate rules & dashboards**, select either **Translate your existing SIEM rules to Elastic** or **Migrate your existing SIEM dashboards to Elastic**, then click **Upload**. The upload flyout appears.
33
+
4. Follow the instructions on the upload flyout to export your Splunk assets as JSON.
@@ -36,27 +40,24 @@ You can ingest your data before migrating your rules, or migrate your rules firs
36
40
37
41
38
42
::::{note}
39
-
The provided query downloads Splunk correlation rules and saved searches. Alternatively, as long as you export your results in a JSON format, you can use a different query. For example:
43
+
The provided queries download a default selection of Splunk assets. Alternatively, as long as you export your results as JSON, you can use a different query. For example, the following custom query would download rules related to just the `splunksysmonsecurity` app:
40
44
41
45
```
42
46
| rest /servicesNS/-/-/saved/searches
43
47
| search is_scheduled=1 AND eai:acl.app=splunksysmonsecurity
The above sample query would download rules related to just the `splunksysmonsecurity` app.
48
51
49
-
We don't recommend downloading all searches (for example with `| rest /servicesNS/-/-/saved/searches`) since most of the data will be irrelevant to SIEM rule migration.
52
+
For rule migration, we recommend against downloading all searches (for example with `| rest /servicesNS/-/-/saved/searches`) since much of the data would be irrelevant to asset migration.
50
53
::::
51
54
52
-
5. Select your JSON file and click **Upload**.
53
-
::::{note}
54
-
If the file is large, you may need to separate it into multiple parts and upload them individually to avoid exceeding your LLM's context window.
55
-
::::
55
+
5. Select your JSON file and click **Upload**. If the file is large, you may need to separate it into multiple parts and upload them individually to avoid exceeding your LLM's context window.
56
56
57
-
6. After you upload your Splunk rules, Automatic Migration will detect whether they use any Splunk macros or lookups. If so, follow the instructions which appear to export and upload them. Alternatively, you can complete this step later — however, until you upload them, some of your migrated rules will have a `partially translated` status. If you upload them now, you don't have to wait on the page for them to be processed — a notification will appear when processing is complete.
57
+
6. After you upload your Splunk assets, Automatic Migration will detect whether they use any macros or lookups. If so, follow the instructions which appear to export and upload them. Alternatively, you can complete this step later — however, until you upload them, some of your migrated assets will have a `partially translated` status. If you upload them now, you don't have to wait on this page for them to be processed — a notification will appear when processing is complete.
58
58
59
-
7. Click **Translate** to start the rule translation process. A name for this migration is automatically created, and you can track its progress on this page. The **More actions** ({icon}`boxes_vertical`) button lets you rename or pause the migration.
59
+
7. Click **Translate** to start the rule translation process. The **Start rules migration** popup appears. Use the dropdown menu to select which AI connector to use. For rule migrations there is a **Match to Elastic prebuilt rules** option, which is enabled by default; when it's enabled, any migrated rules that are similar to an Elastic prebuilt rule are converted to those prebuilt rules (this uses fewer tokens). When it's disabled, each of your rules will be converted into a new custom rule.
60
+
8. Click **Translate**. A name for the migration is automatically created, and you can track its progress on this page. The **More actions** ({icon}`boxes_vertical`) button lets you rename or delete the migration.
@@ -67,22 +68,20 @@ You can ingest your data before migrating your rules, or migrate your rules firs
67
68
You don't need to stay on this page. A notification will appear when the migration is complete.
68
69
69
70
70
-
8. Use the **Add SIEM data with Integrations** section to set up data ingestion from third-party sources. If at least one rule migration has completed, the **Recommended** tab shows integrations that provide the data needed by your translated rules. These include both Elastic-managed integrations and any applicable custom creations you made using [automatic import](/solutions/security/get-started/automatic-import.md).
71
+
9. Use the **Add SIEM data with Integrations** section to set up data ingestion from third-party sources. If at least one rules migration has completed, the **Recommended** tab shows integrations that provide the data needed by your translated rules. These include both Elastic-managed integrations and any applicable custom integrations you made using [automatic import](/solutions/security/get-started/automatic-import.md).
9. When migration is complete, click the notification or return to the **Get started** page then click **View translated rules** to open the **Translated rules** page.
79
+
10. When migration is complete, click the notification or return to the **Get started** page then click **View translated rules**or **View translated dashboards**to open the [**Translated rules**](#the-translated-rules-page) page or the [**Translated dashboards**](#the-translated-dashboards-page) page.
79
80
80
81
81
82
## The Translated rules page
82
83
83
-
This section describes the **Translated rules** page's interface and explains how the data that appears here is derived.
84
-
85
-
When you upload a new batch of rules, they are assigned a name and number, for example `SIEM rule migration 1`, or `SIEM rule migration 2`. Use the **Migrations** dropdown menu in the upper right to select which batch appears.
84
+
This section describes the **Translated rules** page's interface and the data it displays. Use the **Migrations** dropdown menu in the upper right to select which migration appears.
@@ -92,13 +91,17 @@ When you upload a new batch of rules, they are assigned a name and number, for e
92
91
93
92
The table's fields are as follows:
94
93
94
+
***Updated:** The migration date.
95
95
***Name:** The names of Elastic-authored rules cannot be edited until after rule installation. To edit the name of a custom translated rule, click the name and select **Edit**.
96
96
***Status:** The rule's translation status:
97
97
*`Installed`: Already added to Elastic SIEM. Click **View** to manage and enable it.
98
98
*`Translated`: Ready to install. This rule was mapped to an Elastic-authored rule, or translated by Automatic Import. Click **Install** to install it.
99
-
*`Partially translated`: Part of the query could not be translated. You may need to specify an index pattern for the rule query, upload missing macros or lookups, or fix broken rule syntax.
99
+
*`Partially translated`: Part of the query could not be translated. You may need to specify an index pattern for the rule query, upload missing macros or lookups, or fix broken rule syntax.
100
+
::::{note}
101
+
To fix partially translated rules that are missing an index pattern, use the **Update missing index pattern** button. This affects all migrated assets that contain the placeholder `[indexPattern]`, not just those currently visibly on the table page.
102
+
::::
100
103
*`Not translated`: None of the original query could be translated.
101
-
*`Error`: Rule translation failed. Refer to the the error details.
104
+
*`Failed`: Translation failed. Refer to the the error for details.
102
105
***Risk Score:** For Elastic-authored rules, risk scores are predefined. For custom translated rules, risk scores are defined as follows:
103
106
* If the source rule has a field comparable to Elastic's `risk score`, we use that value.
104
107
* Otherwise, if the source rule has a field comparable to Elastic's `rule severity` field, we base the risk score on that value according to [these guidelines](/solutions/security/detect-and-alert/create-detection-rule.md#rule-ui-basic-params).
@@ -125,17 +128,15 @@ The table's fields are as follows:
125
128
:screenshot:
126
129
::::
127
130
128
-
## Finalize translated rules
131
+
###Finalize translated rules and view rule details
129
132
130
-
Once you're on the **Translated rules** page, to install any rules that were partially translated or not translated, you will need to edit them. Optionally, you can also edit custom rules that were successfully translated to finetune them.
133
+
To install any rules that were partially translated or not translated, you will need to edit them. Optionally, you can also edit rules that were successfully translated to finetune them.
131
134
132
135
:::{note}
133
136
You cannot edit Elastic-authored rules using this interface, but after they are installed you can [edit them](/solutions/security/detect-and-alert/manage-detection-rules.md) from the **Rules** page.
134
137
:::
135
138
136
-
### Edit a custom rule
137
-
138
-
Click the rule's name to open the rule's details flyout to the **Translation** tab, which shows the source rule alongside the translated — or partially translated — Elastic version. You can update any part of the rule. When finished, click **Save**.
139
+
Click a rule's name to open its details flyout to the **Translation** tab, which shows the source rule alongside the translated — or partially translated — Elastic version. You can update any part of the rule. When finished, click **Save**.
@@ -144,35 +145,60 @@ Click the rule's name to open the rule's details flyout to the **Translation** t
144
145
::::
145
146
146
147
::::{note}
147
-
If you haven't yet ingested your data, you will likely encounter `Unknown index` or `Unknown column` errors while editing. You can ignore these and add your data later.
148
+
If you haven't yet ingested your data, you may encounter `Unknown index` or `Unknown column` errors. You can ignore these and add your data later.
148
149
::::
149
150
150
-
### View rule details
151
+
The rule details flyout has two other tabs, **Overview** and **Summary**. The **Overview** tab displays information such as the rule's severity, risk score, rule type, and how frequently it runs. The **Summary** tab explains the logic behind how the rule was translated, such as why specific {{esql}} commands were used, or why a source rule was mapped to a particular Elastic-authored rule.
152
+
153
+
## The Translated dashboards page
151
154
152
-
The rule details flyout (which appears when you click on a rule's name in the **Translate rules**table) has two other tabs, **Overview** and **Summary**. The **Overview** tab displays information such as the rule's severity, risk score, rule type, and how frequently it runs. The **Summary**tab explains the logic behind how the rule was translated, such as why specific {{esql}} commands were used, or why a source rule was mapped to a particular Elastic-authored rule.
155
+
This section describes the **Translated dashboards**page's interface and the data it displays. Use the **Migrations**dropdown menu in the upper right to select which migration appears.
153
156
154
-
::::{important}
155
-
All the details about your migrations is stored in the `.kibana-siem-rule-migrations-rules-default` index. You can use [Discover](/explore-analyze/discover.md) to review a variety of metrics, analyze metrics, and more.
***Name:** The names of the translated dashboards cannot be edited until after installation.
166
+
***Updated:** The date when the dashboard was last modified in its source platform.
167
+
***Status:** The dashboard's translation status:
168
+
*`Installed`: Already added to {{elastic-sec}}. Click **View** to manage it.
169
+
*`Translated`: Ready to install. Click **Install** to install it.
170
+
*`Partially translated`: Part of the dashboard could not be translated. Upload any missing macros or lookups, or fix broken syntax.
171
+
*`Not translated`: None of the original dashboard could be translated.
172
+
*`Failed`: Translation failed. Click the dashboard's name to open the details flyout and view error details.
173
+
***Tags:** The dashboard's tags, which identify its source application, and can be used to identify it on the **Dashboards** page.
174
+
***Actions:** To view an `Installed` dashboard, click **View**. To install a `Translated` dashboard, click **Install**. To reprocess a `Failed` dashboard, click **Reprocess**.
175
+
176
+
::::{note}
177
+
To view an explanation of the logic behind how each dashboard was translated, click a dashboard's name to open the dashboard details flyout.
178
+
::::
179
+
180
+
### Finalize translated dashboards
181
+
182
+
Once you're on the **Translated dashboards** page, to install any assets that were partially translated, you will need to edit them. Optionally, you can also edit assets that were successfully translated to finetune them. For more information about editing dashboards, refer to [Building dashboards](/explore-analyze/dashboards/building.md).
183
+
158
184
## Frequently asked questions (FAQ)
159
185
160
-
**How does Automatic Migration handle rules that can't be exactly translated for various reasons, such as feature parity issues?**
186
+
**How does Automatic Migration handle assets that can't be exactly translated for various reasons, such as feature parity issues?**
161
187
162
-
After translation, rules that can't be translated appear with a status of either partially translated (yellow) or not translated (red). From there, you can address them individually.
188
+
After translation, assets that can't be translated appear with a status of either partially translated (yellow) or not translated (red). From there, you can address them individually.
163
189
164
190
**Are nested macros supported?**
165
191
166
192
Yes, Automatic Migration can handle nested macros.
167
193
168
194
**How can we ensure rules stay up to date?**
169
195
170
-
Automatic Migration maps your rules to Elastic-authored rules whenever possible, which are updated automatically. Like all custom rules, rules created by Automatic Migration must be maintained by you.
196
+
With the exception of rules that were matched to Elastic-authored rules (which are updated automatically), assets created by Automatic Migration must be maintained by you.
171
197
172
198
**What index does information about each migration appear in?**
173
199
174
-
No matter how many times you use Automatic Migration, migration data will continue to appear in `.kibana-siem-rule-migrations-rules-default`.
200
+
Rule migration data appears in `.kibana-siem-rule-migrations-rules-default`. Dashboard migration data appears in `.kibana-siem-dashboard-migrations-dashboards-default`. You can use [Discover](/explore-analyze/discover.md) to review a variety of metrics, analyze metrics, and more.
175
201
176
-
**How does Automatic Migration handle Splunk rules which lookup other indices?**
202
+
**How does Automatic Migration handle Splunk assets which lookup other indices?**
177
203
178
-
Rules that fall into this category will typically appear with a status of partially translated. You can use the [`LOOKUP JOIN`](elasticsearch://reference/query-languages/esql/esql-lookup-join.md) capability to help in this situation.
204
+
Assets that fall into this category will typically appear with a status of partially translated. You can use the [`LOOKUP JOIN`](elasticsearch://reference/query-languages/esql/esql-lookup-join.md) capability to help in this situation.
0 commit comments