From 4cc22b33aa6f826c9e3161ebdbfcecb16e7d066a Mon Sep 17 00:00:00 2001 From: Janeen Roberts Date: Mon, 31 Mar 2025 23:41:59 -0400 Subject: [PATCH 1/7] Kibana updates. --- .../saved-object-migrations.md | 25 +++++++++--------- .../prepare-to-upgrade/upgrade-assistant.md | 26 ++++++++++++------- 2 files changed, 28 insertions(+), 23 deletions(-) diff --git a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md index 11c89ecb5b..d2588bcad7 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md +++ b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md @@ -12,41 +12,40 @@ Each time you upgrade {{kib}}, an upgrade migration is performed to ensure that :::: -::::{warning} -{{kib}} 7.12.0 and later uses a new migration process and index naming scheme. Before you upgrade, read the documentation for your version of {{kib}}. -:::: +% ::::{warning} +% {{kib}} 7.12.0 and later uses a new migration process and index naming scheme. % Before you upgrade, read the documentation for your version of {{kib}}. +% :::: ::::{warning} -The `kibana.index` and `xpack.tasks.index` configuration settings are obsolete and no longer taken into account in 8.x. If you are using custom index names, please perform the necessary adaptations before attempting to upgrade to 8.x. +The `kibana.index` and `xpack.tasks.index` configuration settings are obsolete and no longer considered in 9.x. If you use custom index names, please make the necessary adaptations before upgrading to 9.x. :::: +## How saved object migrations work [upgrade-migrations-process] -## How saved objects migrations work [upgrade-migrations-process] - -When you start a new {{kib}} installation, an upgrade migration is performed before starting plugins or serving HTTP traffic. Before you upgrade, shut down old nodes to prevent losing acknowledged writes. To reduce the likelihood of old nodes losing acknowledged writes, {{kib}} 7.12.0 and later adds a write block to the outdated index. +When you start a new {{kib}} installation, an upgrade migration is performed before starting plugins or serving HTTP traffic. Before you upgrade, shut down old nodes to prevent losing acknowledged writes. To reduce the likelihood of old nodes losing acknowledged writes, {{kib}} adds a write block to the outdated index. -Saved objects are stored in multiple indices. Whilst all of them start with the `.kibana*` prefix, other `.kibana*` indices exist, which are not used to store saved objects. The following tables lists the saved objects indices used by each {{kib}} version. +Saved objects are stored in multiple indices. While they all start with the `.kibana*` prefix, other `.kibana*` indices exist but are not used to store saved objects. The following table lists the saved objects indices used by each {{kib}} version. | Upgrading from version | Index | Aliases | | --- | --- | --- | | 6.5.0 through 7.3.x | `.kibana_N` | `.kibana` | | 7.4.0 through 7.11.x | `.kibana_N`
`.kibana_task_manager_N` | `.kibana`
`.kibana_task_manager` | | 7.11.x through 8.7.x | `.kibana_{{kibana_version}}_001`
`.kibana_task_manager_{{kibana_version}}_001` | `.kibana`, `.kibana_{{kibana_version}}`
`.kibana_task_manager`, `.kibana_task_manager_{{kibana_version}}` | -| 8.8.0+ | `.kibana_{{kibana_version}}_001`
`.kibana_alerting_cases_{{kibana_version}}_001``.kibana_analytics_{{kibana_version}}_001``.kibana_ingest_{{kibana_version}}_001``.kibana_task_manager_{{kibana_version}}_001``.kibana_security_solution_{{kibana_version}}_001` | `.kibana`, `.kibana_{{kibana_version}}`
`.kibana_alerting_cases`, `.kibana_alerting_cases_{{kibana_version}}``.kibana_analytics`, `.kibana_analytics_{{kibana_version}}``.kibana_ingest`, `.kibana_ingest_{{kibana_version}}``.kibana_task_manager`, `.kibana_task_manager_{{kibana_version}}``.kibana_security_solution`, `.kibana_security_solution_{{kibana_version}}` | +| 8.8.0+ | `.kibana_{kibana_version}_001`
`.kibana_alerting_cases_{{kibana_version}}_001`
`.kibana_analytics_{kibana_version}_001`
`.kibana_ingest_{kibana_version}_001`
`.kibana_task_manager_{kibana_version}_001`
`.kibana_security_solution_{kibana_version}_001` | .`kibana`, `.kibana_{kibana_version}`
`.kibana_alerting_cases`,
`.kibana_alerting_cases_{kibana_version}`
`.kibana_analytics`,
`.kibana_analytics_{kibana_version}`
`.kibana_ingest`, `.kibana_ingest_{kibana_version}`
`.kibana_task_manager`,
`.kibana_task_manager_{kibana_version}`
`.kibana_security_solution`,
`.kibana_security_solution_{kibana_version}` -Starting on 7.11.0, each of the saved objects indices has a couple of aliases, e.g. the `.kibana_8.8.0_001` index has a *default* alias `.kibana` and a *version* alias `.kibana_8.8.0`. The *default* aliases (e.g. `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version. +Starting on 7.11.0, each of the saved objects indices has a couple of aliases. For example, the `.kibana_8.8.0_001` index has a *default* alias `.kibana` and a *version* alias `.kibana_8.8.0`. The *default* aliases (such as `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version. -Starting on 8.6.0, index names aren’t necessarily aligned with the deployed {{kib}} version. When updates on a certain index are compatible, {{kib}} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for a 8.8.0 deployment: +Starting on 8.6.0, index names aren’t necessarily aligned with the deployed {{kib}} version. When updates on a certain index are compatible, {{kib}} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for an 8.8.0 deployment: * `.kibana_8.8.0_001` index, with `.kibana` and `.kibana_8.8.0` aliases. * `.kibana_task_manager_8.7.0_001` index, with `.kibana_task_manager` and `.kibana_task_manager_8.8.0` aliases. -Starting on 8.8.0, {{kib}} splits the main saved object index into multiple ones, as depicted on the table above. When upgrading from a previous version, the {{kib}} migration process will reindex some saved objects from the `.kibana` index into the new indices, depending on their types. Note that the `.kibana` index still exists, and it continues to store multiple saved object types. +Starting on 8.8.0, {{kib}} splits the main saved object index into multiple ones, as depicted in the table above. When upgrading from a previous version, the {{kib}} migration process will reindex some saved objects from the `.kibana` index into the new indices, depending on their types. Note that the `.kibana` index still exists and continues storing multiple saved object types. ## Old {{kib}} indices [upgrade-migrations-old-indices] -As a deployment is gradually upgraded, multiple {{kib}} indices are created in {{es}}: (`.kibana_1`, `.kibana_2`, `.kibana_7.12.0_001`, `.kibana_7.13.0_001`, `.kibana_8.0.0_001` etc). {{kib}} only uses those indices that the *default* and *version* aliases point to. The other, older {{kib}} saved object indices can be safely deleted, but are left around as a matter of historical record, and to facilitate rolling {{kib}} back to a previous version. +As a deployment is gradually upgraded, multiple {{kib}} indices are created in {{es}}: (`.kibana_1`, `.kibana_2`, `.kibana_7.12.0_001`, `.kibana_7.13.0_001`, `.kibana_8.0.0_001`, etc.). {{kib}} only uses those indices that the *default* and *version* aliases point to. The other, older {{kib}} saved object indices can be safely deleted, but are retained for historical records and to facilitate rolling {{kib}} back to a previous version. diff --git a/deploy-manage/upgrade/prepare-to-upgrade/upgrade-assistant.md b/deploy-manage/upgrade/prepare-to-upgrade/upgrade-assistant.md index b7fc01ac59..da4e690788 100644 --- a/deploy-manage/upgrade/prepare-to-upgrade/upgrade-assistant.md +++ b/deploy-manage/upgrade/prepare-to-upgrade/upgrade-assistant.md @@ -1,32 +1,38 @@ --- mapped_pages: - https://www.elastic.co/guide/en/kibana/current/upgrade-assistant.html +applies_to: + stack: + deployment: + eck: + ess: + ece: + self: --- # Upgrade Assistant [upgrade-assistant] -The Upgrade Assistant helps you prepare for your upgrade to the next version of the {{stack}}. To access the assistant, go to **{{stack-manage-app}} > Upgrade Assistant**. - -The assistant identifies deprecated settings in your configuration and guides you through the process of resolving issues if any deprecated features are enabled. +The Upgrade Assistant helps you [prepare to upgrade](/deploy-manage/upgrade/prepare-to-upgrade.md) to the next version of the {{stack}}. To access the assistant, go to **{{stack-manage-app}} → Upgrade Assistant**. +The assistant identifies deprecated settings in your configuration, and if any of those settings are enabled, it guides you through resolving issues that could prevent a successful upgrade. The Upgrade Assistant also helps resolve issues with older indices created before version 8.0.0, providing options to reindex older indices or mark them as read-only. ## Required permissions [_required_permissions_11] -The `manage` cluster privilege is required to access the **Upgrade assistant**. Additional privileges may be needed to perform certain actions. +To access the Upgrade Assistant, you need the `manage` cluster privilege. You may also need additional privileges to perform specific actions. ## Feature set [_feature_set] -Some features of the Upgrade assistant are only needed when upgrading to a new major version. The feature set enabled by default are those for the very next version from the one Kibana currently runs on. +Some features of the Upgrade Assistant are only needed when upgrading to a new major version. The features enabled by default are those for the next version from the one {{kib}} currently runs on. ## Deprecations [_deprecations] -The Upgrade assistant pulls information about deprecations from the following sources: +The Upgrade Assistant pulls information about deprecations from the following sources: -* Elasticsearch Deprecation Info API -* Elasticsearch deprecation logs -* Kibana deprecations API +* {{es}} deprecation info API +* {{es}} deprecation logs +* {{kib}} deprecations API -For more information about Upgrade Assistant APIs, refer to [Upgrade Assistant APIs](https://www.elastic.co/guide/en/kibana/current/upgrade-assistant-api.html). +For more information about Upgrade Assistant APIs, refer to [Upgrade Assistant APIs](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-upgrade). From d80c0e68ea540fb23ccaa7dbf534bcc15946960a Mon Sep 17 00:00:00 2001 From: Janeen Roberts Date: Tue, 1 Apr 2025 22:10:42 -0400 Subject: [PATCH 2/7] Update saved-object-migrations.md --- .../saved-object-migrations.md | 21 +++++++++++-------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md index d2588bcad7..0c357256af 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md +++ b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md @@ -1,6 +1,13 @@ --- mapped_pages: - https://www.elastic.co/guide/en/kibana/current/saved-object-migrations.html +applies_to: + stack: + deployment: + eck: + ess: + ece: + self: --- # Saved object migrations [saved-object-migrations] @@ -8,7 +15,7 @@ mapped_pages: Each time you upgrade {{kib}}, an upgrade migration is performed to ensure that all [saved objects](/explore-analyze/find-and-organize/saved-objects.md) are compatible with the new version. ::::{note} -{{kib}} includes an [**Upgrade Assistant**](../prepare-to-upgrade/upgrade-assistant.md) to help you prepare for an upgrade. To access the assistant, go to **Stack Management > Upgrade Assistant**. +{{kib}} includes an [**Upgrade Assistant**](../prepare-to-upgrade/upgrade-assistant.md) to help you prepare to upgrade. To access the assistant, go to **Stack Management > Upgrade Assistant**. :::: @@ -16,15 +23,10 @@ Each time you upgrade {{kib}}, an upgrade migration is performed to ensure that % {{kib}} 7.12.0 and later uses a new migration process and index naming scheme. % Before you upgrade, read the documentation for your version of {{kib}}. % :::: - -::::{warning} -The `kibana.index` and `xpack.tasks.index` configuration settings are obsolete and no longer considered in 9.x. If you use custom index names, please make the necessary adaptations before upgrading to 9.x. -:::: - - ## How saved object migrations work [upgrade-migrations-process] -When you start a new {{kib}} installation, an upgrade migration is performed before starting plugins or serving HTTP traffic. Before you upgrade, shut down old nodes to prevent losing acknowledged writes. To reduce the likelihood of old nodes losing acknowledged writes, {{kib}} adds a write block to the outdated index. +When you start a new {{kib}} installation, an upgrade migration is performed before starting plugins or serving HTTP traffic. Before you upgrade, shut down old nodes to prevent losing acknowledged writes. +If the upgrade includes breaking changes, the old saved objects will be reindexed into new indices. Otherwise, the existing indices will be reused, and saved object documents will be updated in place. Saved objects are stored in multiple indices. While they all start with the `.kibana*` prefix, other `.kibana*` indices exist but are not used to store saved objects. The following table lists the saved objects indices used by each {{kib}} version. @@ -33,7 +35,8 @@ Saved objects are stored in multiple indices. While they all start with the `.ki | 6.5.0 through 7.3.x | `.kibana_N` | `.kibana` | | 7.4.0 through 7.11.x | `.kibana_N`
`.kibana_task_manager_N` | `.kibana`
`.kibana_task_manager` | | 7.11.x through 8.7.x | `.kibana_{{kibana_version}}_001`
`.kibana_task_manager_{{kibana_version}}_001` | `.kibana`, `.kibana_{{kibana_version}}`
`.kibana_task_manager`, `.kibana_task_manager_{{kibana_version}}` | -| 8.8.0+ | `.kibana_{kibana_version}_001`
`.kibana_alerting_cases_{{kibana_version}}_001`
`.kibana_analytics_{kibana_version}_001`
`.kibana_ingest_{kibana_version}_001`
`.kibana_task_manager_{kibana_version}_001`
`.kibana_security_solution_{kibana_version}_001` | .`kibana`, `.kibana_{kibana_version}`
`.kibana_alerting_cases`,
`.kibana_alerting_cases_{kibana_version}`
`.kibana_analytics`,
`.kibana_analytics_{kibana_version}`
`.kibana_ingest`, `.kibana_ingest_{kibana_version}`
`.kibana_task_manager`,
`.kibana_task_manager_{kibana_version}`
`.kibana_security_solution`,
`.kibana_security_solution_{kibana_version}` +| 8.8.0 through 8.15.x | `.kibana_{kibana_version}_001`
`.kibana_alerting_cases_{{kibana_version}}_001`
`.kibana_analytics_{kibana_version}_001`
`.kibana_ingest_{kibana_version}_001`
`.kibana_task_manager_{kibana_version}_001`
`.kibana_security_solution_{kibana_version}_001` | .`kibana`, `.kibana_{kibana_version}`
`.kibana_alerting_cases`,
`.kibana_alerting_cases_{kibana_version}`
`.kibana_analytics`,
`.kibana_analytics_{kibana_version}`
`.kibana_ingest`, `.kibana_ingest_{kibana_version}`
`.kibana_task_manager`,
`.kibana_task_manager_{kibana_version}`
`.kibana_security_solution`,
`.kibana_security_solution_{kibana_version}` +| 8.16.0+ | `.kibana_usage_counters_{{kibana_version}}_001`| `.kibana_usage_counters_{{kibana_version}}_001`,
`.kibana_usage_counters` Starting on 7.11.0, each of the saved objects indices has a couple of aliases. For example, the `.kibana_8.8.0_001` index has a *default* alias `.kibana` and a *version* alias `.kibana_8.8.0`. The *default* aliases (such as `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version. From eba2a1076ae6717cb7439ba6fff2a8f59812edc1 Mon Sep 17 00:00:00 2001 From: Janeen Roberts Date: Wed, 2 Apr 2025 17:12:39 -0400 Subject: [PATCH 3/7] Splits index aliases and names into two tables. --- .../saved-object-migrations.md | 36 ++++++++++++------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md index 0c357256af..d34788003f 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md +++ b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md @@ -26,29 +26,39 @@ Each time you upgrade {{kib}}, an upgrade migration is performed to ensure that ## How saved object migrations work [upgrade-migrations-process] When you start a new {{kib}} installation, an upgrade migration is performed before starting plugins or serving HTTP traffic. Before you upgrade, shut down old nodes to prevent losing acknowledged writes. -If the upgrade includes breaking changes, the old saved objects will be reindexed into new indices. Otherwise, the existing indices will be reused, and saved object documents will be updated in place. +If the upgrade includes breaking changes, the old saved objects will be reindexed into new indices. Otherwise, the existing indices will be reused, and saved object documents will be updated. -Saved objects are stored in multiple indices. While they all start with the `.kibana*` prefix, other `.kibana*` indices exist but are not used to store saved objects. The following table lists the saved objects indices used by each {{kib}} version. +Saved objects are stored in multiple indices. While they all start with the `.kibana*` prefix, other `.kibana*` indices exist but are not used to store saved objects. The following table lists the saved object indices used by each {{kib}} version. -| Upgrading from version | Index | Aliases | -| --- | --- | --- | -| 6.5.0 through 7.3.x | `.kibana_N` | `.kibana` | -| 7.4.0 through 7.11.x | `.kibana_N`
`.kibana_task_manager_N` | `.kibana`
`.kibana_task_manager` | -| 7.11.x through 8.7.x | `.kibana_{{kibana_version}}_001`
`.kibana_task_manager_{{kibana_version}}_001` | `.kibana`, `.kibana_{{kibana_version}}`
`.kibana_task_manager`, `.kibana_task_manager_{{kibana_version}}` | -| 8.8.0 through 8.15.x | `.kibana_{kibana_version}_001`
`.kibana_alerting_cases_{{kibana_version}}_001`
`.kibana_analytics_{kibana_version}_001`
`.kibana_ingest_{kibana_version}_001`
`.kibana_task_manager_{kibana_version}_001`
`.kibana_security_solution_{kibana_version}_001` | .`kibana`, `.kibana_{kibana_version}`
`.kibana_alerting_cases`,
`.kibana_alerting_cases_{kibana_version}`
`.kibana_analytics`,
`.kibana_analytics_{kibana_version}`
`.kibana_ingest`, `.kibana_ingest_{kibana_version}`
`.kibana_task_manager`,
`.kibana_task_manager_{kibana_version}`
`.kibana_security_solution`,
`.kibana_security_solution_{kibana_version}` -| 8.16.0+ | `.kibana_usage_counters_{{kibana_version}}_001`| `.kibana_usage_counters_{{kibana_version}}_001`,
`.kibana_usage_counters` +| Upgrading from version | Index | +| --- | --- | +| 6.5.0 | `.kibana_N` | +| 7.4.0 | `.kibana_N`
`.kibana_task_manager_N` | +| 7.11 | `.kibana_{{kibana_version}}_001`
`.kibana_task_manager_{{kibana_version}}_001` | +| 8.8.0 | `.kibana_{kibana_version}_001`
`.kibana_alerting_cases_{{kibana_version}}_001`
`.kibana_analytics_{kibana_version}_001`
`.kibana_ingest_{kibana_version}_001`
`.kibana_task_manager_{kibana_version}_001`
`.kibana_security_solution_{kibana_version}_001` | +| 8.16.0 | `.kibana_usage_counters_{{kibana_version}}_001`| -Starting on 7.11.0, each of the saved objects indices has a couple of aliases. For example, the `.kibana_8.8.0_001` index has a *default* alias `.kibana` and a *version* alias `.kibana_8.8.0`. The *default* aliases (such as `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version. +Starting with 7.11.0, each of the saved objects indices has a couple of aliases. For example, the `.kibana_8.8.0_001` index has a *default* alias `.kibana` and a *version* alias `.kibana_8.8.0`. The *default* aliases (such as `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version. -Starting on 8.6.0, index names aren’t necessarily aligned with the deployed {{kib}} version. When updates on a certain index are compatible, {{kib}} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for an 8.8.0 deployment: +In versions 8.6.0 and newer, index names aren’t necessarily aligned with the deployed {{kib}} version. When updates on a certain index are compatible, {{kib}} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for an 8.8.0 deployment: * `.kibana_8.8.0_001` index, with `.kibana` and `.kibana_8.8.0` aliases. * `.kibana_task_manager_8.7.0_001` index, with `.kibana_task_manager` and `.kibana_task_manager_8.8.0` aliases. -Starting on 8.8.0, {{kib}} splits the main saved object index into multiple ones, as depicted in the table above. When upgrading from a previous version, the {{kib}} migration process will reindex some saved objects from the `.kibana` index into the new indices, depending on their types. Note that the `.kibana` index still exists and continues storing multiple saved object types. +Starting with 8.8.0, {{kib}} splits the main saved object index into multiple ones, as depicted in the table above. When upgrading from a previous version, the {{kib}} migration process will reindex some saved objects from the `.kibana` index into the new indices, depending on their types. Note that the `.kibana` index still exists and continues storing multiple saved object types. + +The following table lists the saved object indices, their relative alias(es), and the version range those aliases apply to. + +| Version range | Index | Aliases | +| --- | --- | --- | +| 6.5.0 through 7.3.x | `.kibana_N` | `.kibana` | +| 7.4.0 through 7.11.x | `.kibana_N`
`.kibana_task_manager_N` | `.kibana`
`.kibana_task_manager` | +| 7.11.x through 8.7.x | `.kibana_{{kibana_version}}_001`
`.kibana_task_manager_{{kibana_version}}_001` | `.kibana`, `.kibana_{{kibana_version}}`
`.kibana_task_manager`, `.kibana_task_manager_{{kibana_version}}` | +| 8.8.0 through 8.15.x | `.kibana_{kibana_version}_001`
`.kibana_alerting_cases_{{kibana_version}}_001`
`.kibana_analytics_{kibana_version}_001`
`.kibana_ingest_{kibana_version}_001`
`.kibana_task_manager_{kibana_version}_001`
`.kibana_security_solution_{kibana_version}_001` | .`kibana`, `.kibana_{kibana_version}`
`.kibana_alerting_cases`,
`.kibana_alerting_cases_{kibana_version}`
`.kibana_analytics`,
`.kibana_analytics_{kibana_version}`
`.kibana_ingest`, `.kibana_ingest_{kibana_version}`
`.kibana_task_manager`,
`.kibana_task_manager_{kibana_version}`
`.kibana_security_solution`,
`.kibana_security_solution_{kibana_version}` +| 8.16.0+ | `.kibana_usage_counters_{{kibana_version}}_001`| `.kibana_usage_counters_{{kibana_version}}_001`,
`.kibana_usage_counters` ## Old {{kib}} indices [upgrade-migrations-old-indices] -As a deployment is gradually upgraded, multiple {{kib}} indices are created in {{es}}: (`.kibana_1`, `.kibana_2`, `.kibana_7.12.0_001`, `.kibana_7.13.0_001`, `.kibana_8.0.0_001`, etc.). {{kib}} only uses those indices that the *default* and *version* aliases point to. The other, older {{kib}} saved object indices can be safely deleted, but are retained for historical records and to facilitate rolling {{kib}} back to a previous version. +When you upgrade a deployment, Elastic creates multiple {{kib}} indices in {{es}}: (`.kibana_1`, `.kibana_2`, `.kibana_7.12.0_001`, `.kibana_7.13.0_001`, `.kibana_8.0.0_001`, etc.). {{kib}} only uses those indices that the *default* and *version* aliases point to. You can safely delete the older {{kib}} saved object indices, but they're retained for historical records and to facilitate rolling {{kib}} back to a previous version. From 1dd1a43d1e2c8cbe61206d558fc6c11a264c4573 Mon Sep 17 00:00:00 2001 From: Gerard Soldevila Date: Mon, 7 Apr 2025 15:45:26 +0200 Subject: [PATCH 4/7] Rewrite evolution of saved object indices --- .../saved-object-migrations.md | 76 +++++++++++++------ 1 file changed, 52 insertions(+), 24 deletions(-) diff --git a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md index d34788003f..db7a1cde00 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md +++ b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md @@ -14,51 +14,79 @@ applies_to: Each time you upgrade {{kib}}, an upgrade migration is performed to ensure that all [saved objects](/explore-analyze/find-and-organize/saved-objects.md) are compatible with the new version. -::::{note} +::::{note} {{kib}} includes an [**Upgrade Assistant**](../prepare-to-upgrade/upgrade-assistant.md) to help you prepare to upgrade. To access the assistant, go to **Stack Management > Upgrade Assistant**. :::: -% ::::{warning} +% ::::{warning} % {{kib}} 7.12.0 and later uses a new migration process and index naming scheme. % Before you upgrade, read the documentation for your version of {{kib}}. % :::: -## How saved object migrations work [upgrade-migrations-process] +## How saved object migrations work [upgrade-migrations-process] When you start a new {{kib}} installation, an upgrade migration is performed before starting plugins or serving HTTP traffic. Before you upgrade, shut down old nodes to prevent losing acknowledged writes. If the upgrade includes breaking changes, the old saved objects will be reindexed into new indices. Otherwise, the existing indices will be reused, and saved object documents will be updated. -Saved objects are stored in multiple indices. While they all start with the `.kibana*` prefix, other `.kibana*` indices exist but are not used to store saved objects. The following table lists the saved object indices used by each {{kib}} version. +Saved objects are stored in multiple indices. While they all start with the `.kibana*` prefix, other `.kibana*` indices exist but are not used to store saved objects. +The indices used to store saved objects and the conventions on their names and aliases have evolved as follows: -| Upgrading from version | Index | -| --- | --- | -| 6.5.0 | `.kibana_N` | -| 7.4.0 | `.kibana_N`
`.kibana_task_manager_N` | -| 7.11 | `.kibana_{{kibana_version}}_001`
`.kibana_task_manager_{{kibana_version}}_001` | -| 8.8.0 | `.kibana_{kibana_version}_001`
`.kibana_alerting_cases_{{kibana_version}}_001`
`.kibana_analytics_{kibana_version}_001`
`.kibana_ingest_{kibana_version}_001`
`.kibana_task_manager_{kibana_version}_001`
`.kibana_security_solution_{kibana_version}_001` | -| 8.16.0 | `.kibana_usage_counters_{{kibana_version}}_001`| +### Kibana 6.5.0 -Starting with 7.11.0, each of the saved objects indices has a couple of aliases. For example, the `.kibana_8.8.0_001` index has a *default* alias `.kibana` and a *version* alias `.kibana_8.8.0`. The *default* aliases (such as `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version. +The `.kibana_N` saved object index was introduced. The `N` suffix was incremented each time an upgrade was performed. At that point, a `.kibana` alias was created / updated to point to the latest version of the index. -In versions 8.6.0 and newer, index names aren’t necessarily aligned with the deployed {{kib}} version. When updates on a certain index are compatible, {{kib}} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for an 8.8.0 deployment: +### Kibana 7.4.0 -* `.kibana_8.8.0_001` index, with `.kibana` and `.kibana_8.8.0` aliases. -* `.kibana_task_manager_8.7.0_001` index, with `.kibana_task_manager` and `.kibana_task_manager_8.8.0` aliases. +A new `.kibana_task_manager_N` was created, which would host the `task` saved objects. It also had a `.kibana_task_manager` alias, pointing to the latest version. + +### Kibana 7.11.0 + +Starting with 7.11.0 the naming convention evolved, and the indices names started containing the version number, along with a `_001` suffix. +Each of the saved objects indices now has a couple of aliases. For example, the `.kibana_7.11.0_001` index has a *default* `.kibana` alias and a `.kibana_7.11.0` *version* alias. The *default* aliases (such as `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version. +A new versioned alias was also introduced: + +| Alias | Version alias | Index name | +| --- | --- | --- | +| .kibana | .kibana_7.11.0 | .kibana_7.11.0_001 | +| .kibana_task_manager | .kibana_task_manager_7.11.0 | .kibana_task_manager_7.11.0_001 | + +### Kibana 8.6.0 + +In versions 8.6.0 and newer, compatible migrations were introduced, allowing to reuse existing saved object indices as long as changes in the mappings were compatible. After this change, index names aren’t necessarily aligned with the deployed {{kib}} version. When updates on a certain index are compatible, {{kib}} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for an 8.7.0 deployment: + +| Alias | Versioned alias | Index name | +| --- | --- | --- | +| .kibana | .kibana_8.7.0 | .kibana_8.7.0_001 | +| .kibana_task_manager | .kibana_task_manager_8.7.0 | .kibana_task_manager_7.17.0_001 | + +### Kibana 8.8.0 Starting with 8.8.0, {{kib}} splits the main saved object index into multiple ones, as depicted in the table above. When upgrading from a previous version, the {{kib}} migration process will reindex some saved objects from the `.kibana` index into the new indices, depending on their types. Note that the `.kibana` index still exists and continues storing multiple saved object types. -The following table lists the saved object indices, their relative alias(es), and the version range those aliases apply to. +| Alias | Version alias | Index name | +| --- | --- | --- | +| .kibana | .kibana_8.8.0 | .kibana_8.8.0_001 | +| .kibana_task_manager | .kibana_task_manager_8.8.0 | .kibana_task_manager_7.17.0_001 | +| .kibana_alerting_cases | .kibana_alerting_cases_8.8.0 | .kibana_alerting_cases_8.8.0_001 | +| .kibana_analytics | .kibana_analytics_8.8.0 | .kibana_analytics_8.8.0_001 | +| .kibana_ingest | .kibana_ingest_8.8.0 | .kibana_ingest_8.8.0_001 | +| .kibana_security_solution | .kibana_security_solution_8.8.0 | .kibana_security_solution_8.8.0_001 | + +### Kibana 8.16.0 and newer + +A new index was introduced, with the goal of storing the `usage-counter` saved object type, which is expected to have a large number of documents. -| Version range | Index | Aliases | +| Alias | Version alias | Index name | | --- | --- | --- | -| 6.5.0 through 7.3.x | `.kibana_N` | `.kibana` | -| 7.4.0 through 7.11.x | `.kibana_N`
`.kibana_task_manager_N` | `.kibana`
`.kibana_task_manager` | -| 7.11.x through 8.7.x | `.kibana_{{kibana_version}}_001`
`.kibana_task_manager_{{kibana_version}}_001` | `.kibana`, `.kibana_{{kibana_version}}`
`.kibana_task_manager`, `.kibana_task_manager_{{kibana_version}}` | -| 8.8.0 through 8.15.x | `.kibana_{kibana_version}_001`
`.kibana_alerting_cases_{{kibana_version}}_001`
`.kibana_analytics_{kibana_version}_001`
`.kibana_ingest_{kibana_version}_001`
`.kibana_task_manager_{kibana_version}_001`
`.kibana_security_solution_{kibana_version}_001` | .`kibana`, `.kibana_{kibana_version}`
`.kibana_alerting_cases`,
`.kibana_alerting_cases_{kibana_version}`
`.kibana_analytics`,
`.kibana_analytics_{kibana_version}`
`.kibana_ingest`, `.kibana_ingest_{kibana_version}`
`.kibana_task_manager`,
`.kibana_task_manager_{kibana_version}`
`.kibana_security_solution`,
`.kibana_security_solution_{kibana_version}` -| 8.16.0+ | `.kibana_usage_counters_{{kibana_version}}_001`| `.kibana_usage_counters_{{kibana_version}}_001`,
`.kibana_usage_counters` +| .kibana | .kibana_8.8.0 | .kibana_8.8.0_001 | +| .kibana_task_manager | .kibana_task_manager_8.8.0 | .kibana_task_manager_7.17.0_001 | +| .kibana_alerting_cases | .kibana_alerting_cases_8.8.0 | .kibana_alerting_cases_8.8.0_001 | +| .kibana_analytics | .kibana_analytics_8.8.0 | .kibana_analytics_8.8.0_001 | +| .kibana_ingest | .kibana_ingest_8.8.0 | .kibana_ingest_8.8.0_001 | +| .kibana_security_solution | .kibana_security_solution_8.8.0 | .kibana_security_solution_8.8.0_001 | +| .kibana_usage_counters | .kibana_usage_counters_8.8.0 | .kibana_usage_counters_8.8.0_001 | -## Old {{kib}} indices [upgrade-migrations-old-indices] +## Old {{kib}} indices [upgrade-migrations-old-indices] When you upgrade a deployment, Elastic creates multiple {{kib}} indices in {{es}}: (`.kibana_1`, `.kibana_2`, `.kibana_7.12.0_001`, `.kibana_7.13.0_001`, `.kibana_8.0.0_001`, etc.). {{kib}} only uses those indices that the *default* and *version* aliases point to. You can safely delete the older {{kib}} saved object indices, but they're retained for historical records and to facilitate rolling {{kib}} back to a previous version. - From cb0c0221794bdb943bee64e2ab83797690152eb7 Mon Sep 17 00:00:00 2001 From: Gerard Soldevila Date: Mon, 7 Apr 2025 15:49:42 +0200 Subject: [PATCH 5/7] Polish some bits --- .../upgrade/deployment-or-cluster/saved-object-migrations.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md index db7a1cde00..e54db0fc2f 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md +++ b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md @@ -37,13 +37,12 @@ The `.kibana_N` saved object index was introduced. The `N` suffix was incremente ### Kibana 7.4.0 -A new `.kibana_task_manager_N` was created, which would host the `task` saved objects. It also had a `.kibana_task_manager` alias, pointing to the latest version. +A new `.kibana_task_manager_N` was created, which would host the `task` saved objects. It also had a `.kibana_task_manager` alias, pointing to the latest version of the index. ### Kibana 7.11.0 Starting with 7.11.0 the naming convention evolved, and the indices names started containing the version number, along with a `_001` suffix. -Each of the saved objects indices now has a couple of aliases. For example, the `.kibana_7.11.0_001` index has a *default* `.kibana` alias and a `.kibana_7.11.0` *version* alias. The *default* aliases (such as `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version. -A new versioned alias was also introduced: +Each of the saved objects indices now has a couple of aliases. For example, the `.kibana_7.11.0_001` index has a *default* `.kibana` alias and a `.kibana_7.11.0` *version* alias. The *default* aliases (such as `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version: | Alias | Version alias | Index name | | --- | --- | --- | From bf86bb2384047fb06addba08a5f0ecf5315fc48d Mon Sep 17 00:00:00 2001 From: Gerard Soldevila Date: Mon, 7 Apr 2025 15:50:59 +0200 Subject: [PATCH 6/7] above -> below --- .../upgrade/deployment-or-cluster/saved-object-migrations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md index e54db0fc2f..7f2af87c29 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md +++ b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md @@ -60,7 +60,7 @@ In versions 8.6.0 and newer, compatible migrations were introduced, allowing to ### Kibana 8.8.0 -Starting with 8.8.0, {{kib}} splits the main saved object index into multiple ones, as depicted in the table above. When upgrading from a previous version, the {{kib}} migration process will reindex some saved objects from the `.kibana` index into the new indices, depending on their types. Note that the `.kibana` index still exists and continues storing multiple saved object types. +Starting with 8.8.0, {{kib}} splits the main saved object index into multiple ones, as depicted in the table below. When upgrading from a previous version, the {{kib}} migration process will reindex some saved objects from the `.kibana` index into the new indices, depending on their types. Note that the `.kibana` index still exists and continues storing multiple saved object types. | Alias | Version alias | Index name | | --- | --- | --- | From aebfe97e1f21bb19a5ee2f42656728ddc0579e0b Mon Sep 17 00:00:00 2001 From: Gerard Soldevila Date: Tue, 8 Apr 2025 09:38:10 +0200 Subject: [PATCH 7/7] Uniformise to present tense --- .../deployment-or-cluster/saved-object-migrations.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md index 7f2af87c29..baf5fb8b47 100644 --- a/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md +++ b/deploy-manage/upgrade/deployment-or-cluster/saved-object-migrations.md @@ -33,15 +33,15 @@ The indices used to store saved objects and the conventions on their names and a ### Kibana 6.5.0 -The `.kibana_N` saved object index was introduced. The `N` suffix was incremented each time an upgrade was performed. At that point, a `.kibana` alias was created / updated to point to the latest version of the index. +The `.kibana_N` saved object index is created. Saved objects are reindexed into a new index, and the `N` suffix is incremented each time an upgrade is performed. A `.kibana` alias is maintained, which points to the latest version of the index. ### Kibana 7.4.0 -A new `.kibana_task_manager_N` was created, which would host the `task` saved objects. It also had a `.kibana_task_manager` alias, pointing to the latest version of the index. +A new `.kibana_task_manager_N` is introduced, which hosts the `task` saved objects. It also has the corresponding `.kibana_task_manager` alias, pointing to the latest version of the index. ### Kibana 7.11.0 -Starting with 7.11.0 the naming convention evolved, and the indices names started containing the version number, along with a `_001` suffix. +Starting with 7.11.0 the naming convention evolves, and the indices names contain the version number, along with a `_001` suffix. Each of the saved objects indices now has a couple of aliases. For example, the `.kibana_7.11.0_001` index has a *default* `.kibana` alias and a `.kibana_7.11.0` *version* alias. The *default* aliases (such as `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version: | Alias | Version alias | Index name | @@ -51,7 +51,7 @@ Each of the saved objects indices now has a couple of aliases. For example, the ### Kibana 8.6.0 -In versions 8.6.0 and newer, compatible migrations were introduced, allowing to reuse existing saved object indices as long as changes in the mappings were compatible. After this change, index names aren’t necessarily aligned with the deployed {{kib}} version. When updates on a certain index are compatible, {{kib}} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for an 8.7.0 deployment: +In versions 8.6.0 and newer, compatible migrations are introduced, allowing to reuse existing saved object indices as long as changes in the mappings are compatible. After this change, index names aren’t necessarily aligned with the deployed {{kib}} version. When updates on a certain index are compatible, {{kib}} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for an 8.7.0 deployment: | Alias | Versioned alias | Index name | | --- | --- | --- | @@ -73,7 +73,7 @@ Starting with 8.8.0, {{kib}} splits the main saved object index into multiple on ### Kibana 8.16.0 and newer -A new index was introduced, with the goal of storing the `usage-counter` saved object type, which is expected to have a large number of documents. +A new index is introduced, with the goal of storing the `usage-counter` saved object type, which is expected to have a large number of documents. | Alias | Version alias | Index name | | --- | --- | --- |