Skip to content
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,52 +1,91 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/kibana/current/saved-object-migrations.html
applies_to:
stack:
deployment:
eck:
ess:
ece:
self:
---

# Saved object migrations [saved-object-migrations]

Each time you upgrade {{kib}}, an upgrade migration is performed to ensure that all [saved objects](/explore-analyze/find-and-organize/saved-objects.md) are compatible with the new version.

::::{note}
{{kib}} includes an [**Upgrade Assistant**](../prepare-to-upgrade/upgrade-assistant.md) to help you prepare for an upgrade. To access the assistant, go to **Stack Management > Upgrade Assistant**.
::::{note}
{{kib}} includes an [**Upgrade Assistant**](../prepare-to-upgrade/upgrade-assistant.md) to help you prepare to upgrade. To access the assistant, go to **Stack Management > Upgrade Assistant**.
::::


::::{warning}
{{kib}} 7.12.0 and later uses a new migration process and index naming scheme. Before you upgrade, read the documentation for your version of {{kib}}.
::::
% ::::{warning}
% {{kib}} 7.12.0 and later uses a new migration process and index naming scheme. % Before you upgrade, read the documentation for your version of {{kib}}.
% ::::

## How saved object migrations work [upgrade-migrations-process]

::::{warning}
The `kibana.index` and `xpack.tasks.index` configuration settings are obsolete and no longer taken into account in 8.x. If you are using custom index names, please perform the necessary adaptations before attempting to upgrade to 8.x.
::::
When you start a new {{kib}} installation, an upgrade migration is performed before starting plugins or serving HTTP traffic. Before you upgrade, shut down old nodes to prevent losing acknowledged writes.
If the upgrade includes breaking changes, the old saved objects will be reindexed into new indices. Otherwise, the existing indices will be reused, and saved object documents will be updated.

Saved objects are stored in multiple indices. While they all start with the `.kibana*` prefix, other `.kibana*` indices exist but are not used to store saved objects.
The indices used to store saved objects and the conventions on their names and aliases have evolved as follows:

### Kibana 6.5.0

The `.kibana_N` saved object index is created. Saved objects are reindexed into a new index, and the `N` suffix is incremented each time an upgrade is performed. A `.kibana` alias is maintained, which points to the latest version of the index.

## How saved objects migrations work [upgrade-migrations-process]
### Kibana 7.4.0

When you start a new {{kib}} installation, an upgrade migration is performed before starting plugins or serving HTTP traffic. Before you upgrade, shut down old nodes to prevent losing acknowledged writes. To reduce the likelihood of old nodes losing acknowledged writes, {{kib}} 7.12.0 and later adds a write block to the outdated index.
A new `.kibana_task_manager_N` is introduced, which hosts the `task` saved objects. It also has the corresponding `.kibana_task_manager` alias, pointing to the latest version of the index.

Saved objects are stored in multiple indices. Whilst all of them start with the `.kibana*` prefix, other `.kibana*` indices exist, which are not used to store saved objects. The following tables lists the saved objects indices used by each {{kib}} version.
### Kibana 7.11.0

| Upgrading from version | Index | Aliases |
Starting with 7.11.0 the naming convention evolves, and the indices names contain the version number, along with a `_001` suffix.
Each of the saved objects indices now has a couple of aliases. For example, the `.kibana_7.11.0_001` index has a *default* `.kibana` alias and a `.kibana_7.11.0` *version* alias. The *default* aliases (such as `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version:

| Alias | Version alias | Index name |
| --- | --- | --- |
| 6.5.0 through 7.3.x | `.kibana_N` | `.kibana` |
| 7.4.0 through 7.11.x | `.kibana_N`<br>`.kibana_task_manager_N` | `.kibana`<br>`.kibana_task_manager` |
| 7.11.x through 8.7.x | `.kibana_{{kibana_version}}_001`<br>`.kibana_task_manager_{{kibana_version}}_001` | `.kibana`, `.kibana_{{kibana_version}}`<br>`.kibana_task_manager`, `.kibana_task_manager_{{kibana_version}}` |
| 8.8.0+ | `.kibana_{{kibana_version}}_001`<br>`.kibana_alerting_cases_{{kibana_version}}_001``.kibana_analytics_{{kibana_version}}_001``.kibana_ingest_{{kibana_version}}_001``.kibana_task_manager_{{kibana_version}}_001``.kibana_security_solution_{{kibana_version}}_001` | `.kibana`, `.kibana_{{kibana_version}}`<br>`.kibana_alerting_cases`, `.kibana_alerting_cases_{{kibana_version}}``.kibana_analytics`, `.kibana_analytics_{{kibana_version}}``.kibana_ingest`, `.kibana_ingest_{{kibana_version}}``.kibana_task_manager`, `.kibana_task_manager_{{kibana_version}}``.kibana_security_solution`, `.kibana_security_solution_{{kibana_version}}` |
| .kibana | .kibana_7.11.0 | .kibana_7.11.0_001 |
| .kibana_task_manager | .kibana_task_manager_7.11.0 | .kibana_task_manager_7.11.0_001 |

### Kibana 8.6.0

Starting on 7.11.0, each of the saved objects indices has a couple of aliases, e.g. the `.kibana_8.8.0_001` index has a *default* alias `.kibana` and a *version* alias `.kibana_8.8.0`. The *default* aliases (e.g. `.kibana` and `.kibana_task_manager`) always point to the most up-to-date saved object indices. Then, *version* aliases are aligned with the deployed {{kib}} version.
In versions 8.6.0 and newer, compatible migrations are introduced, allowing to reuse existing saved object indices as long as changes in the mappings are compatible. After this change, index names aren’t necessarily aligned with the deployed {{kib}} version. When updates on a certain index are compatible, {{kib}} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for an 8.7.0 deployment:

Starting on 8.6.0, index names aren’t necessarily aligned with the deployed {{kib}} version. When updates on a certain index are compatible, {{kib}} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for a 8.8.0 deployment:
| Alias | Versioned alias | Index name |
| --- | --- | --- |
| .kibana | .kibana_8.7.0 | .kibana_8.7.0_001 |
| .kibana_task_manager | .kibana_task_manager_8.7.0 | .kibana_task_manager_7.17.0_001 |

* `.kibana_8.8.0_001` index, with `.kibana` and `.kibana_8.8.0` aliases.
* `.kibana_task_manager_8.7.0_001` index, with `.kibana_task_manager` and `.kibana_task_manager_8.8.0` aliases.
### Kibana 8.8.0

Starting on 8.8.0, {{kib}} splits the main saved object index into multiple ones, as depicted on the table above. When upgrading from a previous version, the {{kib}} migration process will reindex some saved objects from the `.kibana` index into the new indices, depending on their types. Note that the `.kibana` index still exists, and it continues to store multiple saved object types.
Starting with 8.8.0, {{kib}} splits the main saved object index into multiple ones, as depicted in the table below. When upgrading from a previous version, the {{kib}} migration process will reindex some saved objects from the `.kibana` index into the new indices, depending on their types. Note that the `.kibana` index still exists and continues storing multiple saved object types.

| Alias | Version alias | Index name |
| --- | --- | --- |
| .kibana | .kibana_8.8.0 | .kibana_8.8.0_001 |
| .kibana_task_manager | .kibana_task_manager_8.8.0 | .kibana_task_manager_7.17.0_001 |
| .kibana_alerting_cases | .kibana_alerting_cases_8.8.0 | .kibana_alerting_cases_8.8.0_001 |
| .kibana_analytics | .kibana_analytics_8.8.0 | .kibana_analytics_8.8.0_001 |
| .kibana_ingest | .kibana_ingest_8.8.0 | .kibana_ingest_8.8.0_001 |
| .kibana_security_solution | .kibana_security_solution_8.8.0 | .kibana_security_solution_8.8.0_001 |

### Kibana 8.16.0 and newer

A new index is introduced, with the goal of storing the `usage-counter` saved object type, which is expected to have a large number of documents.

| Alias | Version alias | Index name |
| --- | --- | --- |
| .kibana | .kibana_8.8.0 | .kibana_8.8.0_001 |
| .kibana_task_manager | .kibana_task_manager_8.8.0 | .kibana_task_manager_7.17.0_001 |
| .kibana_alerting_cases | .kibana_alerting_cases_8.8.0 | .kibana_alerting_cases_8.8.0_001 |
| .kibana_analytics | .kibana_analytics_8.8.0 | .kibana_analytics_8.8.0_001 |
| .kibana_ingest | .kibana_ingest_8.8.0 | .kibana_ingest_8.8.0_001 |
| .kibana_security_solution | .kibana_security_solution_8.8.0 | .kibana_security_solution_8.8.0_001 |
| .kibana_usage_counters | .kibana_usage_counters_8.8.0 | .kibana_usage_counters_8.8.0_001 |

## Old {{kib}} indices [upgrade-migrations-old-indices]

As a deployment is gradually upgraded, multiple {{kib}} indices are created in {{es}}: (`.kibana_1`, `.kibana_2`, `.kibana_7.12.0_001`, `.kibana_7.13.0_001`, `.kibana_8.0.0_001` etc). {{kib}} only uses those indices that the *default* and *version* aliases point to. The other, older {{kib}} saved object indices can be safely deleted, but are left around as a matter of historical record, and to facilitate rolling {{kib}} back to a previous version.
## Old {{kib}} indices [upgrade-migrations-old-indices]

When you upgrade a deployment, Elastic creates multiple {{kib}} indices in {{es}}: (`.kibana_1`, `.kibana_2`, `.kibana_7.12.0_001`, `.kibana_7.13.0_001`, `.kibana_8.0.0_001`, etc.). {{kib}} only uses those indices that the *default* and *version* aliases point to. You can safely delete the older {{kib}} saved object indices, but they're retained for historical records and to facilitate rolling {{kib}} back to a previous version.
26 changes: 16 additions & 10 deletions deploy-manage/upgrade/prepare-to-upgrade/upgrade-assistant.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,38 @@
---
mapped_pages:
- https://www.elastic.co/guide/en/kibana/current/upgrade-assistant.html
applies_to:
stack:
deployment:
eck:
ess:
ece:
self:
---

# Upgrade Assistant [upgrade-assistant]

The Upgrade Assistant helps you prepare for your upgrade to the next version of the {{stack}}. To access the assistant, go to **{{stack-manage-app}} > Upgrade Assistant**.

The assistant identifies deprecated settings in your configuration and guides you through the process of resolving issues if any deprecated features are enabled.
The Upgrade Assistant helps you [prepare to upgrade](/deploy-manage/upgrade/prepare-to-upgrade.md) to the next version of the {{stack}}. To access the assistant, go to **{{stack-manage-app}} → Upgrade Assistant**.

The assistant identifies deprecated settings in your configuration, and if any of those settings are enabled, it guides you through resolving issues that could prevent a successful upgrade. The Upgrade Assistant also helps resolve issues with older indices created before version 8.0.0, providing options to reindex older indices or mark them as read-only.

## Required permissions [_required_permissions_11]

The `manage` cluster privilege is required to access the **Upgrade assistant**. Additional privileges may be needed to perform certain actions.
To access the Upgrade Assistant, you need the `manage` cluster privilege. You may also need additional privileges to perform specific actions.


## Feature set [_feature_set]

Some features of the Upgrade assistant are only needed when upgrading to a new major version. The feature set enabled by default are those for the very next version from the one Kibana currently runs on.
Some features of the Upgrade Assistant are only needed when upgrading to a new major version. The features enabled by default are those for the next version from the one {{kib}} currently runs on.


## Deprecations [_deprecations]

The Upgrade assistant pulls information about deprecations from the following sources:
The Upgrade Assistant pulls information about deprecations from the following sources:

* Elasticsearch Deprecation Info API
* Elasticsearch deprecation logs
* Kibana deprecations API
* {{es}} deprecation info API
* {{es}} deprecation logs
* {{kib}} deprecations API

For more information about Upgrade Assistant APIs, refer to [Upgrade Assistant APIs](https://www.elastic.co/guide/en/kibana/current/upgrade-assistant-api.html).
For more information about Upgrade Assistant APIs, refer to [Upgrade Assistant APIs](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-upgrade).

Loading