You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: deploy-manage/deploy/elastic-cloud/google-cloud-platform-marketplace.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,20 +4,20 @@ mapped_pages:
4
4
applies_to:
5
5
deployment:
6
6
ess: ga
7
-
serverless: unavailable
7
+
serverless: ga
8
8
products:
9
9
- id: cloud-hosted
10
+
- id: cloud-serverless
10
11
---
11
12
12
13
# Google Cloud Platform Marketplace [ec-billing-gcp]
13
14
14
-
Subscribe to {{ecloud}} directly from the Google Cloud Platform (GCP). You then have the convenience of viewing your {{ecloud}} subscription as part of your GCP bill, and you do not have to supply any additional credit card information to Elastic.
15
+
Subscribe to {{ecloud}} directly from the Google Cloud Platform (GCP). You then have the convenience of viewing your {{ecloud}} subscription as part of your GCP bill, and you do not have to supply any additional credit card information to Elastic. Your investment in Elastic draws against your cloud purchase commitment.
15
16
16
17
Some differences exist when you subscribe to {{ecloud}} through the GCP Marketplace:
17
18
18
-
* There is no trial period. Billing starts when you subscribe to {{ecloud}}.
19
-
* Existing {{ecloud}} organizations cannot be converted to use the GCP Marketplace.
20
-
* Pricing for an {{ecloud}} subscription through the GCP Marketplace follows the pricing outlined on the [{{ecloud}}](https://console.cloud.google.com/marketplace/product/endpoints/elasticsearch-service.gcpmarketplace.elastic.co) page in the GCP Marketplace. Pricing is based the {{ecloud}} [Billing Dimensions](../../cloud-organization/billing/cloud-hosted-deployment-billing-dimensions.md).
19
+
* New customers {{ecloud}} obtain a 7-day trial period. During this period, you can use a single deployment and three projects of {{ecloud}}. After this period, usage-based billing starts, unless you delete your cloud resources. Note that once customers unsubscribe from the GCP offer, their trial will end immediately. Even if they resubscribe, they will not be able to resume the trial.
20
+
* Pricing for an {{ecloud}} subscription through the GCP Marketplace follows the pricing outlined on the [{{ecloud}}](https://console.cloud.google.com/marketplace/product/endpoints/elasticsearch-service.gcpmarketplace.elastic.co) page in the GCP Marketplace. Pricing is based the {{ecloud}} [billing dimensions](../../cloud-organization/billing.md#pricing-model).
21
21
* To access your billing information at any time go to **Account & Billing**. You can also go to **Account & Billing** and then **Usage** to view your usage hours and units per hour.
22
22
23
23
::::{important}
@@ -47,15 +47,15 @@ To subscribe to {{ecloud}} through the GCP Marketplace:
47
47
You are ready to [create your first deployment](create-an-elastic-cloud-hosted-deployment.md).
48
48
49
49
50
-
If you have existing deployments that you want to migrate to your new marketplace account, we recommend using a custom repository to take a snapshot. Then restore that snapshot to a new deployment in your new marketplace account. Check [Snapshot and restore with custom repositories](../../tools/snapshot-and-restore/elastic-cloud-hosted.md) for details.
50
+
If you plan to use {{ech}} and have existing deployments that you want to migrate to your new marketplace account, we recommend using a custom repository to take a snapshot. Then restore that snapshot to a new deployment in your new marketplace account. Check [Snapshot and restore with custom repositories](../../tools/snapshot-and-restore/elastic-cloud-hosted.md) for details.
51
51
52
52
::::{tip}
53
53
Your new account is automatically subscribed to the Enterprise subscription level. You can [change your subscription level](../../cloud-organization/billing/manage-subscription.md).
54
54
::::
55
55
56
56
57
57
58
-
## Changes to your Billing Account[ec-billing-gcp-account-change]
58
+
## Changes to your billing account[ec-billing-gcp-account-change]
59
59
60
60
::::{important}
61
61
To prevent downtime, do not remove the currently used billing account before the switch to the new billing account has been confirmed by Elastic.
Copy file name to clipboardExpand all lines: manage-data/lifecycle/index-lifecycle-management/rollover.md
+74-27Lines changed: 74 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,50 +9,97 @@ products:
9
9
10
10
# Rollover [index-rollover]
11
11
12
-
When indexing time series data like logs or metrics, you can’t write to a single index indefinitely. To meet your indexing and search performance requirements and manage resource usage, you write to an index until some threshold is met and then create a new index and start writing to it instead. Using rolling indices enables you to:
12
+
In {{es}}, the [rollover action](elasticsearch://reference/elasticsearch/index-lifecycle-actions/ilm-rollover.md) replaces your active write index with a new one whenever your index grows beyond a specified size, age, or number of documents.
13
+
This is particularly useful for time-series data, such as logs or metrics where index growth is continuous, in order to meet performance and retention requirements.
13
14
14
-
* Optimize the active index for high ingest rates on high-performance *hot* nodes.
15
-
* Optimize for search performance on *warm* nodes.
16
-
* Shift older, less frequently accessed data to less expensive *cold* nodes,
17
-
* Delete data according to your retention policies by removing entire indices.
15
+
Without rollover, a single index would continue to grow, causing search performance to drop and having a higher administrative burden on the cluster.
18
16
19
-
We recommend using [data streams](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-create-data-stream)to manage time series data. Data streams automatically track the write index while keeping configuration to a minimum.
17
+
The rollover feature is an important part of how [index lifecycle](../index-lifecycle-management/index-lifecycle.md) ({{ilm-init}}) and [data stream lifecycles](../data-stream.md) ({{dlm-init}}) work to keep your indices fast and manageable. By switching the write target of an index, the rollover action provides the following benefits:
20
18
21
-
Each data stream requires an [index template](../../data-store/templates.md) that contains:
19
+
***Lifecycle** - works with lifecycle management ({{ilm-init}} or {{dlm-init}}) to transition the index through its lifecycle actions and allows for granular control over retention cycles
***Queries run faster** - improves search performance
22
22
23
-
* A name or wildcard (`*`) pattern for the data stream.
24
-
* The data stream’s timestamp field. This field must be mapped as a [`date`](elasticsearch://reference/elasticsearch/mapping-reference/date.md) or [`date_nanos`](elasticsearch://reference/elasticsearch/mapping-reference/date_nanos.md) field data type and must be included in every document indexed to the data stream.
25
-
* The mappings and settings applied to each backing index when it’s created.
23
+
Rollover can be triggered via the [API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover), {{ilm-init}}, or {{dlm-init}}.
26
24
27
-
Data streams are designed for append-only data, where the data stream name can be used as the operations (read, write, rollover, shrink etc.) target. If your use case requires data to be updated in place, you can instead manage your time series data using [index aliases](../../data-store/aliases.md). However, there are a few more configuration steps and concepts:
25
+
## How rollover works in {{ilm-init}}
28
26
29
-
* An *index template* that specifies the settings for each new index in the series. You optimize this configuration for ingestion, typically using as many shards as you have hot nodes.
30
-
* An *index alias* that references the entire set of indices.
31
-
* A single index designated as the *write index*. This is the active index that handles all write requests. On each rollover, the new index becomes the write index.
27
+
You define a rollover action in the hot phase of an index lifecycle policy. It will run when any of the configured conditions are met and the write index contains at least one document.
28
+
You can configure the following rollover conditions:
32
29
33
-
::::{note}
34
-
When an index is rolled over, the previous index’s age is updated to reflect the rollover time. This date, rather than the index’s `creation_date`, is used in {{ilm}} `min_age` phase calculations. [Learn more](../../../troubleshoot/elasticsearch/index-lifecycle-management-errors.md#min-age-calculation).
30
+
***Size** - an index will rollover when its shards reach a set size, for example 50 GB.
31
+
***Age** - an index will rollover when it reaches a certain age, for example 7 days.
32
+
***Document count** - an index will rollover when a shard contains a certain number of documents, for example 2 million.
35
33
34
+
::::{tip}
35
+
Rolling over to a new index based on size, document count, or age is preferable to time-based rollovers. Rolling over at an arbitrary time often results in many small indices, which can have a negative impact on performance and resource usage.
36
36
::::
37
37
38
+
After rollover, indices move through other configured index lifecycle phases: warm, cold, frozen, and delete. Rollover creates a new write index while the previous one continues through the lifecycle phases.
38
39
40
+
**Special rules:**
39
41
40
-
## Automatic rollover [ilm-automatic-rollover]
42
+
* Rollover for an empty write index is skipped even if it has an associated `max_age` that would otherwise result in a rollover occurring. A policy can override this behavior if you set `min_docs: 0` in the rollover conditions. This can also be disabled on a cluster-wide basis if you set `indices.lifecycle.rollover.only_if_has_documents` to `false`.
43
+
* Forced rollover occurs if any shard reaches 200 million documents. Usually, a shard will reach 50 GB long before it reaches 200 million documents, but this isn’t the case for space efficient data sets.
41
44
42
-
{{ilm-init}} and the data stream lifecycle enable you to automatically roll over to a new index based on conditions like the index size, document count, or age. When a rollover is triggered, a new index is created, the write alias is updated to point to the new index, and all subsequent updates are written to the new index.
45
+
## Recommended approaches
43
46
44
-
::::{tip}
45
-
Rolling over to a new index based on size, document count, or age is preferable to time-based rollovers. Rolling over at an arbitrary time often results in many small indices, which can have a negative impact on performance and resource usage.
46
-
::::
47
+
Decide your approach to index rotation based on your use case and requirements.
47
48
49
+
| Use case | Recommended approach | Setup benefits and limitations |
Empty indices will not be rolled over, even if they have an associated `max_age` that would otherwise result in a roll over occurring. A policy can override this behavior, and explicitly opt in to rolling over empty indices, by adding a `"min_docs": 0` condition. This can also be disabled on a cluster-wide basis by setting `indices.lifecycle.rollover.only_if_has_documents` to `false`.
51
-
::::
55
+
^1^ Rollover is handled automatically for data streams in {{es-serverless}} projects. {applies_to}`serverless: ga`
52
56
57
+
:::{tip}
58
+
For new projects, use data streams. Unlike aliases, they're simpler to manage by defining lifecycle actions without requiring additional configuration for rollover.
59
+
:::
53
60
54
-
::::{important}
55
-
The rollover action implicitly always rolls over a data stream or alias if one or more shards contain 200000000 or more documents. Normally a shard will reach 50GB long before it reaches 200M documents, but this isn’t the case for space efficient data sets. Search performance will very likely suffer if a shard contains more than 200M documents. This is the reason of the builtin limit.
56
-
::::
57
61
62
+
### Rotating your indices with data streams [rollover-data-stream]
63
+
64
+
We recommend using [data streams](../../data-store/data-streams.md) to manage time series data. When set up to use an {{ilm-init}} policy that includes rollover, a data stream manages the rotation of your indices without additional configuration.
65
+
When targeting a data stream, each time the current write index reaches a specified age or size, a new backing index is generated (with an incremented number and timestamp), and it becomes the data stream's writing index.
66
+
67
+
Each data stream requires an [index template](../../data-store/templates.md) that contains the following:
68
+
69
+
* A name or wildcard (`*`) pattern for the data stream.
70
+
* A configuration that indicates a data stream is used for the index pattern.
71
+
* Optional: The mappings and settings applied to each backing index when it’s created.
72
+
73
+
For more information about this approach, refer to the [Manage time series data with data streams](../index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-with-data-streams) tutorial.
74
+
75
+
:::{tip}
76
+
Data streams are designed for append-only data, where the data stream name can be used as the operations (read, write, rollover, shrink etc.) target. If your use case requires data to be updated in place, you can perform [update or delete operations directly on the backing indices](../../data-store/data-streams/use-data-stream.md#update-delete-docs-in-a-backing-index).
77
+
:::
78
+
79
+
**Data streams naming pattern**<br>
80
+
{{es}} uses a structured naming convention for the backing indices of data streams, following this pattern:
81
+
82
+
```console
83
+
.ds-<DATA-STREAM-NAME>-<yyyy.MM.dd>-<GENERATION>
84
+
```
85
+
For more information about the data stream naming pattern, refer to the [Generation](../../data-store/data-streams.md#data-streams-generation) section of the Data streams page.
86
+
87
+
### Rotating your indices with aliases [rollover-with-aliases]
88
+
89
+
Rotating indices with aliases requires additional configuration steps, including bootstrapping the initial index. For more details about this approach, refer to the [Manage time series data without data streams](../index-lifecycle-management/tutorial-automate-rollover.md#manage-time-series-data-without-data-streams) tutorial.
90
+
91
+
:::{important}
92
+
The use of aliases for rollover requires meeting certain conditions. Review these considerations before applying this approach:
93
+
94
+
* The index name must match the pattern `<INDEX_NAME>-<INDEX_NUMBER>`, for example `my-index-000001`.
95
+
* The `index.lifecycle.rollover_alias` must be configured as the alias to roll over.
96
+
* The index must be the [write index](../../data-store/aliases.md#write-index) for the alias.
97
+
:::
98
+
99
+
::::{note}
100
+
When an alias or data stream is rolled over, the previous write index’s rollover time is stored. This date, rather than the index’s `creation_date`, is used in {{ilm}} `min_age` phase calculations. [Learn more](../../../troubleshoot/elasticsearch/index-lifecycle-management-errors.md#min-age-calculation).
101
+
102
+
::::
58
103
104
+
**Alias-based naming pattern**<br>
105
+
When configured correctly, the newly created write index will have a similar name to one that's been rolled over, however the six-digit, zero-padded suffix will be incremented. For example before rollover, the write index was called `my-index-000001` and after rollover, the newly created index becomes `my-index-000002` and also becomes the new write index. The alias typically shares the base name which in this example is `my-index`.
0 commit comments