diff --git a/deploy-manage/maintenance.md b/deploy-manage/maintenance.md index a6cca8f7b1..7236e6f632 100644 --- a/deploy-manage/maintenance.md +++ b/deploy-manage/maintenance.md @@ -1,25 +1,19 @@ --- mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-manage-kibana.html +applies_to: + deployment: + ess: + ece: + self: --- -# Maintenance [ece-manage-kibana] +# Maintenance [maintenance] -Kibana is an open source analytics and visualization platform designed to work with Elasticsearch, that makes it easy to perform advanced data analysis and to visualize your data in a variety of charts, tables, and maps. Its simple, browser-based interface enables you to quickly create and share dynamic dashboards that display changes to Elasticsearch queries in real time. +This section outlines the key tasks and processes required to maintain a healthy, performant, and secure {{es}} infrastructure and its deployments. -Most deployment templates include a Kibana instance, but if it wasn’t part of the initial deployment you can go to the **Kibana** page and **Enable** Kibana. - -The new Kibana instance takes a few moments to provision. After provisioning Kibana is complete, you can use the endpoint URL to access Kibana. - -::::{tip} -You can log into Kibana as the `elastic` superuser. The password was provided when you created your deployment or can be [reset](users-roles/cluster-or-deployment-auth/built-in-users.md). On AWS and not able to access Kibana? [Check if you need to update your endpoint URL first](../troubleshoot/deployments/cloud-enterprise/common-issues.md#ece-aws-private-ip). -:::: - - -From the deployment **Kibana** page you can also: - -* Terminate your Kibana instance, which stops it. The information is stored in your Elasticsearch cluster, so stopping and restarting should not risk your Kibana information. -* Restart it after stopping. -* Upgrade your Kibana instance version if it is out of sync with your Elasticsearch cluster. -* Delete to fully remove the instance, wipe it from the disk, and stop charges. +The topics covered include: +* **[ECE Maintenance](maintenance/ece.md)**: Explains the procedures for maintaining both the host infrastructure and {{es}} deployments within Elastic Cloud Enterprise (ECE). +* **[Start and Stop services](maintenance/start-stop-services.md)**: Provides step-by-step instructions on how to safely start and stop your {{es}} deployment or {{kib}} instance, particularly when performing actions that require a restart. +* **[Add and remove {{es}} nodes](maintenance/add-and-remove-elasticsearch-nodes.md)**: Guides you through the process of enrolling new nodes or safely removing existing ones from a self-managed {{es}} cluster to optimize resource utilization and cluster performance. diff --git a/deploy-manage/maintenance/add-and-remove-elasticsearch-nodes.md b/deploy-manage/maintenance/add-and-remove-elasticsearch-nodes.md index 858393b6f2..e0abf76677 100644 --- a/deploy-manage/maintenance/add-and-remove-elasticsearch-nodes.md +++ b/deploy-manage/maintenance/add-and-remove-elasticsearch-nodes.md @@ -1,6 +1,9 @@ --- mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/add-elasticsearch-nodes.html +applies_to: + deployment: + self: --- # Add and Remove Elasticsearch nodes [add-elasticsearch-nodes] @@ -21,9 +24,18 @@ When you add more nodes to a cluster, it automatically allocates replica shards. :alt: A cluster with three nodes ::: - ## Enroll nodes in an existing cluster [_enroll_nodes_in_an_existing_cluster_5] +::::{tip} +Refer to the following pages to learn more about how to add nodes to your cluster in different environments: + +* [autoscaling](../autoscaling.md) +* [{{ece}}](../deploy/cloud-enterprise/resize-deployment.md) +* [{{ech}}](../deploy/elastic-cloud/configure.md) +* [{{eck}}](../deploy/cloud-on-k8s/update-deployments.md) + +:::: + You can enroll additional nodes on your local machine to experiment with how an {{es}} cluster with multiple nodes behaves. ::::{note} @@ -31,7 +43,6 @@ To add a node to a cluster running on multiple machines, you must also set [`dis :::: - When {{es}} starts for the first time, the security auto-configuration process binds the HTTP layer to `0.0.0.0`, but only binds the transport layer to localhost. This intended behavior ensures that you can start a single-node cluster with security enabled by default without any additional configuration. Before enrolling a new node, additional actions such as binding to an address other than `localhost` or satisfying bootstrap checks are typically necessary in production clusters. During that time, an auto-generated enrollment token could expire, which is why enrollment tokens aren’t generated automatically. @@ -64,21 +75,18 @@ To enroll new nodes in your cluster, create an enrollment token with the `elasti For more information about discovery and shard allocation, refer to [*Discovery and cluster formation*](../distributed-architecture/discovery-cluster-formation.md) and [Cluster-level shard allocation and routing settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html). - ## Master-eligible nodes [add-elasticsearch-nodes-master-eligible] As nodes are added or removed Elasticsearch maintains an optimal level of fault tolerance by automatically updating the cluster’s *voting configuration*, which is the set of [master-eligible nodes](../distributed-architecture/clusters-nodes-shards/node-roles.md#master-node-role) whose responses are counted when making decisions such as electing a new master or committing a new cluster state. It is recommended to have a small and fixed number of master-eligible nodes in a cluster, and to scale the cluster up and down by adding and removing master-ineligible nodes only. However there are situations in which it may be desirable to add or remove some master-eligible nodes to or from a cluster. - ### Adding master-eligible nodes [modules-discovery-adding-nodes] If you wish to add some nodes to your cluster, simply configure the new nodes to find the existing cluster and start them up. Elasticsearch adds the new nodes to the voting configuration if it is appropriate to do so. During master election or when joining an existing formed cluster, a node sends a join request to the master in order to be officially added to the cluster. - ### Removing master-eligible nodes [modules-discovery-removing-nodes] When removing master-eligible nodes, it is important not to remove too many all at the same time. For instance, if there are currently seven master-eligible nodes and you wish to reduce this to three, it is not possible simply to stop four of the nodes at once: to do so would leave only three nodes remaining, which is less than half of the voting configuration, which means the cluster cannot take any further actions. @@ -108,7 +116,6 @@ Although the voting configuration exclusions API is most useful for down-scaling Voting exclusions are only required when removing at least half of the master-eligible nodes from a cluster in a short time period. They are not required when removing master-ineligible nodes, nor are they required when removing fewer than half of the master-eligible nodes. :::: - Adding an exclusion for a node creates an entry for that node in the voting configuration exclusions list, which has the system automatically try to reconfigure the voting configuration to remove that node and prevents it from returning to the voting configuration once it has removed. The current list of exclusions is stored in the cluster state and can be inspected as follows: ```console @@ -129,4 +136,3 @@ DELETE /_cluster/voting_config_exclusions # to return to the voting configuration in the future. DELETE /_cluster/voting_config_exclusions?wait_for_removal=false ``` - diff --git a/deploy-manage/maintenance/ece.md b/deploy-manage/maintenance/ece.md index 34c4b70c5d..5ec8df745c 100644 --- a/deploy-manage/maintenance/ece.md +++ b/deploy-manage/maintenance/ece.md @@ -1,9 +1,17 @@ +--- +applies_to: + deployment: + ece: +--- + # ECE maintenance -% What needs to be done: Write from scratch +Elastic Cloud Enterprise (ECE), being a self-managed Elastic Stack deployment platform, abstracts much of the complexity of running {{es}}, but still requires regular maintenance at both the platform and deployment levels. Maintenance activities range from managing individual deployments to performing infrastructure-level updates on ECE hosts. + +## Deployment maintenance and host infrastructure maintenance [ece-deployment-host-infra-maintenance] -% GitHub issue: https://github.com/elastic/docs-projects/issues/353 +[Deployment maintenance](ece/deployments-maintenance.md) focuses on managing individual {{es}} and {{kib}} instances within ECE. This includes actions such as [pausing instances](ece/pause-instance.md), [stopping request routing to nodes](ece/start-stop-routing-requests.md), and [moving instances between allocators](ece/move-nodes-instances-from-allocators.md) to optimize resource usage or prepare for maintenance. These tasks help maintain service availability and performance without affecting the underlying infrastructure. -% Scope notes: Introduction about ECE maintenance and activities / actions. Explain the difference between deployments maintenance and ECE hosts infrastructure maintenance. +[ECE host infrastructure maintenance](ece/perform-ece-hosts-maintenance.md) involves managing virtual machines that host ECE itself. This includes tasks like applying operating system patches, upgrading software, or decommissioning hosts. Infrastructure maintenance often requires more careful planning, as it can impact multiple deployments running on the affected hosts. Methods such as placing allocators into [maintenance mode](ece/enable-maintenance-mode.md) and redistributing workloads provide a smooth transition during maintenance operations. -⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file +This section provides guidance on best practices for both types of maintenance, helping you maintain a resilient ECE environment. diff --git a/deploy-manage/maintenance/ece/delete-ece-hosts.md b/deploy-manage/maintenance/ece/delete-ece-hosts.md index ebb901db79..ecb816436e 100644 --- a/deploy-manage/maintenance/ece/delete-ece-hosts.md +++ b/deploy-manage/maintenance/ece/delete-ece-hosts.md @@ -1,6 +1,9 @@ --- mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-delete-runner.html +applies_to: + deployment: + ece: --- # Delete ECE hosts [ece-delete-runner] @@ -16,13 +19,11 @@ To delete hosts: 1. [Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md). 2. From the **Platform** menu, select **Hosts**. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. + Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. 3. For hosts that hold the allocator role: - - 1. [Enable maintenance mode](enable-maintenance-mode.md) on the allocator. - 2. [Move all nodes off the allocator](move-nodes-instances-from-allocators.md) and to other allocators in your installation. + 1. [Enable maintenance mode](enable-maintenance-mode.md) on the allocator. + 2. [Move all nodes off the allocator](move-nodes-instances-from-allocators.md) and to other allocators in your installation. 4. Go to **Hosts** and select a host. 5. Select **Manage roles** from the **Manage host** menu and remove all assigned roles. @@ -30,8 +31,6 @@ To delete hosts: 7. Remove *all running* containers from the host, starting from the container with name `frc-runners-runner`. Then remove the storage directory (the default `/mnt/data/elastic/`). You can use the recommended [cleanup command](../../uninstall/uninstall-elastic-cloud-enterprise.md). Upon doing so, the UI should reflect the host is **Disconnected**, allowing the host to be deleted. 8. Select **Delete host** and confirm. -::::{tip} +::::{tip} Refresh the page if the button isn’t active. :::: - - diff --git a/deploy-manage/maintenance/ece/deployments-maintenance.md b/deploy-manage/maintenance/ece/deployments-maintenance.md index 4a07187355..ab61c73117 100644 --- a/deploy-manage/maintenance/ece/deployments-maintenance.md +++ b/deploy-manage/maintenance/ece/deployments-maintenance.md @@ -1,18 +1,18 @@ --- mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-maintenance-mode-deployments.html +applies_to: + deployment: + ece: --- # Deployments maintenance [ece-maintenance-mode-deployments] In some circumstances, you might need to temporarily restrict access to a node so you can perform corrective actions that might otherwise be difficult to complete. For example, if your cluster is being overwhelmed by requests because it is undersized for its workload, its nodes might not respond to efforts to resize. -These actions act as a maintenance mode for cluster node. Performing these actions can stop the cluster from becoming completely unresponsive so that you can resolve operational issues much more effectively. +These actions act as a maintenance mode for cluster node. Performing these actions can stop the cluster from becoming unresponsive so that you can resolve operational issues much more effectively. * [**Stop routing to the instance**](start-stop-routing-requests.md): Block requests from being routed to the cluster node. This is a less invasive action than pausing the cluster. * [**Pause an instance**](pause-instance.md): Suspend the node immediately by stopping the container that the node runs on without completing existing requests. This is a more aggressive action to regain control of an unresponsive node. As an alternative, to quickly add capacity to a deployment if it is unhealthy or at capacity, you can also [override the resource limit for a deployment](../../deploy/cloud-enterprise/resource-overrides.md). - - - diff --git a/deploy-manage/maintenance/ece/enable-maintenance-mode.md b/deploy-manage/maintenance/ece/enable-maintenance-mode.md index 138879baf3..3f57ab6d09 100644 --- a/deploy-manage/maintenance/ece/enable-maintenance-mode.md +++ b/deploy-manage/maintenance/ece/enable-maintenance-mode.md @@ -1,6 +1,9 @@ --- mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-maintenance-mode.html +applies_to: + deployment: + ece: --- # Enable maintenance mode [ece-maintenance-mode] @@ -12,34 +15,13 @@ To put an allocator into maintenance mode: 1. [Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md). 2. From the **Platform** menu, select **Allocators**. 3. Choose the allocator you want to work with and select **Enable Maintenance Mode**. Confirm the action. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - + Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. After the allocator enters maintenance mode, no new Elasticsearch nodes or Kibana instances will be started on the allocator. Existing nodes will continue to work as expected. You can now safely perform actions like [moving nodes off the allocator](move-nodes-instances-from-allocators.md). If you want to make the allocator fully active again, select **Disable Maintenance Mode**. Confirm the action. -::::{tip} -If you need the existing instances to stop routing requests you can [stop routing requests](deployments-maintenance.md) to disable incoming requests to particular instances. You can also massively disable all allocator instances routing with the [allocator-toggle-routing-requests.sh](https://download.elastic.co/cloud/allocator-toggle-routing-requests.sh) script. The script runs with the following parameters in the form environment variables: - -* `API_URL` Url of the administration API. -* `AUTH_HEADER` Curl format string representing the authentication header. -* `ALLOCATOR_ID` Action target allocator id. -* `ENABLE_TRAFFIC` Wether traffic to the selected allocator instances should be enabled (`true`) or disabled (`false`). - -This is an example of script execution to disable routing on all instances running on a given allocator: In this example the script disables routing on all instances running on a given allocator: - -```bash -AUTH_HEADER="Authorization: ApiKey $(cat ~/api.key)" API_URL="https://adminconsole:12443" ALLOCATOR_ID="192.168.44.10" ENABLE_TRAFFIC=false ./allocator-toggle-routing-requests.sh -``` - -The same script can be used to enable traffic again: - -```bash -AUTH_HEADER="Authorization: ApiKey $(cat ~/api.key)" API_URL="https://adminconsole:12443" ALLOCATOR_ID="192.168.44.10" ENABLE_TRAFFIC=true ./allocator-toggle-routing-requests.sh -``` +::::{tip} +If you need the existing instances to stop routing requests, refer to the [stop routing request documentation](start-stop-routing-requests.md) to learn more. :::: - - diff --git a/deploy-manage/maintenance/ece/maintenance-activities.md b/deploy-manage/maintenance/ece/maintenance-activities.md index 7ac140925f..52c320aa7f 100644 --- a/deploy-manage/maintenance/ece/maintenance-activities.md +++ b/deploy-manage/maintenance/ece/maintenance-activities.md @@ -1,9 +1,44 @@ +--- +applies_to: + deployment: + ece: +--- # Maintenance activities -% What needs to be done: Write from scratch +Maintenance activities ensure the smooth operation and scalability of your {{es}} installation. This section provides guidelines on performing essential maintenance tasks while minimizing downtime and maintaining high availability. -% GitHub issue: https://github.com/elastic/docs-projects/issues/353 +## Available maintenance operations -% Scope notes: summarize the list of activites +### [Enable maintenance mode](enable-maintenance-mode.md) -⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file +Before performing maintenance on an allocator, you should enable maintenance mode to prevent new Elasticsearch clusters and Kibana instances from being provisioned. This ensures that existing deployments can be safely moved to other allocators or adjusted without disruption. + +### [Scale out installation](scale-out-installation.md) + +You can scale out your installation by adding capacity to meet growing demand or improve high availability. This process involves installing ECE on additional hosts, assigning roles to new hosts, and resizing deployments to utilize the expanded resources. + +### [Move nodes and instances between allocators](move-nodes-instances-from-allocators.md) + +Moving {{es}} nodes, {{kib}} instances, and other components between allocators may be necessary to free up space, avoid downtime, or handle allocator failures. The process involves selecting target allocators and ensuring enough capacity to accommodate the migration. + +### [Perform ECE host maintenance](perform-ece-hosts-maintenance.md) + +Maintaining ECE hosts is critical for applying system patches, performing hardware upgrades, and ensuring compliance with security standards. Learn about the various methods of maintaining hosts, and their impact on your ECE installation. + +### [Delete ECE hosts](delete-ece-hosts.md) + +If a host is no longer required or is faulty, it can be removed from the Elastic Cloud Enterprise installation. Deleting a host only removes it from the installation but does not uninstall the software from the physical machine. Before deletion, allocators should be placed in maintenance mode, and nodes should be migrated to avoid disruption. + +## Best practices for maintenance + +* Always check available capacity before making changes. + +* Use maintenance mode to avoid unexpected disruptions. + +* Move nodes strategically to maintain high availability. + +* Perform maintenance during off-peak hours when possible. + +* Regularly review and optimize resource allocation. + +By following these guidelines, you can ensure the stability and efficiency of your environment while carrying out necessary maintenance activities. diff --git a/deploy-manage/maintenance/ece/move-nodes-instances-from-allocators.md b/deploy-manage/maintenance/ece/move-nodes-instances-from-allocators.md index be82f5ded7..02ace2127b 100644 --- a/deploy-manage/maintenance/ece/move-nodes-instances-from-allocators.md +++ b/deploy-manage/maintenance/ece/move-nodes-instances-from-allocators.md @@ -1,6 +1,9 @@ --- mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-move-nodes.html +applies_to: + deployment: + ece: --- # Move nodes or instances from allocators [ece-move-nodes] @@ -12,13 +15,11 @@ You might need to move Elasticsearch nodes, Kibana instances, and other componen * To make room on an allocator: You can move some smaller deployments to another allocator if you need additional room for a larger one on an allocator. * To move deployments after a failure: When host failures happen, you can move all deployments from the affected allocator to a healthy allocator quickly before spending any time on fixing the failure. -::::{tip} +::::{tip} When you move all nodes from an existing allocator to the new one, ECE migrates the data to new nodes. The migration can take some time, especially when deployments contain large amounts of data and have a heavy workload. Is your deployment under a heavy workload? You might need to [stop routing requests](deployments-maintenance.md) first. :::: - - -## Before you begin [ece_before_you_begin_9] +## Before you begin [ece_before_you_begin_9] Before you move the nodes and instances that are part of a deployment, you need to make sure that you have sufficient capacity on another allocator. For example: If you have a deployment with a single 32 GB Elasticsearch node and a 4 GB Kibana instance, the allocator that you are moving the deployment to needs to have at least 36 GB of capacity. Note that moving nodes does not actually move the same node onto a different allocator. Under the covers, Elastic Cloud Enterprise creates a new node and then migrates the data for you. @@ -28,51 +29,46 @@ If you followed our recommendation and [tagged your allocators](../../deploy/clo When you move all nodes from an existing allocator to the new one, ECE migrates the data to new nodes. The migration can take some time, especially when clusters contain large amounts of data and have a heavy workload. Is your cluster under a heavy workload? You might need to [stop routing requests](deployments-maintenance.md) first. +## Moving nodes from allocators [move-nodes-from-allocators] + To move nodes from one allocator to another one: 1. [Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md). 2. From the **Platform** menu, select **Allocators**. 3. Review the list of all allocators that are part of this installation and look for allocators that are unhealthy or find the allocator that you want to free up. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. + Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. 4. Recommended: [Put the allocator into maintenance mode](enable-maintenance-mode.md) before continuing. 5. Select the name of an unhealthy allocator and then choose **Move Nodes** from the menu. 6. Select the nodes you want, then choose **Move Nodes**. 7. To customize how you would like to move the nodes, select **Customize settings**, choose your options, then select **Move nodes**. + ::::{important} + Review **Customize Settings** before proceeding to move nodes. + :::: - ::::{important} - Review **Customize Settings** before proceeding to move nodes. - :::: - - - Gracefully move data - : (Default) Gracefully move the data from the instances we’re about to remove from the cluster before stopping them. Never disable this setting at the same time as enabling `override_failsafe` on a non-Highly Available cluster since it can result in data loss. + Gracefully move data + : (Default) Gracefully move the data from the instances we’re about to remove from the cluster before stopping them. Never disable this setting at the same time as enabling `override_failsafe` on a non-Highly Available cluster since it can result in data loss. - Skip snapshot - : If an allocator has failed or is otherwise unhealthy, select this option to move the nodes but disable the snapshot attempt. As this can perform potentially destructive actions on the deployment, do not use this option on a healthy allocator unless you are an advanced user. + Skip snapshot + : If an allocator has failed or is otherwise unhealthy, select this option to move the nodes but disable the snapshot attempt. As this can perform potentially destructive actions on the deployment, do not use this option on a healthy allocator unless you are an advanced user. - Restore snapshot to latest success - : Restore the cluster to the last successful snapshot. Recommended for single-node clusters hosted on unhealthy allocators. Any data indexed after the last snapshot was taken is lost. + Restore snapshot to latest success + : Restore the cluster to the last successful snapshot. Recommended for single-node clusters hosted on unhealthy allocators. Any data indexed after the last snapshot was taken is lost. - Extended maintenance - : Keep new instances in maintenance mode until a snapshot has been restored. If not enabled, new instances remain in maintenance mode only until they can join a cluster. + Extended maintenance + : Keep new instances in maintenance mode until a snapshot has been restored. If not enabled, new instances remain in maintenance mode only until they can join a cluster. - Set target allocators - : Request that instances be moved to the specified allocators. If no allocators are specified, or those specified are unsuitable for the instances being moved, then any suitable healthy allocator can be used. - - Reallocate - : Create new containers for all nodes in the cluster. - - Set Timeout - : On by default. - - -::::{tip} -If you did not enable maintenance mode, set a target allocator under the advanced options when moving nodes to make sure the nodes do not end up on the same allocator again. By default, moving a node moves it to any allocator that has enough capacity. -:::: + Set target allocators + : Request that instances be moved to the specified allocators. If no allocators are specified, or those specified are unsuitable for the instances being moved, then any suitable healthy allocator can be used. + Reallocate + : Create new containers for all nodes in the cluster. -1. Repeat **step 6** for each of the node types until no nodes remain on the allocator. -2. Optionally, once the nodes have been moved, **Delete Allocator**. + Set Timeout + : On by default. + ::::{tip} + If you did not enable maintenance mode, set a target allocator under the advanced options when moving nodes to make sure the nodes do not end up on the same allocator again. By default, moving a node moves it to any allocator that has enough capacity. + :::: +8. Repeat **step 6** for each of the node types until no nodes remain on the allocator. +9. Optionally, once the nodes have been moved, **Delete Allocator**. diff --git a/deploy-manage/maintenance/ece/pause-instance.md b/deploy-manage/maintenance/ece/pause-instance.md index 6e6301d1c6..3f51c58d80 100644 --- a/deploy-manage/maintenance/ece/pause-instance.md +++ b/deploy-manage/maintenance/ece/pause-instance.md @@ -1,6 +1,9 @@ --- mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-maintenance-mode-pausing.html +applies_to: + deployment: + ece: --- # Pause instance [ece-maintenance-mode-pausing] diff --git a/deploy-manage/maintenance/ece/perform-ece-hosts-maintenance.md b/deploy-manage/maintenance/ece/perform-ece-hosts-maintenance.md index b99702e27d..7a218030ab 100644 --- a/deploy-manage/maintenance/ece/perform-ece-hosts-maintenance.md +++ b/deploy-manage/maintenance/ece/perform-ece-hosts-maintenance.md @@ -1,6 +1,9 @@ --- mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-perform-host-maintenance.html +applies_to: + deployment: + ece: --- # Perform ECE hosts maintenance [ece-perform-host-maintenance] @@ -20,21 +23,18 @@ You can perform these maintenance actions on the hosts in your ECE installation Which method you choose depends on how invasive your host maintenance needs to be. If your host maintenance could affect ECE, use the destructive method that first deletes the host from your installation. These methods include a step that moves any hosted Elasticsearch clusters and Kibana instances off the affected hosts and are generally considered safe, provided that your ECE installation still has sufficient resources available to operate after the host has been removed. - -## By disabling the Docker daemon [ece-perform-host-maintenance-docker-disable] +## By disabling the Docker daemon [ece-perform-host-maintenance-docker-disable] This method lets you perform maintenance actions on hosts without first removing the associated host from your Elastic Cloud Enterprise installation. It works by disabling the Docker daemon. The host remains a part of your ECE installation throughout these steps but will be offline and the resources it provides will not be available. To perform host maintenance: 1. Recommended: If the host holds the allocator role and you have enough spare capacity: - - 1. [Enable maintenance mode](enable-maintenance-mode.md) on the allocator. - 2. [Move all nodes off the allocator](move-nodes-instances-from-allocators.md) and to other allocators in your installation. Moving all nodes lets you retain the same level of redundancy for highly available Elasticsearch clusters and ensures that other clusters without high availability remain available. - - ::::{important} - Skipping Step 1 will affect the availability of clusters with nodes on the allocator. - :::: + 1. [Enable maintenance mode](enable-maintenance-mode.md) on the allocator. + 2. [Move all nodes off the allocator](move-nodes-instances-from-allocators.md) and to other allocators in your installation. Moving all nodes lets you retain the same level of redundancy for highly available Elasticsearch clusters and ensures that other clusters without high availability remain available. + ::::{important} + Skipping Step 1 will affect the availability of clusters with nodes on the allocator. + :::: 2. Disable the Docker daemon: @@ -69,21 +69,18 @@ To perform host maintenance: After the host shows a green status in the Cloud UI, it is fully functional again and can be used as before. - -## By deleting the host (destructive) [ece-perform-host-maintenance-delete-runner] +## By deleting the host (destructive) [ece-perform-host-maintenance-delete-runner] This method lets you perform potentially destructive maintenance actions on hosts. It works by deleting the associated host, which removes the host from your Elastic Cloud Enterprise installation. To add the host to your ECE installation again after host maintenance is complete, you must reinstall ECE. To perform host maintenance: 1. If the host holds the allocator role: - - 1. [Enable maintenance mode](enable-maintenance-mode.md) on the allocator. - 2. [Move all nodes off the allocator](move-nodes-instances-from-allocators.md) and to other allocators in your installation. Moving all nodes lets you retain the same level of redundancy for highly available clusters and ensures that other clusters without high availability remain available. - - ::::{important} - Do not skip this step or you will affect the availability of clusters with nodes on the allocator. You are in the process of removing the host from your installation and whatever ECE artifacts are stored on it will be lost. - :::: + 1. [Enable maintenance mode](enable-maintenance-mode.md) on the allocator. + 2. [Move all nodes off the allocator](move-nodes-instances-from-allocators.md) and to other allocators in your installation. Moving all nodes lets you retain the same level of redundancy for highly available clusters and ensures that other clusters without high availability remain available. + ::::{important} + Do not skip this step or you will affect the availability of clusters with nodes on the allocator. You are in the process of removing the host from your installation and whatever ECE artifacts are stored on it will be lost. + :::: 2. [Delete the host from your ECE installation](delete-ece-hosts.md). 3. Perform the maintenance on your host, such as enabling encryption of data at rest. @@ -92,8 +89,7 @@ To perform host maintenance: After the host shows a green status in the Cloud UI, the host is part of your ECE installation again and can be used as before. - -## By shutting down the host (less destructive) [ece-perform-host-maintenance-shutdown-host] +## By shutting down the host (less destructive) [ece-perform-host-maintenance-shutdown-host] This method lets you perform potentially destructive maintenance actions on hosts. It works by temporarily shutting down an ECE host, e.g. for data center moves or planned power outages. It is offered as an non-guaranteed and less destructive alternative to fully [deleting a host](#ece-perform-host-maintenance-delete-runner) from your ECE installation. @@ -101,13 +97,11 @@ To shut down the host: 1. Disable traffic from load balancers. 2. Shut down all allocators: - - 1. [Enable maintenance mode](enable-maintenance-mode.md) on the allocator. - 2. [Move all nodes off the allocator](move-nodes-instances-from-allocators.md) and to other allocators in your installation. Moving all nodes lets you retain the same level of redundancy for highly available clusters and ensures that other clusters without high availability remain available. - - ::::{important} - Do not skip this step or you will affect the availability of clusters with nodes on the allocator. You are in the process of removing the host from your installation and whatever ECE artifacts are stored on it will be lost. - :::: + 1. [Enable maintenance mode](enable-maintenance-mode.md) on the allocator. + 2. [Move all nodes off the allocator](move-nodes-instances-from-allocators.md) and to other allocators in your installation. Moving all nodes lets you retain the same level of redundancy for highly available clusters and ensures that other clusters without high availability remain available. + ::::{important} + Do not skip this step or you will affect the availability of clusters with nodes on the allocator. You are in the process of removing the host from your installation and whatever ECE artifacts are stored on it will be lost. + :::: 3. Shut down all non-director hosts. 4. Shut down directors. @@ -123,4 +117,3 @@ After performing maintenance, start up the host: 3. Start all remaining hosts. 4. Re-enable traffic from load balancers. - diff --git a/deploy-manage/maintenance/ece/scale-out-installation.md b/deploy-manage/maintenance/ece/scale-out-installation.md index b300a76f2f..bc736cb8a5 100644 --- a/deploy-manage/maintenance/ece/scale-out-installation.md +++ b/deploy-manage/maintenance/ece/scale-out-installation.md @@ -1,6 +1,9 @@ --- mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-add-capacity.html +applies_to: + deployment: + ece: --- # Scale out your installation [ece-add-capacity] @@ -11,21 +14,16 @@ Check the available capacity: 1. [Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md). 2. From the **Platform** menu, select **Allocators** to view the available capacity. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - - In this case 3 GB and 6 GB on two different allocators: - - :::{image} ../../../images/cloud-enterprise-ece-available-capacity.png - :alt: The available capacity in an installation - ::: - + Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. + In this case 3 GB and 6 GB on two different allocators: + :::{image} ../../../images/cloud-enterprise-ece-available-capacity.png + :alt: The available capacity in an installation + ::: If this is not sufficient, add more capacity to your installation: -* [Install Elastic Cloud Enterprise on additional hosts](../../deploy/cloud-enterprise/install-ece-on-additional-hosts.md) to create additional capacity. -* [Add capacity](https://www.elastic.co/docs/api/doc/cloud-enterprise/operation/operation-set-allocator-settings) to existing allocators by updating the allocator settings when adding memory to the host. -* [Assign roles](../../deploy/cloud-enterprise/assign-roles-to-hosts.md) to the additional hosts. If you need to handle a larger search or logging workload, assign the new hosts the allocator role. -* (Optional) [Tag allocators](../../deploy/cloud-enterprise/ece-configuring-ece-tag-allocators.md) to the new host to indicate what kind of hardware you have available. -* [Resize your deployment](../../deploy/cloud-enterprise/resize-deployment.md) to handle a larger workload. - +1. [Install Elastic Cloud Enterprise on additional hosts](../../deploy/cloud-enterprise/install-ece-on-additional-hosts.md) to create additional capacity. +2. [Add capacity](https://www.elastic.co/docs/api/doc/cloud-enterprise/operation/operation-set-allocator-settings) to existing allocators by updating the allocator settings when adding memory to the host. +3. [Assign roles](../../deploy/cloud-enterprise/assign-roles-to-hosts.md) to the additional hosts. If you need to handle a larger search or logging workload, assign the new hosts the allocator role. +4. (Optional) [Tag allocators](../../deploy/cloud-enterprise/ece-configuring-ece-tag-allocators.md) to the new host to indicate what kind of hardware you have available. +5. [Resize your deployment](../../deploy/cloud-enterprise/resize-deployment.md) to handle a larger workload. diff --git a/deploy-manage/maintenance/ece/start-stop-routing-requests.md b/deploy-manage/maintenance/ece/start-stop-routing-requests.md index f0e42a2a10..1b78959ac6 100644 --- a/deploy-manage/maintenance/ece/start-stop-routing-requests.md +++ b/deploy-manage/maintenance/ece/start-stop-routing-requests.md @@ -2,22 +2,54 @@ mapped_urls: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-maintenance-mode-routing.html - https://www.elastic.co/guide/en/cloud/current/ec-maintenance-mode-routing.html +applies_to: + deployment: + ece: --- -# Start and stop routing requests +# Start and stop routing requests [maintenance-mode-routing] -% What needs to be done: Refine +The {{ecloud}} proxy service routes traffic from external sources to the deployment, between deployments, and between products within a deployment. For example, it routes API requests from your local machine to your deployment, CCR and CCS requests between your deployments, and communications between {{kib}} and {{es}}. It does not direct the TCP traffic between {{es}} nodes, nor does it manage requests starting within {{es}} outwards to external sources such as to snapshot repositories. -% GitHub issue: https://github.com/elastic/docs-projects/issues/353 +The {{ecloud}} proxy routes HTTP requests to its deployment’s individual product instances through the product’s endpoint. By default, instances are enabled to route HTTP traffic and will report no special messaging. -% Use migrated content from existing pages that map to this page: +It might be helpful to temporarily block upstream requests in order to protect some or all instances or products within your deployment. For example, you might stop request routing in the following cases: -% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-maintenance-mode-routing.md -% - [ ] ./raw-migrated-files/cloud/cloud/ec-maintenance-mode-routing.md +* If another team within your company starts streaming new data into your production {{integrations-server}} without previous load testing, both it and {{es}} might experience performance issues. You might consider stopping routing requests on all {{integrations-server}} instances in order to protect your downstream {{es}} instance. +* If {{es}} is being overwhelmed by upstream requests, it might experience increased response times or even become unresponsive. This might impact your ability to resize components in your deployment and increase the duration of pending plans or increase the chance of plan changes failing. Because every {{es}} node is an [implicit coordinating node](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html), you should stop routing requests across all {{es}} nodes to completely block upstream traffic. -⚠️ **This page is a work in progress.** ⚠️ +## Considerations [request-routing-considerations] -The documentation team is working to combine content pulled from the following pages: +* {{ecloud}} will automatically set and remove routing blocks during plan changes. Elastic recommends avoiding manually overriding these settings for a deployment while its plans are pending. +* The [{{es}} API console](../../../explore-analyze/query-filter/tools/console.md) bypasses {{ecloud}} proxy routing blocks against {{es}} to enable administrative tasks while plan changes are pending. You should generally default traffic to the {{es}} endpoint. However, if you enable **Stop routing requests** across all {{es}} nodes, you need to use this UI to administer your cluster. +* While {{es}} has **Stop routing requests** set across all nodes, other products with the deployment may become unhealthy. This is because {{es}} is a prerequisite for those other products, such as {{kib}}. In {{kib}}, this results in a [**Kibana server is not ready yet**](../../../troubleshoot/kibana/error-server-not-ready.md) message. +* Enabling **Stop routing requests** does not affect your [billing](../../../deploy-manage/cloud-organization/billing.md). If needed, you can stop charges for a deployment by [deleting the deployment](../../../deploy-manage/uninstall/delete-a-cloud-deployment.md). -* [/raw-migrated-files/cloud/cloud-enterprise/ece-maintenance-mode-routing.md](/raw-migrated-files/cloud/cloud-enterprise/ece-maintenance-mode-routing.md) -* [/raw-migrated-files/cloud/cloud/ec-maintenance-mode-routing.md](/raw-migrated-files/cloud/cloud/ec-maintenance-mode-routing.md) \ No newline at end of file +## Stop routing requests [stop-routing-requests] + +To block HTTP requests for an instance, select **Stop routing requests** under from instance’s menu. + +The instance will then report **Not routing requests**. It will complete existing requested traffic, but not be sent new requests. + +You can stop routing requests to disable incoming requests to particular instances. You can also massively disable all allocator instances routing with the [allocator-toggle-routing-requests.sh](https://download.elastic.co/cloud/allocator-toggle-routing-requests.sh) script. The script runs with the following parameters in the form environment variables: + +* `API_URL` Url of the administration API. +* `AUTH_HEADER` Curl format string representing the authentication header. +* `ALLOCATOR_ID` Action target allocator id. +* `ENABLE_TRAFFIC` Wether traffic to the selected allocator instances should be enabled (`true`) or disabled (`false`). + +This is an example of script execution to disable routing on all instances running on a given allocator: + +```bash +AUTH_HEADER="Authorization: ApiKey $(cat ~/api.key)" API_URL="https://adminconsole:12443" ALLOCATOR_ID="192.168.44.10" ENABLE_TRAFFIC=false ./allocator-toggle-routing-requests.sh +``` + +The same script can be used to enable traffic again: + +```bash +AUTH_HEADER="Authorization: ApiKey $(cat ~/api.key)" API_URL="https://adminconsole:12443" ALLOCATOR_ID="192.168.44.10" ENABLE_TRAFFIC=true ./allocator-toggle-routing-requests.sh +``` + +## Restart routing requests [restart-routing-requests] + +To unblock HTTP requests for an instance, select **Start routing requests** under from instance’s menu. diff --git a/deploy-manage/maintenance/start-stop-services.md b/deploy-manage/maintenance/start-stop-services.md index 758a44ecc4..f0173b0ebf 100644 --- a/deploy-manage/maintenance/start-stop-services.md +++ b/deploy-manage/maintenance/start-stop-services.md @@ -1,9 +1,24 @@ +--- +applies_to: + deployment: + ess: + ece: + self: +--- + # Start and stop services -% What needs to be done: Write from scratch +This section covers the recommended procedures for: + +* [starting and stopping self-managed Elasticsearch nodes](start-stop-services/start-stop-elasticsearch.md) +* [starting and stopping self-managed Kibana instances](start-stop-services/start-stop-kibana.md) +* [restarting an ECE deployment](start-stop-services/restart-an-ece-deployment.md) +* [restarting {{ech}} deployments](start-stop-services/restart-cloud-hosted-deployment.md) +* [full cluster and rolling restarts for self-managed clusters](start-stop-services/full-cluster-restart-rolling-restart-procedures.md) -% GitHub issue: https://github.com/elastic/docs-projects/issues/353 +::::{note} +In ECK, when a resource – like {{es}} or {{kib}} – is declared, the reconciliation loop ensures the desired state is maintained. There is no built-in stop mechanism in Kubernetes because it’s designed for declarative state management. You either define a resource, and Kubernetes ensures it’s running, or you delete it. You can restart instances by deleting Pods, as the platform will start them immediately. -% Scope notes: Super brief summary +:::: -⚠️ **This page is a work in progress.** ⚠️ \ No newline at end of file +Following these guidelines helps prevent data loss, minimize downtime, and maintain optimal performance across different environments. diff --git a/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md b/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md index d3d70b8f51..147b0ba4cd 100644 --- a/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md +++ b/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md @@ -1,6 +1,9 @@ --- mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/restart-cluster.html +applies_to: + deployment: + self: --- # Full Cluster restart and rolling restart procedures [restart-cluster] @@ -12,35 +15,30 @@ Nodes exceeding the low watermark threshold will be slow to restart. Reduce the :::: - - ## Full-cluster restart [restart-cluster-full] 1. **Disable shard allocation.** + When you shut down a data node, the allocation process waits for `index.unassigned.node_left.delayed_timeout` (by default, one minute) before starting to replicate the shards on that node to other nodes in the cluster, which can involve a lot of I/O. Since the node is shortly going to be restarted, this I/O is unnecessary. You can avoid racing the clock by [disabling allocation](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html#cluster-routing-allocation-enable) of replicas before shutting down [data nodes](../../distributed-architecture/clusters-nodes-shards/node-roles.md#data-node-role): - When you shut down a data node, the allocation process waits for `index.unassigned.node_left.delayed_timeout` (by default, one minute) before starting to replicate the shards on that node to other nodes in the cluster, which can involve a lot of I/O. Since the node is shortly going to be restarted, this I/O is unnecessary. You can avoid racing the clock by [disabling allocation](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html#cluster-routing-allocation-enable) of replicas before shutting down [data nodes](../../distributed-architecture/clusters-nodes-shards/node-roles.md#data-node-role): - - ```console - PUT _cluster/settings + ```console + PUT _cluster/settings { "persistent": { "cluster.routing.allocation.enable": "primaries" } } - ``` + ``` - You can also consider [gateway settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html) when restarting large clusters to reduce initial strain while nodes are processing [through discovery](../../distributed-architecture/discovery-cluster-formation.md). + You can also consider [gateway settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html) when restarting large clusters to reduce initial strain while nodes are processing [through discovery](../../distributed-architecture/discovery-cluster-formation.md). 2. **Stop indexing and perform a flush.** + Performing a [flush](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-flush) speeds up shard recovery. - Performing a [flush](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-flush) speeds up shard recovery. - - ```console - POST /_flush - ``` - + ```console + POST /_flush + ``` -1. **Temporarily stop the tasks associated with active {{ml}} jobs and {{dfeeds}}.** (Optional) +3. **Temporarily stop the tasks associated with active {{ml}} jobs and {{dfeeds}}.** (Optional) {{ml-cap}} features require specific [subscriptions](https://www.elastic.co/subscriptions). @@ -56,7 +54,7 @@ Nodes exceeding the low watermark threshold will be slow to restart. Reduce the * [Stop all {{dfeeds}} and close all jobs](https://www.elastic.co/guide/en/machine-learning/current/ml-ad-run-jobs.html#ml-ad-close-job). This option saves the model state at the time of closure. When you reopen the jobs after the cluster restart, they use the exact same model. However, saving the latest model state takes longer than using upgrade mode, especially if you have a lot of jobs or jobs with large model states. -2. **Shut down all nodes.** +4. **Shut down all nodes.** * If you are running {{es}} with `systemd`: @@ -76,12 +74,10 @@ Nodes exceeding the low watermark threshold will be slow to restart. Reduce the kill $(cat pid) ``` -3. **Perform any needed changes.** -4. **Restart nodes.** - - If you have dedicated master nodes, start them first and wait for them to form a cluster and elect a master before proceeding with your data nodes. You can check progress by looking at the logs. - - As soon as enough master-eligible nodes have discovered each other, they form a cluster and elect a master. At that point, you can use the [cat health](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-health) and [cat nodes](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-nodes) APIs to monitor nodes joining the cluster: +5. **Perform any needed changes.** +6. **Restart nodes.** + If you have dedicated master nodes, start them first and wait for them to form a cluster and elect a master before proceeding with your data nodes. You can check progress by looking at the logs. + As soon as enough master-eligible nodes have discovered each other, they form a cluster and elect a master. At that point, you can use the [cat health](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-health) and [cat nodes](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-nodes) APIs to monitor nodes joining the cluster: ```console GET _cat/health @@ -89,17 +85,14 @@ Nodes exceeding the low watermark threshold will be slow to restart. Reduce the GET _cat/nodes ``` - The `status` column returned by `_cat/health` shows the health of each node in the cluster: `red`, `yellow`, or `green`. + The `status` column returned by `_cat/health` shows the health of each node in the cluster: `red`, `yellow`, or `green`. -5. **Wait for all nodes to join the cluster and report a status of yellow.** +7. **Wait for all nodes to join the cluster and report a status of yellow.** + When a node joins the cluster, it begins to recover any primary shards that are stored locally. The [`_cat/health`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-health) API initially reports a `status` of `red`, indicating that not all primary shards have been allocated. + Once a node recovers its local shards, the cluster `status` switches to `yellow`, indicating that all primary shards have been recovered, but not all replica shards are allocated. This is to be expected because you have not yet re-enabled allocation. Delaying the allocation of replicas until all nodes are `yellow` allows the master to allocate replicas to nodes that already have local shard copies. - When a node joins the cluster, it begins to recover any primary shards that are stored locally. The [`_cat/health`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-health) API initially reports a `status` of `red`, indicating that not all primary shards have been allocated. - - Once a node recovers its local shards, the cluster `status` switches to `yellow`, indicating that all primary shards have been recovered, but not all replica shards are allocated. This is to be expected because you have not yet re-enabled allocation. Delaying the allocation of replicas until all nodes are `yellow` allows the master to allocate replicas to nodes that already have local shard copies. - -6. **Re-enable allocation.** - - When all nodes have joined the cluster and recovered their primary shards, re-enable allocation by restoring `cluster.routing.allocation.enable` to its default: +8. **Re-enable allocation.** + When all nodes have joined the cluster and recovered their primary shards, re-enable allocation by restoring `cluster.routing.allocation.enable` to its default: ```console PUT _cluster/settings @@ -110,9 +103,8 @@ Nodes exceeding the low watermark threshold will be slow to restart. Reduce the } ``` - Once allocation is re-enabled, the cluster starts allocating replica shards to the data nodes. At this point it is safe to resume indexing and searching, but your cluster will recover more quickly if you can wait until all primary and replica shards have been successfully allocated and the status of all nodes is `green`. - - You can monitor progress with the [`_cat/health`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-health) and [`_cat/recovery`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-recovery) APIs: + Once allocation is re-enabled, the cluster starts allocating replica shards to the data nodes. At this point it is safe to resume indexing and searching, but your cluster will recover more quickly if you can wait until all primary and replica shards have been successfully allocated and the status of all nodes is `green`. + You can monitor progress with the [`_cat/health`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-health) and [`_cat/recovery`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-recovery) APIs: ```console GET _cat/health @@ -120,23 +112,19 @@ Nodes exceeding the low watermark threshold will be slow to restart. Reduce the GET _cat/recovery ``` -7. **Restart machine learning jobs.** (Optional) - - If you temporarily halted the tasks associated with your {{ml}} jobs, use the [set upgrade mode API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-set-upgrade-mode) to return them to active states: +9. **Restart machine learning jobs.** (Optional) + If you temporarily halted the tasks associated with your {{ml}} jobs, use the [set upgrade mode API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-set-upgrade-mode) to return them to active states: ```console POST _ml/set_upgrade_mode?enabled=false ``` - If you closed all {{ml}} jobs before stopping the nodes, open the jobs and start the datafeeds from {{kib}} or with the [open jobs](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-open-job) and [start datafeed](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-start-datafeed) APIs. - - + If you closed all {{ml}} jobs before stopping the nodes, open the jobs and start the datafeeds from {{kib}} or with the [open jobs](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-open-job) and [start datafeed](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-start-datafeed) APIs. ## Rolling restart [restart-cluster-rolling] 1. **Disable shard allocation.** - - When you shut down a data node, the allocation process waits for `index.unassigned.node_left.delayed_timeout` (by default, one minute) before starting to replicate the shards on that node to other nodes in the cluster, which can involve a lot of I/O. Since the node is shortly going to be restarted, this I/O is unnecessary. You can avoid racing the clock by [disabling allocation](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html#cluster-routing-allocation-enable) of replicas before shutting down [data nodes](../../distributed-architecture/clusters-nodes-shards/node-roles.md#data-node-role): + When you shut down a data node, the allocation process waits for `index.unassigned.node_left.delayed_timeout` (by default, one minute) before starting to replicate the shards on that node to other nodes in the cluster, which can involve a lot of I/O. Since the node is shortly going to be restarted, this I/O is unnecessary. You can avoid racing the clock by [disabling allocation](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html#cluster-routing-allocation-enable) of replicas before shutting down [data nodes](../../distributed-architecture/clusters-nodes-shards/node-roles.md#data-node-role): ```console PUT _cluster/settings @@ -147,33 +135,29 @@ Nodes exceeding the low watermark threshold will be slow to restart. Reduce the } ``` - You can also consider [gateway settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html) when restarting large clusters to reduce initial strain while nodes are processing [through discovery](../../distributed-architecture/discovery-cluster-formation.md). + You can also consider [gateway settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html) when restarting large clusters to reduce initial strain while nodes are processing [through discovery](../../distributed-architecture/discovery-cluster-formation.md). 2. **Stop non-essential indexing and perform a flush.** (Optional) - - While you can continue indexing during the rolling restart, shard recovery can be faster if you temporarily stop non-essential indexing and perform a [flush](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-flush). + While you can continue indexing during the rolling restart, shard recovery can be faster if you temporarily stop non-essential indexing and perform a [flush](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-flush). ```console POST /_flush ``` 3. **Temporarily stop the tasks associated with active {{ml}} jobs and {{dfeeds}}.** (Optional) - - {{ml-cap}} features require specific [subscriptions](https://www.elastic.co/subscriptions). - - You have two options to handle {{ml}} jobs and {{dfeeds}} when you shut down a cluster: - - * Temporarily halt the tasks associated with your {{ml}} jobs and {{dfeeds}} and prevent new jobs from opening by using the [set upgrade mode API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-set-upgrade-mode): + {{ml-cap}} features require specific [subscriptions](https://www.elastic.co/subscriptions). + You have two options to handle {{ml}} jobs and {{dfeeds}} when you shut down a cluster: + * Temporarily halt the tasks associated with your {{ml}} jobs and {{dfeeds}} and prevent new jobs from opening by using the [set upgrade mode API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-set-upgrade-mode): ```console POST _ml/set_upgrade_mode?enabled=true ``` - When you disable upgrade mode, the jobs resume using the last model state that was automatically saved. This option avoids the overhead of managing active jobs during the shutdown and is faster than explicitly stopping {{dfeeds}} and closing jobs. + When you disable upgrade mode, the jobs resume using the last model state that was automatically saved. This option avoids the overhead of managing active jobs during the shutdown and is faster than explicitly stopping {{dfeeds}} and closing jobs. - * [Stop all {{dfeeds}} and close all jobs](https://www.elastic.co/guide/en/machine-learning/current/ml-ad-run-jobs.html#ml-ad-close-job). This option saves the model state at the time of closure. When you reopen the jobs after the cluster restart, they use the exact same model. However, saving the latest model state takes longer than using upgrade mode, especially if you have a lot of jobs or jobs with large model states. + * [Stop all {{dfeeds}} and close all jobs](https://www.elastic.co/guide/en/machine-learning/current/ml-ad-run-jobs.html#ml-ad-close-job). This option saves the model state at the time of closure. When you reopen the jobs after the cluster restart, they use the exact same model. However, saving the latest model state takes longer than using upgrade mode, especially if you have a lot of jobs or jobs with large model states. - * If you perform a rolling restart, you can also leave your machine learning jobs running. When you shut down a machine learning node, its jobs automatically move to another node and restore the model states. This option enables your jobs to continue running during the shutdown but it puts increased load on the cluster. + * If you perform a rolling restart, you can also leave your machine learning jobs running. When you shut down a machine learning node, its jobs automatically move to another node and restore the model states. This option enables your jobs to continue running during the shutdown but it puts increased load on the cluster. 4. **Shut down a single node in case of rolling restart.** @@ -197,16 +181,14 @@ Nodes exceeding the low watermark threshold will be slow to restart. Reduce the 5. **Perform any needed changes.** 6. **Restart the node you changed.** - - Start the node and confirm that it joins the cluster by checking the log file or by submitting a `_cat/nodes` request: + Start the node and confirm that it joins the cluster by checking the log file or by submitting a `_cat/nodes` request: ```console GET _cat/nodes ``` 7. **Reenable shard allocation.** - - For data nodes, once the node has joined the cluster, remove the `cluster.routing.allocation.enable` setting to enable shard allocation and start using the node: + For data nodes, once the node has joined the cluster, remove the `cluster.routing.allocation.enable` setting to enable shard allocation and start using the node: ```console PUT _cluster/settings @@ -218,17 +200,13 @@ Nodes exceeding the low watermark threshold will be slow to restart. Reduce the ``` 8. **Repeat in case of rolling restart.** - - When the node has recovered and the cluster is stable, repeat these steps for each node that needs to be changed. + When the node has recovered and the cluster is stable, repeat these steps for each node that needs to be changed. 9. **Restart machine learning jobs.** (Optional) - - If you temporarily halted the tasks associated with your {{ml}} jobs, use the [set upgrade mode API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-set-upgrade-mode) to return them to active states: + If you temporarily halted the tasks associated with your {{ml}} jobs, use the [set upgrade mode API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-set-upgrade-mode) to return them to active states: ```console POST _ml/set_upgrade_mode?enabled=false ``` - If you closed all {{ml}} jobs before stopping the nodes, open the jobs and start the datafeeds from {{kib}} or with the [open jobs](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-open-job) and [start datafeed](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-start-datafeed) APIs. - - + If you closed all {{ml}} jobs before stopping the nodes, open the jobs and start the datafeeds from {{kib}} or with the [open jobs](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-open-job) and [start datafeed](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-start-datafeed) APIs. diff --git a/deploy-manage/maintenance/start-stop-services/restart-an-ece-deployment.md b/deploy-manage/maintenance/start-stop-services/restart-an-ece-deployment.md index bf5b1e0a4d..55d4697ac5 100644 --- a/deploy-manage/maintenance/start-stop-services/restart-an-ece-deployment.md +++ b/deploy-manage/maintenance/start-stop-services/restart-an-ece-deployment.md @@ -1,6 +1,9 @@ --- mapped_pages: - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-restart-deployment.html +applies_to: + deployment: + ece: --- # Restart an ECE deployment [ece-restart-deployment] @@ -15,4 +18,3 @@ To restart a running or a stopped deployment: 3. In the **Deployment Management** section, select **Restart** and follow the steps to restart a deployment. You can also edit the configuration of the deployment before restarting it. - diff --git a/deploy-manage/maintenance/start-stop-services/restart-cloud-hosted-deployment.md b/deploy-manage/maintenance/start-stop-services/restart-cloud-hosted-deployment.md index f6dba06f83..33ded47d53 100644 --- a/deploy-manage/maintenance/start-stop-services/restart-cloud-hosted-deployment.md +++ b/deploy-manage/maintenance/start-stop-services/restart-cloud-hosted-deployment.md @@ -2,23 +2,49 @@ mapped_urls: - https://www.elastic.co/guide/en/cloud/current/ec-restart-deployment.html - https://www.elastic.co/guide/en/cloud/current/ec-api-deployment-other.html +applies_to: + deployment: + ess: --- # Restart a Cloud Hosted deployment -% What needs to be done: Refine +You can restart your {{es}} deployment through the deployment overview UI or by using an API. -% Scope notes: api example +## Restart your deployment through the deployment overview [ec-restart-deployment] -% Use migrated content from existing pages that map to this page: +You might need to restart your deployment while addressing issues, like cycling garbage collection. -% - [ ] ./raw-migrated-files/cloud/cloud/ec-restart-deployment.md -% - [ ] ./raw-migrated-files/cloud/cloud/ec-api-deployment-other.md -% Notes: api example +On the deployment overview, from the **Action** drop-down menu select **Restart {{es}}**. -⚠️ **This page is a work in progress.** ⚠️ +You can choose to restart without downtime or you can restart all nodes simultaneously. -The documentation team is working to combine content pulled from the following pages: +Note that if you are looking to restart {{es}} to clear out [deployment activity](../../../deploy-manage/deploy/elastic-cloud/keep-track-of-deployment-activity.md) plan failures, you may instead run a [no-op plan](../../../troubleshoot/monitoring/deployment-health-warnings.md) to re-synchronize the last successful configuration settings between Elasticsearch Service and {{es}}. -* [/raw-migrated-files/cloud/cloud/ec-restart-deployment.md](/raw-migrated-files/cloud/cloud/ec-restart-deployment.md) -* [/raw-migrated-files/cloud/cloud/ec-api-deployment-other.md](/raw-migrated-files/cloud/cloud/ec-api-deployment-other.md) \ No newline at end of file +## Restart an {{es}} resource by using an API [ec_restart_an_elasticsearch_resource] + +Restart an Elasticsearch resource by calling the following API request: + +```sh +curl -XPOST \ +-H "Authorization: ApiKey $EC_API_KEY" \ +"https://api.elastic-cloud.com/api/v1/deployments/$DEPLOYMENT_ID/$RESOURCE_KIND/$REF_ID/_restart" +``` + +`DEPLOYMENT_ID` The ID of the deployment returned in the response for `POST /deployments` + +`RESOURCE_KIND` Type of the deployment resource. Depending on the version to be deployed, it can be `elasticsearch`, `kibana`, `apm`, `integrations_server`, `appsearch` or `enterprise_search` + +`REF_ID` Name given to each resource type in the attribute `ref_id`. `main-elasticsearch` in the preceding example + +## Shut down a Elasticsearch Service deployment [ec_shut_down_a_elasticsearch_service_deployment] + +Shut down a Elasticsearch Service deployment by calling the following API request: + +```sh +curl -XPOST \ +-H "Authorization: ApiKey $EC_API_KEY" \ +"https://api.elastic-cloud.com/api/v1/deployments/$DEPLOYMENT_ID/_shutdown" +``` + +`DEPLOYMENT_ID` The ID of the deployment returned in the response for `POST /deployments` diff --git a/deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md b/deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md index 48ba2743c8..27cd822ad2 100644 --- a/deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md +++ b/deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md @@ -2,32 +2,336 @@ mapped_urls: - https://www.elastic.co/guide/en/elasticsearch/reference/current/starting-elasticsearch.html - https://www.elastic.co/guide/en/elasticsearch/reference/current/stopping-elasticsearch.html +applies_to: + deployment: + self: --- # Start and stop Elasticsearch -% What needs to be done: Refine +Understanding how to properly start and stop Elasticsearch is essential for maintaining a stable and efficient cluster. This guide outlines the recommended methods for starting and stopping {{es}} safely, considering the different installation types, including package-based installations, Docker containers, and manually extracted archives. -% GitHub issue: https://github.com/elastic/docs-projects/issues/353 +## Starting Elasticsearch [starting-elasticsearch] -% Scope notes: Combine into one topic +The method for starting {{es}} varies depending on how you installed it. -% Use migrated content from existing pages that map to this page: +### Archive packages (`.tar.gz`) [start-targz] -% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/starting-elasticsearch.md -% - [ ] ./raw-migrated-files/elasticsearch/elasticsearch-reference/stopping-elasticsearch.md +If you installed {{es}} with a `.tar.gz` package, you can start {{es}} from the command line. -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +#### Run {{es}} from the command line [_run_es_from_the_command_line] -$$$start-deb$$$ +Run the following command to start {{es}} from the command line: -$$$start-rpm$$$ +```sh +./bin/elasticsearch +``` -$$$_enroll_nodes_in_an_existing_cluster_3$$$ +When starting {{es}} for the first time, security features are enabled and configured by default. The following security configuration occurs automatically: -$$$start-es-deb-systemd$$$ +* Authentication and authorization are enabled, and a password is generated for the `elastic` built-in superuser. +* Certificates and keys for TLS are generated for the transport and HTTP layer, and TLS is enabled and configured with these keys and certificates. +* An enrollment token is generated for {{kib}}, which is valid for 30 minutes. -**This page is a work in progress.** The documentation team is working to combine content pulled from the following pages: +The password for the `elastic` user and the enrollment token for {{kib}} are output to your terminal. -* [/raw-migrated-files/elasticsearch/elasticsearch-reference/starting-elasticsearch.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/starting-elasticsearch.md) -* [/raw-migrated-files/elasticsearch/elasticsearch-reference/stopping-elasticsearch.md](/raw-migrated-files/elasticsearch/elasticsearch-reference/stopping-elasticsearch.md) \ No newline at end of file +We recommend storing the `elastic` password as an environment variable in your shell. Example: + +```sh +export ELASTIC_PASSWORD="your_password" +``` + +If you have password-protected the {{es}} keystore, you will be prompted to enter the keystore’s password. See [Secure settings](../../../deploy-manage/security/secure-settings.md) for more details. + +By default {{es}} prints its logs to the console (`stdout`) and to the `.log` file within the [logs directory](../../../deploy-manage/deploy/self-managed/important-settings-configuration.md#path-settings). {{es}} logs some information while it is starting, but after it has finished initializing it will continue to run in the foreground and won’t log anything further until something happens that is worth recording. While {{es}} is running you can interact with it through its HTTP interface which is on port `9200` by default. + +To stop {{es}}, press `Ctrl-C`. + +::::{note} +All scripts packaged with {{es}} require a version of Bash that supports arrays and assume that Bash is available at `/bin/bash`. As such, Bash should be available at this path either directly or via a symbolic link. +:::: + +#### Run as a daemon [_run_as_a_daemon] + +To run Elasticsearch as a daemon, specify `-d` on the command line, and record the process ID in a file using the `-p` option: + +```sh +./bin/elasticsearch -d -p pid +``` + +If you have password-protected the {{es}} keystore, you will be prompted to enter the keystore’s password. Refer to [Secure settings](../../../deploy-manage/security/secure-settings.md) for more details. + +Log messages can be found in the `$ES_HOME/logs/` directory. + +To shut down Elasticsearch, kill the process ID recorded in the `pid` file: + +```sh +pkill -F pid +``` + +::::{note} +The {{es}} `.tar.gz` package does not include the `systemd` module. To manage {{es}} as a service, use the [Debian](../../../deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md#start-deb) or [RPM](../../../deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md#start-rpm) package instead. +:::: + +### Archive packages (`.zip`) [start-zip] + +If you installed {{es}} on Windows with a `.zip` package, you can start {{es}} from the command line. If you want {{es}} to start automatically at boot time without any user interaction, [install {{es}} as a service](../../../deploy-manage/deploy/self-managed/install-elasticsearch-with-zip-on-windows.md#windows-service). + +#### Run {{es}} from the command line [_run_es_from_the_command_line_2] + +Run the following command to start {{es}} from the command line: + +```sh +.\bin\elasticsearch.bat +``` + +When starting {{es}} for the first time, security features are enabled and configured by default. The following security configuration occurs automatically: + +* Authentication and authorization are enabled, and a password is generated for the `elastic` built-in superuser. +* Certificates and keys for TLS are generated for the transport and HTTP layer, and TLS is enabled and configured with these keys and certificates. +* An enrollment token is generated for {{kib}}, which is valid for 30 minutes. + +The password for the `elastic` user and the enrollment token for {{kib}} are output to your terminal. + +We recommend storing the `elastic` password as an environment variable in your shell. Example: + +```sh +$ELASTIC_PASSWORD = "your_password" +``` + +If you have password-protected the {{es}} keystore, you will be prompted to enter the keystore’s password. See [Secure settings](../../../deploy-manage/security/secure-settings.md) for more details. + +By default {{es}} prints its logs to the console (`STDOUT`) and to the `.log` file within the [logs directory](../../../deploy-manage/deploy/self-managed/important-settings-configuration.md#path-settings). {{es}} logs some information while it is starting, but after it has finished initializing it will continue to run in the foreground and won’t log anything further until something happens that is worth recording. While {{es}} is running you can interact with it through its HTTP interface which is on port `9200` by default. + +To stop {{es}}, press `Ctrl-C`. + +### Debian packages [start-deb] + +#### Running Elasticsearch with `systemd` [start-es-deb-systemd] + +To configure Elasticsearch to start automatically when the system boots up, run the following commands: + +```sh +sudo /bin/systemctl daemon-reload +sudo /bin/systemctl enable elasticsearch.service +``` + +Elasticsearch can be started and stopped as follows: + +```sh +sudo systemctl start elasticsearch.service +sudo systemctl stop elasticsearch.service +``` + +These commands provide no feedback as to whether Elasticsearch was started successfully or not. Instead, this information will be written in the log files located in `/var/log/elasticsearch/`. + +If you have password-protected your {{es}} keystore, you will need to provide `systemd` with the keystore password using a local file and systemd environment variables. This local file should be protected while it exists and may be safely deleted once Elasticsearch is up and running. + +```sh +echo "keystore_password" > /path/to/my_pwd_file.tmp +chmod 600 /path/to/my_pwd_file.tmp +sudo systemctl set-environment ES_KEYSTORE_PASSPHRASE_FILE=/path/to/my_pwd_file.tmp +sudo systemctl start elasticsearch.service +``` + +By default the Elasticsearch service doesn’t log information in the `systemd` journal. To enable `journalctl` logging, the `--quiet` option must be removed from the `ExecStart` command line in the `elasticsearch.service` file. + +When `systemd` logging is enabled, the logging information are available using the `journalctl` commands: + +To tail the journal: + +```sh +sudo journalctl -f +``` + +To list journal entries for the elasticsearch service: + +```sh +sudo journalctl --unit elasticsearch +``` + +To list journal entries for the elasticsearch service starting from a given time: + +```sh +sudo journalctl --unit elasticsearch --since "2016-10-30 18:17:16" +``` + +Check `man journalctl` or [https://www.freedesktop.org/software/systemd/man/journalctl.html](https://www.freedesktop.org/software/systemd/man/journalctl.md) for more command line options. + +::::{admonition} Startup timeouts with older `systemd` versions +:class: tip + +By default {{es}} sets the `TimeoutStartSec` parameter to `systemd` to `900s`. If you are running at least version 238 of `systemd` then {{es}} can automatically extend the startup timeout, and will do so repeatedly until startup is complete even if it takes longer than 900s. + +Versions of `systemd` prior to 238 do not support the timeout extension mechanism and will terminate the {{es}} process if it has not fully started up within the configured timeout. If this happens, {{es}} will report in its logs that it was shut down normally a short time after it started: + +```text +[2022-01-31T01:22:31,077][INFO ][o.e.n.Node ] [instance-0000000123] starting ... +... +[2022-01-31T01:37:15,077][INFO ][o.e.n.Node ] [instance-0000000123] stopping ... +``` + +However the `systemd` logs will report that the startup timed out: + +```text +Jan 31 01:22:30 debian systemd[1]: Starting Elasticsearch... +Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Start operation timed out. Terminating. +Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Main process exited, code=killed, status=15/TERM +Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Failed with result 'timeout'. +Jan 31 01:37:15 debian systemd[1]: Failed to start Elasticsearch. +``` + +To avoid this, upgrade your `systemd` to at least version 238. You can also temporarily work around the problem by extending the `TimeoutStartSec` parameter. + +:::: + +### Docker images [start-docker] + +If you installed a Docker image, you can start {{es}} from the command line. There are different methods depending on whether you’re using development mode or production mode. See [Run {{es}} in Docker](../../../deploy-manage/deploy/self-managed/install-elasticsearch-with-docker.md#docker-cli-run-dev-mode). + +### RPM packages [start-rpm] + +#### Running Elasticsearch with `systemd` [start-es-rpm-systemd] + +To configure Elasticsearch to start automatically when the system boots up, run the following commands: + +```sh +sudo /bin/systemctl daemon-reload +sudo /bin/systemctl enable elasticsearch.service +``` + +Elasticsearch can be started and stopped as follows: + +```sh +sudo systemctl start elasticsearch.service +sudo systemctl stop elasticsearch.service +``` + +These commands provide no feedback as to whether Elasticsearch was started successfully or not. Instead, this information will be written in the log files located in `/var/log/elasticsearch/`. + +If you have password-protected your {{es}} keystore, you will need to provide `systemd` with the keystore password using a local file and systemd environment variables. This local file should be protected while it exists and may be safely deleted once Elasticsearch is up and running. + +```sh +echo "keystore_password" > /path/to/my_pwd_file.tmp +chmod 600 /path/to/my_pwd_file.tmp +sudo systemctl set-environment ES_KEYSTORE_PASSPHRASE_FILE=/path/to/my_pwd_file.tmp +sudo systemctl start elasticsearch.service +``` + +By default the Elasticsearch service doesn’t log information in the `systemd` journal. To enable `journalctl` logging, the `--quiet` option must be removed from the `ExecStart` command line in the `elasticsearch.service` file. + +When `systemd` logging is enabled, the logging information are available using the `journalctl` commands: + +To tail the journal: + +```sh +sudo journalctl -f +``` + +To list journal entries for the elasticsearch service: + +```sh +sudo journalctl --unit elasticsearch +``` + +To list journal entries for the elasticsearch service starting from a given time: + +```sh +sudo journalctl --unit elasticsearch --since "2016-10-30 18:17:16" +``` + +Check `man journalctl` or [https://www.freedesktop.org/software/systemd/man/journalctl.html](https://www.freedesktop.org/software/systemd/man/journalctl.md) for more command line options. + +::::{admonition} Startup timeouts with older `systemd` versions +:class: tip + +By default {{es}} sets the `TimeoutStartSec` parameter to `systemd` to `900s`. If you are running at least version 238 of `systemd` then {{es}} can automatically extend the startup timeout, and will do so repeatedly until startup is complete even if it takes longer than 900s. + +Versions of `systemd` prior to 238 do not support the timeout extension mechanism and will terminate the {{es}} process if it has not fully started up within the configured timeout. If this happens, {{es}} will report in its logs that it was shut down normally a short time after it started: + +```text +[2022-01-31T01:22:31,077][INFO ][o.e.n.Node ] [instance-0000000123] starting ... +... +[2022-01-31T01:37:15,077][INFO ][o.e.n.Node ] [instance-0000000123] stopping ... +``` + +However the `systemd` logs will report that the startup timed out: + +```text +Jan 31 01:22:30 debian systemd[1]: Starting Elasticsearch... +Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Start operation timed out. Terminating. +Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Main process exited, code=killed, status=15/TERM +Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Failed with result 'timeout'. +Jan 31 01:37:15 debian systemd[1]: Failed to start Elasticsearch. +``` + +To avoid this, upgrade your `systemd` to at least version 238. You can also temporarily work around the problem by extending the `TimeoutStartSec` parameter. + +:::: + +## Stopping Elasticsearch [stopping-elasticsearch] + +An orderly shutdown of Elasticsearch ensures that Elasticsearch has a chance to cleanup and close outstanding resources. For example, a node that is shutdown in an orderly fashion will remove itself from the cluster, sync translogs to disk, and perform other related cleanup activities. You can help ensure an orderly shutdown by properly stopping Elasticsearch. + +If you’re running Elasticsearch as a service, you can stop Elasticsearch via the service management functionality provided by your installation. + +If you’re running Elasticsearch directly, you can stop Elasticsearch by sending control-C if you’re running Elasticsearch in the console, or by sending `SIGTERM` to the Elasticsearch process on a POSIX system. You can obtain the PID to send the signal to via various tools (for example, `ps` or `jps`): + +```sh +$ jps | grep Elasticsearch +14542 Elasticsearch +``` + +From the Elasticsearch startup logs: + +```sh +[2016-07-07 12:26:18,908][INFO ][node ] [I8hydUG] version[5.0.0-alpha4], pid[15399], build[3f5b994/2016-06-27T16:23:46.861Z], OS[Mac OS X/10.11.5/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_92/25.92-b14] +``` + +Or by specifying a location to write a PID file to on startup (`-p `): + +```sh +$ ./bin/elasticsearch -p /tmp/elasticsearch-pid -d +$ cat /tmp/elasticsearch-pid && echo +15516 +$ kill -SIGTERM 15516 +``` + +### Stopping on Fatal Errors [fatal-errors] + +During the life of the Elasticsearch virtual machine, certain fatal errors could arise that put the virtual machine in a questionable state. Such fatal errors include out of memory errors, internal errors in virtual machine, and serious I/O errors. + +When Elasticsearch detects that the virtual machine has encountered such a fatal error Elasticsearch will attempt to log the error and then will halt the virtual machine. When Elasticsearch initiates such a shutdown, it does not go through an orderly shutdown as described above. The Elasticsearch process will also return with a special status code indicating the nature of the error. + +Killed by jvmkiller agent +: 158 + +User or kernel SIGTERM +: 143 + +Slain by kernel oom-killer +: 137 + +Segmentation fault +: 134 + +JVM internal error +: 128 + +Out of memory error +: 127 + +Stack overflow error +: 126 + +Unknown virtual machine error +: 125 + +Serious I/O error +: 124 + +Bootstrap check failure +: 78 + +Unknown fatal error +: 1 diff --git a/deploy-manage/maintenance/start-stop-services/start-stop-kibana.md b/deploy-manage/maintenance/start-stop-services/start-stop-kibana.md index 8600febf36..a55e4e8d29 100644 --- a/deploy-manage/maintenance/start-stop-services/start-stop-kibana.md +++ b/deploy-manage/maintenance/start-stop-services/start-stop-kibana.md @@ -1,19 +1,20 @@ --- mapped_pages: - https://www.elastic.co/guide/en/kibana/current/start-stop.html +applies_to: + deployment: + self: --- # Start and stop Kibana [start-stop] The method for starting and stopping {{kib}} varies depending on how you installed it. If a password protected keystore is used, the environment variable `KBN_KEYSTORE_PASSPHRASE_FILE` can be used to point to a file containing the password, the environment variable `KEYSTORE_PASSWORD` can be defined, or you will be prompted to enter to enter the password on startup, - -## Archive packages (`.tar.gz`) [start-start-targz] +## Archive packages (`.tar.gz`) [start-start-targz] If you installed {{kib}} on Linux or Darwin with a `.tar.gz` package, you can start and stop {{kib}} from the command line. - -### Run {{kib}} from the command line [_run_kib_from_the_command_line] +### Run {{kib}} from the command line [run-kibana-from-command-line] Kibana can be started from the command line as follows: @@ -34,14 +35,11 @@ If you need to reset the password for the `elastic` user or other built-in users :::: - - -## Archive packages (`.zip`) [start-stop-zip] +## Archive packages (`.zip`) [start-stop-zip] If you installed {{kib}} on Windows with a `.zip` package, you can stop and start {{kib}} from the command line. - -### Run {{kib}} from the command line [_run_kib_from_the_command_line_2] +### Run {{kib}} from the command line [_run_kib_from_the_command_line_2] Kibana can be started from the command line as follows: @@ -62,12 +60,9 @@ If you need to reset the password for the `elastic` user or other built-in users :::: +## Debian and RPM packages [start-stop-deb-rpm] - -## Debian and RPM packages [start-stop-deb-rpm] - - -### Run {{kib}} with `systemd` [_run_kib_with_systemd] +### Run {{kib}} with `systemd` [_run_kib_with_systemd] To configure {{kib}} to start automatically when the system starts, run the following commands: @@ -83,8 +78,4 @@ sudo systemctl start kibana.service sudo systemctl stop kibana.service ``` -These commands provide no feedback as to whether {{kib}} was started successfully or not. Log information can be accessed via `journalctl -u kibana.service`. - - - -$$$_run_kibana_from_the_command_line$$$ \ No newline at end of file +These commands provide no feedback as to whether {{kib}} was started successfully or not. Log information can be accessed via `journalctl -u kibana.service`. \ No newline at end of file diff --git a/raw-migrated-files/cloud/cloud-enterprise/ece-maintenance-mode-routing.md b/raw-migrated-files/cloud/cloud-enterprise/ece-maintenance-mode-routing.md deleted file mode 100644 index 0b5fe1236b..0000000000 --- a/raw-migrated-files/cloud/cloud-enterprise/ece-maintenance-mode-routing.md +++ /dev/null @@ -1,30 +0,0 @@ -# Request routing [ece-maintenance-mode-routing] - -The {{ecloud}} proxy service routes traffic from external sources to the deployment, between deployments, and between products within a deployment. For example, it routes API requests from your local machine to your deployment, CCR and CCS requests between your deployments, and communications between {{kib}} and {{es}}. It does not direct the TCP traffic between {{es}} nodes, nor does it manage requests starting within {{es}} outwards to external sources such as to snapshot repositories. - -The {{ecloud}} proxy routes HTTP requests to its deployment’s individual product instances through the product’s endpoint. By default, instances are enabled to route HTTP traffic and will report no special messaging. - -It might be helpful to temporarily block upstream requests in order to protect some or all instances or products within your deployment. For example, you might stop request routing in the following cases: - -* If another team within your company starts streaming new data into your production {{integrations-server}} without previous load testing, both it and {{es}} might experience performance issues. You might consider stopping routing requests on all {{integrations-server}} instances in order to protect your downstream {{es}} instance. -* If {{es}} is being overwhelmed by upstream requests, it might experience increased response times or even become unresponsive. This might impact your ability to resize components in your deployment and increase the duration of pending plans or increase the chance of plan changes failing. Because every {{es}} node is an [implicit coordinating node](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html), you should stop routing requests across all {{es}} nodes to completely block upstream traffic. - - -## Considerations [ece_considerations] - -* {{ecloud}} will automatically set and remove routing blocks during plan changes. Elastic recommends avoiding manually overriding these settings for a deployment while its plans are pending. -* The [{{es}} API console](../../../explore-analyze/query-filter/tools/console.md) bypasses {{ecloud}} proxy routing blocks against {{es}} to enable administrative tasks while plan changes are pending. You should generally default traffic to the {{es}} endpoint. However, if you enable **Stop routing requests** across all {{es}} nodes, you need to use this UI to administer your cluster. -* While {{es}} has **Stop routing requests** set across all nodes, other products with the deployment may become unhealthy. This is because {{es}} is a prerequisite for those other products, such as {{kib}}. In {{kib}}, this results in a [**Kibana server is not ready yet**](/troubleshoot/kibana/error-server-not-ready.md) message. - - -## Stop routing requests [ece_stop_routing_requests] - -To block HTTP requests for an instance, select **Stop routing requests** under from instance’s menu. - -The instance will then report **Not routing requests**. It will complete existing requested traffic, but not be sent new requests. - - -## Restart routing requests [ece_restart_routing_requests] - -To unblock HTTP requests for an instance, select **Start routing requests** under from instance’s menu. - diff --git a/raw-migrated-files/cloud/cloud/ec-api-deployment-other.md b/raw-migrated-files/cloud/cloud/ec-api-deployment-other.md deleted file mode 100644 index ac298a22ed..0000000000 --- a/raw-migrated-files/cloud/cloud/ec-api-deployment-other.md +++ /dev/null @@ -1,32 +0,0 @@ -# Other deployment operations [ec-api-deployment-other] - - -## Restart an Elasticsearch resource [ec_restart_an_elasticsearch_resource] - -Restart an Elasticsearch resource. - -```sh -curl -XPOST \ --H "Authorization: ApiKey $EC_API_KEY" \ -"https://api.elastic-cloud.com/api/v1/deployments/$DEPLOYMENT_ID/$RESOURCE_KIND/$REF_ID/_restart" -``` - -`DEPLOYMENT_ID` The ID of the deployment returned in the response for `POST /deployments` - -`RESOURCE_KIND` Type of the deployment resource. Depending on the version to be deployed, it can be `elasticsearch`, `kibana`, `apm`, `integrations_server`, `appsearch` or `enterprise_search` - -`REF_ID` Name given to each resource type in the attribute `ref_id`. `main-elasticsearch` in the preceding example - - -## Shut down a Elasticsearch Service deployment [ec_shut_down_a_elasticsearch_service_deployment] - -Shut down a Elasticsearch Service deployment. - -```sh -curl -XPOST \ --H "Authorization: ApiKey $EC_API_KEY" \ -"https://api.elastic-cloud.com/api/v1/deployments/$DEPLOYMENT_ID/_shutdown" -``` - -`DEPLOYMENT_ID` The ID of the deployment returned in the response for `POST /deployments` - diff --git a/raw-migrated-files/cloud/cloud/ec-restart-deployment.md b/raw-migrated-files/cloud/cloud/ec-restart-deployment.md deleted file mode 100644 index 58438135cd..0000000000 --- a/raw-migrated-files/cloud/cloud/ec-restart-deployment.md +++ /dev/null @@ -1,10 +0,0 @@ -# Restart your deployment [ec-restart-deployment] - -You might need to restart your deployment while addressing issues, like cycling garbage collection. - -On the deployment overview, from the **Action** drop-down menu select **Restart {{es}}**. - -You can choose to restart without downtime or you can restart all nodes simultaneously. - -Note that if you are looking to restart {{es}} to clear out [deployment activity](../../../deploy-manage/deploy/elastic-cloud/keep-track-of-deployment-activity.md) plan failures, you may instead run a [no-op plan](../../../troubleshoot/monitoring/deployment-health-warnings.md) to re-synchronize the last successful configuration settings between Elasticsearch Service and {{es}}. - diff --git a/raw-migrated-files/elasticsearch/elasticsearch-reference/security-basic-setup.md b/raw-migrated-files/elasticsearch/elasticsearch-reference/security-basic-setup.md index 41c6fd8324..d7652bb592 100644 --- a/raw-migrated-files/elasticsearch/elasticsearch-reference/security-basic-setup.md +++ b/raw-migrated-files/elasticsearch/elasticsearch-reference/security-basic-setup.md @@ -114,7 +114,7 @@ Complete the following steps **for each node in your cluster**. To join the same ``` 3. Complete the previous steps for each node in your cluster. -4. On **every** node in your cluster, start {{es}}. The method for [starting](starting-elasticsearch.md) and [stopping](starting-elasticsearch.md) {{es}} varies depending on how you installed it. +4. On **every** node in your cluster, start {{es}}. The method for [starting and stopping](../../../deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md) {{es}} varies depending on how you installed it. For example, if you installed {{es}} with an archive distribution (`tar.gz` or `.zip`), you can enter `Ctrl+C` on the command line to stop {{es}}. diff --git a/raw-migrated-files/elasticsearch/elasticsearch-reference/starting-elasticsearch.md b/raw-migrated-files/elasticsearch/elasticsearch-reference/starting-elasticsearch.md deleted file mode 100644 index 912e8616f1..0000000000 --- a/raw-migrated-files/elasticsearch/elasticsearch-reference/starting-elasticsearch.md +++ /dev/null @@ -1,338 +0,0 @@ -# Starting Elasticsearch [starting-elasticsearch] - -The method for starting {{es}} varies depending on how you installed it. - - -## Archive packages (`.tar.gz`) [start-targz] - -If you installed {{es}} with a `.tar.gz` package, you can start {{es}} from the command line. - - -### Run {{es}} from the command line [_run_es_from_the_command_line] - -Run the following command to start {{es}} from the command line: - -```sh -./bin/elasticsearch -``` - -When starting {{es}} for the first time, security features are enabled and configured by default. The following security configuration occurs automatically: - -* Authentication and authorization are enabled, and a password is generated for the `elastic` built-in superuser. -* Certificates and keys for TLS are generated for the transport and HTTP layer, and TLS is enabled and configured with these keys and certificates. -* An enrollment token is generated for {{kib}}, which is valid for 30 minutes. - -The password for the `elastic` user and the enrollment token for {{kib}} are output to your terminal. - -We recommend storing the `elastic` password as an environment variable in your shell. Example: - -```sh -export ELASTIC_PASSWORD="your_password" -``` - -If you have password-protected the {{es}} keystore, you will be prompted to enter the keystore’s password. See [Secure settings](../../../deploy-manage/security/secure-settings.md) for more details. - -By default {{es}} prints its logs to the console (`stdout`) and to the `.log` file within the [logs directory](../../../deploy-manage/deploy/self-managed/important-settings-configuration.md#path-settings). {{es}} logs some information while it is starting, but after it has finished initializing it will continue to run in the foreground and won’t log anything further until something happens that is worth recording. While {{es}} is running you can interact with it through its HTTP interface which is on port `9200` by default. - -To stop {{es}}, press `Ctrl-C`. - -::::{note} -All scripts packaged with {{es}} require a version of Bash that supports arrays and assume that Bash is available at `/bin/bash`. As such, Bash should be available at this path either directly or via a symbolic link. -:::: - - - -### Enroll nodes in an existing cluster [_enroll_nodes_in_an_existing_cluster_3] - -When {{es}} starts for the first time, the security auto-configuration process binds the HTTP layer to `0.0.0.0`, but only binds the transport layer to localhost. This intended behavior ensures that you can start a single-node cluster with security enabled by default without any additional configuration. - -Before enrolling a new node, additional actions such as binding to an address other than `localhost` or satisfying bootstrap checks are typically necessary in production clusters. During that time, an auto-generated enrollment token could expire, which is why enrollment tokens aren’t generated automatically. - -Additionally, only nodes on the same host can join the cluster without additional configuration. If you want nodes from another host to join your cluster, you need to set `transport.host` to a [supported value](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#network-interface-values) (such as uncommenting the suggested value of `0.0.0.0`), or an IP address that’s bound to an interface where other hosts can reach it. Refer to [transport settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#transport-settings) for more information. - -To enroll new nodes in your cluster, create an enrollment token with the `elasticsearch-create-enrollment-token` tool on any existing node in your cluster. You can then start a new node with the `--enrollment-token` parameter so that it joins an existing cluster. - -1. In a separate terminal from where {{es}} is running, navigate to the directory where you installed {{es}} and run the [`elasticsearch-create-enrollment-token`](https://www.elastic.co/guide/en/elasticsearch/reference/current/create-enrollment-token.html) tool to generate an enrollment token for your new nodes. - - ```sh - bin/elasticsearch-create-enrollment-token -s node - ``` - - Copy the enrollment token, which you’ll use to enroll new nodes with your {{es}} cluster. - -2. From the installation directory of your new node, start {{es}} and pass the enrollment token with the `--enrollment-token` parameter. - - ```sh - bin/elasticsearch --enrollment-token - ``` - - {{es}} automatically generates certificates and keys in the following directory: - - ```sh - config/certs - ``` - -3. Repeat the previous step for any new nodes that you want to enroll. - - -### Run as a daemon [_run_as_a_daemon] - -To run Elasticsearch as a daemon, specify `-d` on the command line, and record the process ID in a file using the `-p` option: - -```sh -./bin/elasticsearch -d -p pid -``` - -If you have password-protected the {{es}} keystore, you will be prompted to enter the keystore’s password. See [Secure settings](../../../deploy-manage/security/secure-settings.md) for more details. - -Log messages can be found in the `$ES_HOME/logs/` directory. - -To shut down Elasticsearch, kill the process ID recorded in the `pid` file: - -```sh -pkill -F pid -``` - -::::{note} -The {{es}} `.tar.gz` package does not include the `systemd` module. To manage {{es}} as a service, use the [Debian](../../../deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md#start-deb) or [RPM](../../../deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md#start-rpm) package instead. -:::: - - - -## Archive packages (`.zip`) [start-zip] - -If you installed {{es}} on Windows with a `.zip` package, you can start {{es}} from the command line. If you want {{es}} to start automatically at boot time without any user interaction, [install {{es}} as a service](../../../deploy-manage/deploy/self-managed/install-elasticsearch-with-zip-on-windows.md#windows-service). - - -### Run {{es}} from the command line [_run_es_from_the_command_line_2] - -Run the following command to start {{es}} from the command line: - -```sh -.\bin\elasticsearch.bat -``` - -When starting {{es}} for the first time, security features are enabled and configured by default. The following security configuration occurs automatically: - -* Authentication and authorization are enabled, and a password is generated for the `elastic` built-in superuser. -* Certificates and keys for TLS are generated for the transport and HTTP layer, and TLS is enabled and configured with these keys and certificates. -* An enrollment token is generated for {{kib}}, which is valid for 30 minutes. - -The password for the `elastic` user and the enrollment token for {{kib}} are output to your terminal. - -We recommend storing the `elastic` password as an environment variable in your shell. Example: - -```sh -$ELASTIC_PASSWORD = "your_password" -``` - -If you have password-protected the {{es}} keystore, you will be prompted to enter the keystore’s password. See [Secure settings](../../../deploy-manage/security/secure-settings.md) for more details. - -By default {{es}} prints its logs to the console (`STDOUT`) and to the `.log` file within the [logs directory](../../../deploy-manage/deploy/self-managed/important-settings-configuration.md#path-settings). {{es}} logs some information while it is starting, but after it has finished initializing it will continue to run in the foreground and won’t log anything further until something happens that is worth recording. While {{es}} is running you can interact with it through its HTTP interface which is on port `9200` by default. - -To stop {{es}}, press `Ctrl-C`. - - -### Enroll nodes in an existing cluster [_enroll_nodes_in_an_existing_cluster_4] - -When {{es}} starts for the first time, the security auto-configuration process binds the HTTP layer to `0.0.0.0`, but only binds the transport layer to localhost. This intended behavior ensures that you can start a single-node cluster with security enabled by default without any additional configuration. - -Before enrolling a new node, additional actions such as binding to an address other than `localhost` or satisfying bootstrap checks are typically necessary in production clusters. During that time, an auto-generated enrollment token could expire, which is why enrollment tokens aren’t generated automatically. - -Additionally, only nodes on the same host can join the cluster without additional configuration. If you want nodes from another host to join your cluster, you need to set `transport.host` to a [supported value](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#network-interface-values) (such as uncommenting the suggested value of `0.0.0.0`), or an IP address that’s bound to an interface where other hosts can reach it. Refer to [transport settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#transport-settings) for more information. - -To enroll new nodes in your cluster, create an enrollment token with the `elasticsearch-create-enrollment-token` tool on any existing node in your cluster. You can then start a new node with the `--enrollment-token` parameter so that it joins an existing cluster. - -1. In a separate terminal from where {{es}} is running, navigate to the directory where you installed {{es}} and run the [`elasticsearch-create-enrollment-token`](https://www.elastic.co/guide/en/elasticsearch/reference/current/create-enrollment-token.html) tool to generate an enrollment token for your new nodes. - - ```sh - bin\elasticsearch-create-enrollment-token -s node - ``` - - Copy the enrollment token, which you’ll use to enroll new nodes with your {{es}} cluster. - -2. From the installation directory of your new node, start {{es}} and pass the enrollment token with the `--enrollment-token` parameter. - - ```sh - bin\elasticsearch --enrollment-token - ``` - - {{es}} automatically generates certificates and keys in the following directory: - - ```sh - config\certs - ``` - -3. Repeat the previous step for any new nodes that you want to enroll. - - -## Debian packages [start-deb] - - -### Running Elasticsearch with `systemd` [start-es-deb-systemd] - -To configure Elasticsearch to start automatically when the system boots up, run the following commands: - -```sh -sudo /bin/systemctl daemon-reload -sudo /bin/systemctl enable elasticsearch.service -``` - -Elasticsearch can be started and stopped as follows: - -```sh -sudo systemctl start elasticsearch.service -sudo systemctl stop elasticsearch.service -``` - -These commands provide no feedback as to whether Elasticsearch was started successfully or not. Instead, this information will be written in the log files located in `/var/log/elasticsearch/`. - -If you have password-protected your {{es}} keystore, you will need to provide `systemd` with the keystore password using a local file and systemd environment variables. This local file should be protected while it exists and may be safely deleted once Elasticsearch is up and running. - -```sh -echo "keystore_password" > /path/to/my_pwd_file.tmp -chmod 600 /path/to/my_pwd_file.tmp -sudo systemctl set-environment ES_KEYSTORE_PASSPHRASE_FILE=/path/to/my_pwd_file.tmp -sudo systemctl start elasticsearch.service -``` - -By default the Elasticsearch service doesn’t log information in the `systemd` journal. To enable `journalctl` logging, the `--quiet` option must be removed from the `ExecStart` command line in the `elasticsearch.service` file. - -When `systemd` logging is enabled, the logging information are available using the `journalctl` commands: - -To tail the journal: - -```sh -sudo journalctl -f -``` - -To list journal entries for the elasticsearch service: - -```sh -sudo journalctl --unit elasticsearch -``` - -To list journal entries for the elasticsearch service starting from a given time: - -```sh -sudo journalctl --unit elasticsearch --since "2016-10-30 18:17:16" -``` - -Check `man journalctl` or [https://www.freedesktop.org/software/systemd/man/journalctl.html](https://www.freedesktop.org/software/systemd/man/journalctl.md) for more command line options. - -::::{admonition} Startup timeouts with older `systemd` versions -:class: tip - -By default {{es}} sets the `TimeoutStartSec` parameter to `systemd` to `900s`. If you are running at least version 238 of `systemd` then {{es}} can automatically extend the startup timeout, and will do so repeatedly until startup is complete even if it takes longer than 900s. - -Versions of `systemd` prior to 238 do not support the timeout extension mechanism and will terminate the {{es}} process if it has not fully started up within the configured timeout. If this happens, {{es}} will report in its logs that it was shut down normally a short time after it started: - -```text -[2022-01-31T01:22:31,077][INFO ][o.e.n.Node ] [instance-0000000123] starting ... -... -[2022-01-31T01:37:15,077][INFO ][o.e.n.Node ] [instance-0000000123] stopping ... -``` - -However the `systemd` logs will report that the startup timed out: - -```text -Jan 31 01:22:30 debian systemd[1]: Starting Elasticsearch... -Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Start operation timed out. Terminating. -Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Main process exited, code=killed, status=15/TERM -Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Failed with result 'timeout'. -Jan 31 01:37:15 debian systemd[1]: Failed to start Elasticsearch. -``` - -To avoid this, upgrade your `systemd` to at least version 238. You can also temporarily work around the problem by extending the `TimeoutStartSec` parameter. - -:::: - - - -## Docker images [start-docker] - -If you installed a Docker image, you can start {{es}} from the command line. There are different methods depending on whether you’re using development mode or production mode. See [Run {{es}} in Docker](../../../deploy-manage/deploy/self-managed/install-elasticsearch-with-docker.md#docker-cli-run-dev-mode). - - -## RPM packages [start-rpm] - - -### Running Elasticsearch with `systemd` [start-es-rpm-systemd] - -To configure Elasticsearch to start automatically when the system boots up, run the following commands: - -```sh -sudo /bin/systemctl daemon-reload -sudo /bin/systemctl enable elasticsearch.service -``` - -Elasticsearch can be started and stopped as follows: - -```sh -sudo systemctl start elasticsearch.service -sudo systemctl stop elasticsearch.service -``` - -These commands provide no feedback as to whether Elasticsearch was started successfully or not. Instead, this information will be written in the log files located in `/var/log/elasticsearch/`. - -If you have password-protected your {{es}} keystore, you will need to provide `systemd` with the keystore password using a local file and systemd environment variables. This local file should be protected while it exists and may be safely deleted once Elasticsearch is up and running. - -```sh -echo "keystore_password" > /path/to/my_pwd_file.tmp -chmod 600 /path/to/my_pwd_file.tmp -sudo systemctl set-environment ES_KEYSTORE_PASSPHRASE_FILE=/path/to/my_pwd_file.tmp -sudo systemctl start elasticsearch.service -``` - -By default the Elasticsearch service doesn’t log information in the `systemd` journal. To enable `journalctl` logging, the `--quiet` option must be removed from the `ExecStart` command line in the `elasticsearch.service` file. - -When `systemd` logging is enabled, the logging information are available using the `journalctl` commands: - -To tail the journal: - -```sh -sudo journalctl -f -``` - -To list journal entries for the elasticsearch service: - -```sh -sudo journalctl --unit elasticsearch -``` - -To list journal entries for the elasticsearch service starting from a given time: - -```sh -sudo journalctl --unit elasticsearch --since "2016-10-30 18:17:16" -``` - -Check `man journalctl` or [https://www.freedesktop.org/software/systemd/man/journalctl.html](https://www.freedesktop.org/software/systemd/man/journalctl.md) for more command line options. - -::::{admonition} Startup timeouts with older `systemd` versions -:class: tip - -By default {{es}} sets the `TimeoutStartSec` parameter to `systemd` to `900s`. If you are running at least version 238 of `systemd` then {{es}} can automatically extend the startup timeout, and will do so repeatedly until startup is complete even if it takes longer than 900s. - -Versions of `systemd` prior to 238 do not support the timeout extension mechanism and will terminate the {{es}} process if it has not fully started up within the configured timeout. If this happens, {{es}} will report in its logs that it was shut down normally a short time after it started: - -```text -[2022-01-31T01:22:31,077][INFO ][o.e.n.Node ] [instance-0000000123] starting ... -... -[2022-01-31T01:37:15,077][INFO ][o.e.n.Node ] [instance-0000000123] stopping ... -``` - -However the `systemd` logs will report that the startup timed out: - -```text -Jan 31 01:22:30 debian systemd[1]: Starting Elasticsearch... -Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Start operation timed out. Terminating. -Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Main process exited, code=killed, status=15/TERM -Jan 31 01:37:15 debian systemd[1]: elasticsearch.service: Failed with result 'timeout'. -Jan 31 01:37:15 debian systemd[1]: Failed to start Elasticsearch. -``` - -To avoid this, upgrade your `systemd` to at least version 238. You can also temporarily work around the problem by extending the `TimeoutStartSec` parameter. - -:::: - - diff --git a/raw-migrated-files/elasticsearch/elasticsearch-reference/stopping-elasticsearch.md b/raw-migrated-files/elasticsearch/elasticsearch-reference/stopping-elasticsearch.md deleted file mode 100644 index e814905816..0000000000 --- a/raw-migrated-files/elasticsearch/elasticsearch-reference/stopping-elasticsearch.md +++ /dev/null @@ -1,68 +0,0 @@ -# Stopping Elasticsearch [stopping-elasticsearch] - -An orderly shutdown of Elasticsearch ensures that Elasticsearch has a chance to cleanup and close outstanding resources. For example, a node that is shutdown in an orderly fashion will remove itself from the cluster, sync translogs to disk, and perform other related cleanup activities. You can help ensure an orderly shutdown by properly stopping Elasticsearch. - -If you’re running Elasticsearch as a service, you can stop Elasticsearch via the service management functionality provided by your installation. - -If you’re running Elasticsearch directly, you can stop Elasticsearch by sending control-C if you’re running Elasticsearch in the console, or by sending `SIGTERM` to the Elasticsearch process on a POSIX system. You can obtain the PID to send the signal to via various tools (e.g., `ps` or `jps`): - -```sh -$ jps | grep Elasticsearch -14542 Elasticsearch -``` - -From the Elasticsearch startup logs: - -```sh -[2016-07-07 12:26:18,908][INFO ][node ] [I8hydUG] version[5.0.0-alpha4], pid[15399], build[3f5b994/2016-06-27T16:23:46.861Z], OS[Mac OS X/10.11.5/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_92/25.92-b14] -``` - -Or by specifying a location to write a PID file to on startup (`-p `): - -```sh -$ ./bin/elasticsearch -p /tmp/elasticsearch-pid -d -$ cat /tmp/elasticsearch-pid && echo -15516 -$ kill -SIGTERM 15516 -``` - - -## Stopping on Fatal Errors [fatal-errors] - -During the life of the Elasticsearch virtual machine, certain fatal errors could arise that put the virtual machine in a questionable state. Such fatal errors include out of memory errors, internal errors in virtual machine, and serious I/O errors. - -When Elasticsearch detects that the virtual machine has encountered such a fatal error Elasticsearch will attempt to log the error and then will halt the virtual machine. When Elasticsearch initiates such a shutdown, it does not go through an orderly shutdown as described above. The Elasticsearch process will also return with a special status code indicating the nature of the error. - -Killed by jvmkiller agent -: 158 - -User or kernel SIGTERM -: 143 - -Slain by kernel oom-killer -: 137 - -Segmentation fault -: 134 - -JVM internal error -: 128 - -Out of memory error -: 127 - -Stack overflow error -: 126 - -Unknown virtual machine error -: 125 - -Serious I/O error -: 124 - -Bootstrap check failure -: 78 - -Unknown fatal error -: 1 - diff --git a/raw-migrated-files/stack-docs/elastic-stack/installing-stack-demo-self.md b/raw-migrated-files/stack-docs/elastic-stack/installing-stack-demo-self.md index b9dd343658..f3a77580aa 100644 --- a/raw-migrated-files/stack-docs/elastic-stack/installing-stack-demo-self.md +++ b/raw-migrated-files/stack-docs/elastic-stack/installing-stack-demo-self.md @@ -276,7 +276,7 @@ To set up a second {{es}} node, the initial steps are similar to those that you Note the following tips about enrollment tokens: 1. An enrollment token has a lifespan of 30 minutes. In case the `elasticsearch-reconfigure-node` command returns an `Invalid enrollment token` error, try generating a new token. - 2. Be sure not to confuse an [{{es}} enrollment token](../../../deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md#_enroll_nodes_in_an_existing_cluster_3) (for enrolling {{es}} nodes in an existing cluster) with a [{{kib}} enrollment token](../../../deploy-manage/maintenance/start-stop-services/start-stop-kibana.md#_run_kibana_from_the_command_line) (to enroll your {{kib}} instance with {{es}}, as described in the next section). These two tokens are not interchangeable. + 2. Be sure not to confuse an [{{es}} enrollment token](../../../deploy-manage/maintenance/add-and-remove-elasticsearch-nodes.md) (for enrolling {{es}} nodes in an existing cluster) with a [{{kib}} enrollment token](../../../deploy-manage/maintenance/start-stop-services/start-stop-kibana.md#run-kibana-from-command-line) (to enroll your {{kib}} instance with {{es}}, as described in the next section). These two tokens are not interchangeable. :::: diff --git a/raw-migrated-files/toc.yml b/raw-migrated-files/toc.yml index e34c939dd1..eb14c7896f 100644 --- a/raw-migrated-files/toc.yml +++ b/raw-migrated-files/toc.yml @@ -50,7 +50,6 @@ toc: - file: cloud/cloud-enterprise/ece-ingest-guides.md - file: cloud/cloud-enterprise/ece-install-offline.md - file: cloud/cloud-enterprise/ece-integrations-server-api-example.md - - file: cloud/cloud-enterprise/ece-maintenance-mode-routing.md - file: cloud/cloud-enterprise/ece-manage-apm-settings.md - file: cloud/cloud-enterprise/ece-manage-enterprise-search-settings.md - file: cloud/cloud-enterprise/ece-manage-integrations-server.md @@ -133,7 +132,6 @@ toc: - file: cloud/cloud/ec-access-kibana.md - file: cloud/cloud/ec-activity-page.md - file: cloud/cloud/ec-add-user-settings.md - - file: cloud/cloud/ec-api-deployment-other.md - file: cloud/cloud/ec-autoscaling.md - file: cloud/cloud/ec-billing-stop.md - file: cloud/cloud/ec-cloud-ingest-data.md @@ -169,7 +167,6 @@ toc: - file: cloud/cloud/ec-planning.md - file: cloud/cloud/ec-prepare-production.md - file: cloud/cloud/ec-regional-deployment-aliases.md - - file: cloud/cloud/ec-restart-deployment.md - file: cloud/cloud/ec-restore-across-clusters.md - file: cloud/cloud/ec-restoring-snapshots.md - file: cloud/cloud/ec-saas-metrics-accessing.md @@ -516,8 +513,6 @@ toc: - file: elasticsearch/elasticsearch-reference/shard-request-cache.md - file: elasticsearch/elasticsearch-reference/snapshot-restore.md - file: elasticsearch/elasticsearch-reference/snapshots-restore-snapshot.md - - file: elasticsearch/elasticsearch-reference/starting-elasticsearch.md - - file: elasticsearch/elasticsearch-reference/stopping-elasticsearch.md - file: elasticsearch/elasticsearch-reference/xpack-autoscaling.md - file: ingest-docs/fleet/index.md children: