|
| 1 | +--- |
| 2 | +title: "Azure Operator Nexus: Running bare metal actions directly with nexusctl" |
| 3 | +description: Learn how to bypass Azure and run bare metal actions directly in an emergency using nexusctl. |
| 4 | +author: DanCrank |
| 5 | +ms.author: danielcrank |
| 6 | +ms.service: azure-operator-nexus |
| 7 | +ms.topic: how-to |
| 8 | +ms.date: 08/26/2024 |
| 9 | +ms.custom: template-how-to, devx-track-azurecli |
| 10 | +--- |
| 11 | + |
| 12 | +# Run emergency bare metal actions outside of Azure using nexusctl |
| 13 | + |
| 14 | +This article describes the `nexusctl` utility, which can be used in break-glass (emergency) situations to |
| 15 | +run simple actions on bare metal machines without using the Azure console or command-line interface (CLI). |
| 16 | + |
| 17 | +> [!CAUTION] |
| 18 | +> Do not perform any action against management servers without first consulting with Microsoft support personnel. Doing so could affect the integrity of the Operator Nexus Cluster. |
| 19 | +
|
| 20 | +> [!IMPORTANT] |
| 21 | +> Disruptive command requests against a Kubernetes Control Plane (KCP) node are rejected if there is another disruptive action command already running against another KCP node or if the full KCP is not available. This check is done to maintain the integrity of the Nexus instance and ensure multiple KCP nodes don't become non-operational at once due to simultaneous disruptive actions. If multiple nodes become non-operational, it will break the healthy quorum threshold of the Kubernetes Control Plane. |
| 22 | +> |
| 23 | +> Powering off a KCP node is the only nexusctl action considered disruptive in the context of this check. |
| 24 | +
|
| 25 | +## Prerequisites |
| 26 | + |
| 27 | +- A [BareMetalMachineKeySet](./howto-baremetal-bmm-ssh.md) must be available to allow ssh access to the bare metal machines. The user must have superuser privilege level. |
| 28 | +- The platform Kubernetes must be up and running on site. |
| 29 | + |
| 30 | +## Overview |
| 31 | + |
| 32 | +`nexusctl` is a stand-alone program that can be run using `nc-toolbox` from an `ssh` session on any control-plane or management-plane node. Since `nexusctl` is contained in the `nc-toolbox-breakglass` container image and isn't installed directly on the host, it must be run with a command-line like: |
| 33 | + |
| 34 | +``` |
| 35 | +sudo nc-toolbox nc-toolbox-breakglass nexusctl <command> [subcommand] [options] |
| 36 | +``` |
| 37 | + |
| 38 | +(`nc-toolbox` must always be run as root or with `sudo`.) |
| 39 | + |
| 40 | +Like most other command-line programs, the `--help` option can be used with any command or subcommand to see more information: |
| 41 | + |
| 42 | +``` |
| 43 | +sudo nc-toolbox nc-toolbox-breakglass nexusctl --help |
| 44 | +sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal --help |
| 45 | +sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal power-off --help |
| 46 | +``` |
| 47 | + |
| 48 | +etc. |
| 49 | + |
| 50 | +> [!NOTE] |
| 51 | +> |
| 52 | +> > There is no bulk execution against multiple machines. Commands are executed on a machine by machine basis. |
| 53 | +
|
| 54 | +## Power off a bare metal machine |
| 55 | + |
| 56 | +A single bare metal machine can be powered off by connecting to a control-plane or management-plane node via ssh and running the command: |
| 57 | + |
| 58 | +``` |
| 59 | +sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal power-off --name <machine name> |
| 60 | +``` |
| 61 | + |
| 62 | +If the command is accepted, `nexusctl` responds with another command line that can be used to view the status of the long-running operation. Prefix this command with `sudo nc-toolbox nc-toolbox-breakglass`, as follows: |
| 63 | + |
| 64 | +``` |
| 65 | +sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal power-off --status --name <machine name> --operation-id <operation-id> |
| 66 | +``` |
| 67 | + |
| 68 | +The status is blank until the operation completes and reaches either a "succeeded" or "failed" state. While it's blank, assume that the operation is still in progress. |
| 69 | + |
| 70 | +## Start a bare metal machine |
| 71 | + |
| 72 | +A single bare metal machine can be started by connecting to a control-plane or management-plane node via ssh and running the command: |
| 73 | + |
| 74 | +``` |
| 75 | +sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal start --name <machine name> |
| 76 | +``` |
| 77 | + |
| 78 | +If the command is accepted, `nexusctl` responds with another command line that can be used to view the status of the long-running operation. Prefix this command with `sudo nc-toolbox nc-toolbox-breakglass`, as follows: |
| 79 | + |
| 80 | +``` |
| 81 | +sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal start --status --name <machine name> --operation-id <operation-id> |
| 82 | +``` |
| 83 | + |
| 84 | +The status is blank until the operation completes and reaches either a "succeeded" or "failed" state. While it's blank, assume that the operation is still in progress. |
| 85 | + |
| 86 | +## Unmanage a bare metal machine (set to unmanaged state) |
| 87 | + |
| 88 | +A single bare metal machine can be switched to an unmanaged state by connecting to a control-plane or management-plane node via ssh and running the command: |
| 89 | + |
| 90 | +``` |
| 91 | +sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal unmanage --name <machine name> |
| 92 | +``` |
| 93 | + |
| 94 | +While in an unmanaged state, no actions are permitted for that machine, except for returning it to a managed state (see next section). This function can be used to keep a bare metal machine powered off if it's in a rebooting crash loop. |
| 95 | + |
| 96 | +`unmanage` isn't a long-running command, so there's no associated command to check operation status. |
| 97 | + |
| 98 | +## Manage a bare metal machine (set to managed state) |
| 99 | + |
| 100 | +A single bare metal machine can be switched to a managed state by connecting to a control-plane or management-plane node via ssh and running the command: |
| 101 | + |
| 102 | +``` |
| 103 | +sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal manage --name <machine name> |
| 104 | +``` |
| 105 | + |
| 106 | +`manage` isn't a long-running command, so there's no associated command to check operation status. |
0 commit comments