documentation for nexusctl, first pass

Author: Dan Crank

articles/operator-nexus/TOC.yml

@@ -29,7 +29,7 @@
         - name: Network Fabric read-only commands
           href: concepts-network-fabric-read-only-commands.md
         - name: Network Fabric read write commands
-          href: concepts-network-fabric-read-write-commands.md   
+          href: concepts-network-fabric-read-write-commands.md
         - name: Disable Border Gateway Protocol neighbors
           href: concepts-disable-border-gateway-protocol-neighbors.md
     - name: Isolation Domains
@@ -46,14 +46,14 @@
     - name: Nexus Kubernetes
       expanded: false
       items:
-      - name: Overview
-        href: concepts-nexus-kubernetes-cluster.md
-      - name: Resource Placement
-        href: concepts-nexus-kubernetes-placement.md
-      - name: Networking
-        href: concepts-nexus-networking.md
-      - name: Workload Network Types
-        href: concepts-nexus-workload-network-types.md
+        - name: Overview
+          href: concepts-nexus-kubernetes-cluster.md
+        - name: Resource Placement
+          href: concepts-nexus-kubernetes-placement.md
+        - name: Networking
+          href: concepts-nexus-networking.md
+        - name: Workload Network Types
+          href: concepts-nexus-workload-network-types.md
     - name: Observability
       expanded: false
       items:
@@ -290,6 +290,8 @@
           href: howto-baremetal-run-read.md
         - name: BareMetal Run-Data-Extract Execution
           href: howto-baremetal-run-data-extract.md
+        - name: Running BareMetal actions directly with nexusctl
+          href: howto-baremetal-nexusctl.md
     - name: Troubleshoot Control Plane Quorum
       href: troubleshoot-control-plane-quorum.md
     - name: Troubleshoot Bare Metal Machine Provisioning
@@ -358,4 +360,3 @@
       items:
         - name: 2404.2
           href: release-notes-2404.2.md
-        

articles/operator-nexus/howto-baremetal-nexusctl.md

@@ -0,0 +1,149 @@
+---
+title: "Azure Operator Nexus: Running bare metal actions directly with nexusctl"
+description: Learn how to bypass Azure and run bare metal actions directly in an emergency using nexusctl.
+author: DanCrank
+ms.author: danielcrank
+ms.service: azure-operator-nexus
+ms.topic: how-to
+ms.date: 08/26/2024
+ms.custom: template-how-to, devx-track-azurecli
+---
+
+# Run emergency bare metal actions outside of Azure using nexusctl
+
+This article describes the `nexusctl` utility, which can be used in break-glass (emergency) situations to
+run simple actions on bare metal machines without using the Azure console or command-line interface (CLI).
+
+> [!CAUTION]
+> Do not perform any action against management servers without first consulting with Microsoft support personnel. Doing so could affect the integrity of the Operator Nexus Cluster.
+
+> [!IMPORTANT]
+> Disruptive command requests against a Kubernetes Control Plane (KCP) node are rejected if there is another disruptive action command already running against another KCP node or if the full KCP is not available. This check is done to maintain the integrity of the Nexus instance and ensure multiple KCP nodes don't go down at once due to simultaneous disruptive actions. If multiple nodes go down, it will break the healthy quorum threshold of the Kubernetes Control Plane.
+>
+> Powering off a KCP node is the only nexusctl action considered disruptive in the context of this check.
+
+## Prerequisites
+
+1. Create a [BareMetalMachineKeySet](./howto-baremetal-bmm-ssh.md) to allow ssh access to the bare metal machines.
+
+## Overview
+
+`nexusctl` is a stand-alone program that can be run using `nc-toolbox` from an `ssh` session on any management node. Since `nexusctl` is contained in the `nc-toolbox-breakglass` container image and isn't installed directly on the host, it must be run with a command-line like:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl <command> [subcommand] [options]
+```
+
+(`nc-toolbox` must always be run as root or with `sudo`.)
+
+Like most other command-line programs, the `--help` option can be used with any command or subcommand to see more information:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl --help
+sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal --help
+sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal power-off --help
+```
+
+etc.
+
+## Power off a bare metal machine
+
+A single bare metal machine can be powered off by connecting to a management node via ssh and running the command:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal power-off --name <machine name>
+```
+
+If the command is accepted, `nexusctl` responds with another command line that can be used to view the status of the long-running operation. Prefix this command with `sudo nc-toolbox nc-toolbox-breakglass`, as follows:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal power-off --status --name <machine name> --operation-id <operation-id>
+```
+
+The status is blank until the operation completes and reaches either a "succeeded" or "failed" state. While it's blank, assume that the operation is still in progress.
+
+## Start a bare metal machine
+
+A single bare metal machine can be started from a power-off state by connecting to a management node via ssh and running the command:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal start --name <machine name>
+```
+
+If the command is accepted, `nexusctl` responds with another command line that can be used to view the status of the long-running operation. Prefix this command with `sudo nc-toolbox nc-toolbox-breakglass`, as follows:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal start --status --name <machine name> --operation-id <operation-id>
+```
+
+The status is blank until the operation completes and reaches either a "succeeded" or "failed" state. While it's blank, assume that the operation is still in progress.
+
+## Unmanage a bare metal machine (set to unmanaged state)
+
+A single bare metal machine can be moved from a managed state to an unmanaged state by connecting to a management node via ssh and running the command:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal unmanage --name <machine name>
+```
+
+While in an unmanaged state, no actions are permitted for that machine, except for returning it to a managed state (see next section).
+
+`unmanage` isn't a long-running command, so there's no associated command to check operation status.
+
+## Manage a bare metal machine (set to managed state)
+
+A single bare metal machine can be moved from an unmanaged state to a managed state by connecting to a management node via ssh and running the command:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl baremetal manage --name <machine name>
+```
+
+`manage` isn't a long-running command, so there's no associated command to check operation status.
+
+## Create users on storage appliances
+
+User accounts can be created on the Pure storage appliance by connecting to a management node via ssh and running the command:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl storage users create --file <user-file> --keyvault <keyvault>
+```
+
+`user-file` is the name of a file containing user names and roles, one per line, with a comma between the user name and the role. Allowed roles are `readonly`, `storage_admin`, and `array_admin`. For example:
+
+```
+userA,readonly
+userB,storage_admin
+userC,array_admin
+```
+
+`keyvault` is the name of the keyvault in which the user passwords are stored. `nexusctl` generates these passwords.
+
+If a user in the given list already exists on the appliance, their account and password isn't changed.
+
+## Delete users on storage appliances
+
+User accounts can be deleted on the Pure storage appliance by connecting to a management node via ssh and running the command:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl storage users delete --file <user-file> --keyvault <keyvault>
+```
+
+`user-file` is the name of a file containing user names and roles, as described in the previous section. For each user in the list, if it exists on the appliance with the role specified, the user is deleted and their password removed from the keyvault. If the user doesn't exist on the appliance, or exists with a different role, they aren't deleted.
+
+## List users on storage appliances
+
+The current user accounts on the storage appliance can be listed by connecting to a management node via ssh and running the command:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl storage users get
+```
+
+## Rotate passwords for users on storage appliances
+
+Passwords can be rotated for users on the Pure storage appliance by connecting to a management node via ssh and running the command:
+
+```
+sudo nc-toolbox nc-toolbox-breakglass nexusctl storage users rotate --file <user-file> --keyvault <keyvault>
+```
+
+`user-file` is the name of a file containing user names and roles, as described in the previous section. For each user in the list, if it exists on the appliance with the role specified, a new password is generated for the user and stored in the keyvault. If the user doesn't exist on the appliance, or exists with a different role, their password isn't changed.