Skip to content

HOLD FOR RELEASE: use cli to get commands for node joins #3216

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
May 7, 2025
Merged

HOLD FOR RELEASE: use cli to get commands for node joins #3216

Changes from 3 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
fbf420a
automate node joins
paigecalvert Apr 28, 2025
be868a2
edits
paigecalvert Apr 28, 2025
b5ad9c9
edits
paigecalvert Apr 29, 2025
282dd0c
edits
paigecalvert Apr 29, 2025
09dad27
rm optional ha step for simplicity
paigecalvert Apr 29, 2025
1a345d0
edit description of automating node joins
paigecalvert Apr 29, 2025
97e73b0
edits for updated join experience
paigecalvert Apr 29, 2025
35c7a40
edits for updated join experience
paigecalvert Apr 29, 2025
d60e688
edits
paigecalvert Apr 29, 2025
62b0af5
edits
paigecalvert Apr 29, 2025
44cf317
edits
paigecalvert Apr 29, 2025
656d8d0
edits
paigecalvert Apr 29, 2025
88d096c
language edits for consistency
paigecalvert Apr 30, 2025
97cf782
Update docs/enterprise/embedded-manage-nodes.mdx
paigecalvert Apr 30, 2025
f29fd7f
change command name back
paigecalvert Apr 30, 2025
e569616
combine online and air gap steps for adding ndoes
paigecalvert Apr 30, 2025
637d986
remove alpha and beta warnings
paigecalvert May 6, 2025
909dc7e
remove alpha beta limitations
paigecalvert May 6, 2025
8a9e64b
Merge branch 'main' into 122803
paigecalvert May 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 49 additions & 19 deletions docs/enterprise/embedded-manage-nodes.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,23 +18,19 @@ Multi-node clusters with Embedded Cluster have the following limitations:

* Setting node roles with the Embedded Cluster Config [roles](/reference/embedded-config#roles) key is Beta.

## Add Nodes to a Cluster (Beta) {#add-nodes}

You can add nodes to create a multi-node cluster in online (internet-connected) and air-gapped (limited or no outbound internet access) environments. The Admin Console provides the join command that you use to join nodes to the cluster.
* The `join print-command` command always returns the command for joining a node with the controller role. It does not support printing the join command for any custom node roles defined in the Embedded Cluster Config `roles` key. See [Automate Controller Node Joins](#automate-node-joins) below.
Copy link
Contributor Author

@paigecalvert paigecalvert Apr 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ added to Limitations section


:::note
Multi-node clusters are not highly available by default. For information about enabling high availability, see [Enable High Availability for Multi-Node Clusters (Alpha)](#ha) below.
:::
## Add Nodes to a Cluster (Beta) {#add-nodes}

To add nodes to a cluster:
To add a node to a cluster with Embedded Cluster:

1. (Optional) In the Embedded Cluster Config, configure the `roles` key to customize node roles. For more information, see [roles](/reference/embedded-config#roles) in _Embedded Cluster Config_. When you are done, create and promote a new release with the updated Config.

1. Do one of the following to get the join command from the Admin Console:
1. Do one of the following:

1. To add nodes during the application installation process, follow the steps in [Online Installation with Embedded Cluster](/enterprise/installing-embedded) or [Air Gap Installation with Embedded Cluster](/enterprise/installing-embedded-air-gap) to install. A **Nodes** screen is displayed as part of the installation flow in the Admin Console that allows you to choose a node role and copy the relevant join command.
1. Follow the steps in [Online Installation with Embedded Cluster](/enterprise/installing-embedded) or [Air Gap Installation with Embedded Cluster](/enterprise/installing-embedded-air-gap) to install. A **Nodes** screen is displayed as part of the installation flow in the Admin Console that allows you to choose a node role and copy the relevant join command.

1. Otherwise, if you have already installed the application:
1. Otherwise, to add a node to an existing cluster:

1. Log in to the Admin Console.

Expand All @@ -44,27 +40,29 @@ To add nodes to a cluster:

<img alt="Add node page in the Admin Console" src="/images/admin-console-add-node.png" width="600px"/>
Copy link
Contributor Author

@paigecalvert paigecalvert Apr 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ removed out of date screenshot of admin console


[View a larger version of this image](/images/admin-console-add-node.png)
[View a larger version of this image](/images/admin-console-add-node.png)

1. Either on the Admin Console **Nodes** screen that is displayed during installation or in the **Add a Node** dialog, select one or more roles for the new node that you will join. Copy the join command.
1. Select one or more roles for the node. Copy the join command.

Note the following:

* If the Embedded Cluster Config [roles](/reference/embedded-config#roles) key is not configured, all new nodes joined to the cluster are assigned the `controller` role by default. The `controller` role designates nodes that run the Kubernetes control plane. Controller nodes can also run other workloads, such as application or Replicated KOTS workloads.

* The role cannot be changed after a node is added. If you need to change a node’s role, reset the node and add it again with the new role.

1. Do one of the following to make the Embedded Cluster installation assets available on the machine that you will join to the cluster:
1. Do one of the following to make the Embedded Cluster installation assets available on the node that you will join to the cluster:

* **For online (internet-connected) installations**: SSH onto the machine that you will join. Then, use the same commands that you ran during installation to download and untar the Embedded Cluster installation assets on the machine. See [Online Installation with Embedded Cluster](/enterprise/installing-embedded).
* **For online (internet-connected) installations**: SSH onto the node that you will join. Then, use the same commands that you ran during installation to download and untar the Embedded Cluster installation assets on the node. See [Online Installation with Embedded Cluster](/enterprise/installing-embedded).

* **For air gap installations with limited or no outbound internet access**: On a machine that has internet access, download the Embedded Cluster installation assets (including the air gap bundle) using the same command that you ran during installation. See [Air Gap Installation with Embedded Cluster](/enterprise/installing-embedded-air-gap). Then, move the downloaded assets to the air-gapped machine that you will join, and untar.
* **For air gap installations with limited or no outbound internet access**: On a node that has internet access, download the Embedded Cluster installation assets (including the air gap bundle) using the same command that you ran during installation. See [Air Gap Installation with Embedded Cluster](/enterprise/installing-embedded-air-gap). Then, move the downloaded assets to the air-gapped node that you will join, and untar.

:::important
The Embedded Cluster installation assets on each node must all be the same version. If you use a different version than what is installed elsewhere in the cluster, the cluster will not be stable. To download a specific version of the Embedded Cluster assets, select a version in the **Embedded cluster install instructions** dialog.
:::

1. On the machine that you will join to the cluster, run the join command that you copied from the Admin Console.
1. SSH onto the node that you will join to the cluster.

1. Run the join command that you copied.

**Example:**

Expand All @@ -77,9 +75,34 @@ To add nodes to a cluster:
sudo ./APP_SLUG join --airgap-bundle APP_SLUG.airgap 10.128.0.32:30000 TxXboDstBAamXaPdleSK7Lid
```

1. (Optional) If you are adding a third controller node to the cluster, respond to the prompt asking if you want to enable high availability. For more information, see [Create a Multi-Node Cluster with High Availability](#create-ha) below.

1. In the Admin Console, either on the installation **Nodes** screen or on the **Cluster Management** page, verify that the node appears. Wait for the node's status to change to Ready.

1. Repeat these steps for each node you want to add.
1. Repeat these steps for each node you want to add.

## Automate Controller Node Joins (Beta) {#automate-node-joins}
Copy link
Contributor Author

@paigecalvert paigecalvert Apr 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ added new section on automating node joins that explains how to get the join command for controllers from the command line.

Note: I decided to specify "controller nodes" in the heading etc to make it clear that the print command is just for controllers. We can update this to remove the "controller" callouts if/when that changes

Copy link
Contributor Author

@paigecalvert paigecalvert Apr 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also , it seemed like "Automate Node Joins" as the heading would speak to people who are wanting to write automating tests, but lmk if you have other ideas for how this info might be titled


With Embedded Cluster, you can use the command line to automate the process of joining controller nodes to a cluster. This is especially useful when testing multi-node Embedded Cluster installations where you need to programmatically add nodes.

To automate controller node joins with Embedded Cluster:

1. On a node that is already joined to the cluster, run the following command to get the command for adding a controller node:

```bash
sudo ./APP_SLUG join print-command
```
Where `APP_SLUG` is the unique application slug.

:::note
The `join print-command` command always returns the command for joining a node with the controller role. It does not support printing the join command for any custom node roles defined in the Embedded Cluster Config `roles` key.
:::

1. Copy the join command.

1. SSH onto the node that you want to join as a controller.

1. Run the join command that you copied.

## High Availability for Multi-Node Clusters (Alpha) {#ha}

Expand Down Expand Up @@ -125,9 +148,16 @@ To create a multi-node HA cluster:

1. Set up a cluster with at least two controller nodes. You can do an online (internet-connected) or air gap installation. For more information, see [Online Installation with Embedded Cluster](/enterprise/installing-embedded) or [Air Gap Installation with Embedded Cluster](/enterprise/installing-embedded-air-gap).

1. SSH onto a third node that you want to join to the cluster as a controller.
1. Get the command for joining the third controller node either in the Admin Console **Cluster Management** tab or by running the following command on an existing node:

```bash
sudo ./APP_SLUG join print-command
```
Where `APP_SLUG` is the unique application slug.

1. SSH onto the node that you want to add as a third controller.
Copy link
Contributor Author

@paigecalvert paigecalvert Apr 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ I also added mention of the print command to the HA procedure since this procedure does assume the user is joining a third controller node


1. On the third node, run the join command provided in the Admin Console **Cluster Management** tab.
1. On the node, run the join command that you copied.

**Example:**

Expand Down