Merge branch 'main' into 536-eks-cluster-encryption

Author: joneszc

docs/docs/explanations/advanced-provider-configuration.md

@@ -112,31 +112,25 @@ Nebari supports setting an existing KMS key while deploying Nebari to implement
 created in Nebari's EKS cluster. The KMS key must be a **Symmetric** key set to **encrypt and decrypt** data.
 
 :::warning
-
 Enabling EKS cluster secrets encryption, by setting `amazon_web_services.eks_kms_arn`, is an
 _irreversible_ action and re-deploying Nebari to try to remove a previously set `eks_kms_arn` will fail.
 On the other hand, if you try to change the KMS key in use for cluster encryption, by re-deploying Nebari
 after setting a _different_ key ARN, the re-deploy should succeed but the KMS key used for encryption will
 not actually change in the cluster config and the original key will remain set. The integrity of a faulty
 deployment can be restored, following a failed re-deploy attempt to remove a previously set KMS key, by
 simply re-deploying Nebari while ensuring `eks_kms_arn` is set to the original KMS key ARN.
-
 :::
 
 :::danger
-
 If the KMS key used for envelope encryption of secrets is ever deleted, then there is no way to recover
 the EKS cluster.
-
 :::
 
 :::note
-
 After enabling cluster encryption on your cluster, you must encrypt all existing secrets with the
 new key by running the following command:
 `kubectl get secrets --all-namespaces -o json | kubectl annotate --overwrite -f - kms-encryption-timestamp="time value"`
 Consult [Encrypt K8s secrets with AWS KMS on existing clusters](https://docs.aws.amazon.com/eks/latest/userguide/enable-kms.html) for more information.
-
 :::
 
 Here is an example of how you would set KMS key ARN in `nebari-config.yaml`.
@@ -147,6 +141,63 @@ amazon_web_services:
   eks_kms_arn: 'arn:aws:kms:us-west-2:01234567890:key/<aws-kms-key-id>'
 ```
 
+### Launch Templates (Optional)
+
+Nebari supports configuring launch templates for your node groups, enabling you to customize settings like the AMI ID and pre-bootstrap commands. This is particularly useful if you need to use a custom AMI or perform specific actions before the node joins the cluster.
+
+:::warning
+If you add a `launch_template` to an existing node group that was previously created without one, AWS will treat this as a change requiring the replacement of the entire node group. This action will trigger a reallocation of resources, effectively destroying the current node group and recreating it. This behavior is due to how AWS handles self-managed node groups versus those using launch templates with custom settings.
+:::
+
+:::tip
+To avoid unexpected downtime or data loss, consider creating a new node group with the launch template settings and migrating your workloads accordingly. This approach allows you to implement the new configuration without disrupting your existing resources.
+:::
+
+#### Configuring a Launch Template
+
+To configure a launch template for a node group in your `nebari-config.yaml`, add the `launch_template` section under the desired node group:
+
+```yaml
+amazon_web_services:
+  region: us-west-2
+  kubernetes_version: "1.18"
+  node_groups:
+    custom-node-group:
+      instance: "m5.large"
+      min_nodes: 1
+      max_nodes: 5
+      gpu: false  # Set to true if using GPU instances
+      launch_template:
+        # Replace with your custom AMI ID
+        ami_id: ami-0abcdef1234567890
+        # Command to run before the node joins the cluster
+        pre_bootstrap_command: |
+          #!/bin/bash
+          # This script is executed before the node is bootstrapped
+          # You can use this script to install additional packages or configure the node
+          # For example, to install the `htop` package, you can run:
+          # sudo apt-get update
+          # sudo apt-get install -y htop"
+```
+
+**Parameters:**
+
+- `ami_id` (Optional): The ID of the custom AMI to use for the nodes in this group; this assumes the AMI provided is an EKS-optimized AMI derivative. If specified, the `ami_type` is automatically set to `CUSTOM`.
+- `pre_bootstrap_command` (Optional): A command or script to execute on the node before
+  it joins the Kubernetes cluster. This can be used for custom setup or configuration
+  tasks. The format should be a single string in conformation with the shell syntax.
+  This command is injected in the `user_data` field of the launch template. For more
+  information, see [User Data](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html).
+
+> If you're using a `launch_template` with a custom `ami_id`, there's an issue with updating the `scaling.desired_size` via Nebari configuration (terraform). To scale up, you must recreate the node group or adjust the scaling settings directly in the AWS Console UI (recommended). We are aware of this inconsistency and plan to address it in a future update.
+
+:::note
+If an `ami_id` is not provided, AWS will use the default Amazon Linux 2 AMI for the
+specified instance type. You can find the latest optimized AMI IDs for Amazon EKS in your
+cluster region by inspecting its respective SSM parameters. For more information, see
+[Retrieve recommended Amazon Linux AMI IDs](https://docs.aws.amazon.com/eks/latest/userguide/retrieve-ami-id.html).
+:::
+
 </TabItem>
 
 <TabItem value="azure" label="Azure">