diff --git a/_partials/pcg/_pcg-initial-installation.mdx b/_partials/pcg/_pcg-initial-installation.mdx index 8f7b124ef75..1f009368a3c 100644 --- a/_partials/pcg/_pcg-initial-installation.mdx +++ b/_partials/pcg/_pcg-initial-installation.mdx @@ -54,12 +54,6 @@ partial_name: pcg-initial-installation 5. If you want to configure your PCG to use a proxy network, complete the following fields, as appropriate. - :::info - - By default, proxy environment variables (`HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY`) configured during PCG installation are propagated to all PCG cluster nodes, as well as the nodes of all tenant workload clusters deployed with the PCG. However, proxy CA certificates are only propagated to PCG cluster nodes; they are not propagated the nodes of tenant workload clusters. - - ::: - | **Parameter** | **Description** | | :-------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **HTTPS Proxy** | (Optional) Leave blank unless using an HTTPS proxy. This setting propagates to all PCG nodes and all workload cluster nodes deployed using the PCG. Example: `https://:@:`. | @@ -67,75 +61,13 @@ partial_name: pcg-initial-installation | **No Proxy** | (Optional) If using an HTTP or HTTPS proxy, provide a comma-separated list of Classless Inter-Domain Routing (CIDR) addresses, hostnames, and domain names that should bypass the proxy. This setting propagates to all PCG nodes to bypass the proxy server, as well as all workload cluster nodes deployed using the PCG. Example if you have a self-hosted environment: `my.company.com,10.10.0.0/16`. | | **Proxy CA Certificate Filepath** | (Optional) If using an HTTP or HTTPS proxy, provide the filepath on the installer host to the proxy’s CA certificate file. This certificate is copied to each PCG node during PCG deployment and added to the node trust store. The CA certificate file _must_ be named `ca.crt`. Example: `/usr/local/share/ca-certificates/ca.crt`.

**NOTE**: Proxy CA certificates are _not_ automatically propagated to workload clusters using the PCG; these certificates must be added at either the tenant level or cluster profile OS layer. -
- - Configure Proxy CA Certificate for Workload Clusters - - If you are configuring proxy CA certificates for your PCG, they must also be added to workload clusters at the tenant level or cluster profile level in the OS layer. - - - If configured at the tenant level, _all_ workload clusters provisioned from the tenant, with the exception of managed Kubernetes clusters (EKS, AKS, and GKE) and Edge clusters, will have the CA certificate injected into their cluster nodes. - - - If configured at the cluster profile level, only workload clusters deployed using the cluster profile will be injected with the CA certificate. -
- To configure your proxy CA certificate for your workload clusters, use one of the following methods. - - - - - - Take the following approach to propagate your proxy server CA certificate to _all_ workload cluster nodes provisioned from the tenant, with the exception of managed Kubernetes clusters (EKS, AKS, and GKE) and Edge clusters. - - 1. Log in to [Palette](https://console.spectrocloud.com) as a tenant admin. - - 2. From the left main menu, select **Tenant Settings**. - - 3. From the **Tenant Settings Menu**, below **Platform**, select **Certificates**. - - 4. Select **Add A New Certificate**. - - 5. In the **Add Certificate** dialog, enter the **Certificate Name** and **Certificate** value. - - 6. **Confirm** your changes. - - - - - - Take the following approach to propagate proxy server CA certificates on a per-cluster basis. - - 1. Log in to [Palette](https://console.spectrocloud.com). - - 2. From the left main menu, select **Profiles**. - - 3. Choose an existing cluster profile or . For more information, refer to the guide. - - 4. In the OS layer of your cluster profile, add your CA certificate to the `content` field under `kubeadmconfig.files`. The CA certificate file _must_ be named `ca.crt`. - ```yaml hideClipboard title="Example OS layer configuration" {1,10,13-16} - kubeadmconfig: - preKubeadmCommands: - - echo "Executing pre kube admin config commands" - - update-ca-certificates - - 'systemctl restart containerd; sleep 3' - - 'while [ ! -S /var/run/containerd/containerd.sock ]; do echo "Waiting for containerd..."; sleep 1; done' - postKubeadmCommands: - - echo "Executing post kube admin config commands" - files: - - targetPath: /usr/local/share/ca-certificates/ca.crt - targetOwner: "root:root" - targetPermissions: "0644" - content: | - -----BEGIN CERTIFICATE----- - *************************** - -----END CERTIFICATE----- - ``` - - 5. Select **Confirm Updates**. - - - - - -
+ :::info + + By default, proxy environment variables (`HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY`) configured during PCG installation are propagated to all PCG cluster nodes, as well as the nodes of all workload clusters deployed with the PCG. However, proxy CA certificates are only propagated to PCG cluster nodes; they are not propagated workload cluster nodes. You must configure proxy CA certificates separately at either the tenant level or the cluster profile OS layer. For detailed instructions, including how to mount CA certificates into pods, refer to [Configure Proxy CA Certificates for Workload Clusters](/clusters/pcg/manage-pcg/configure-proxy-ca-certs). + + For more information on how PCG environment variables are injected into nodes, refer to our PCG [Architecture](/clusters/pcg/architecture/) page. + + ::: 6. Enter the following network details. diff --git a/docs/docs-content/architecture/palette-namespaces-pods.md b/docs/docs-content/architecture/palette-namespaces-pods.md index ddf849e7b30..3c1e924b1ce 100644 --- a/docs/docs-content/architecture/palette-namespaces-pods.md +++ b/docs/docs-content/architecture/palette-namespaces-pods.md @@ -45,7 +45,7 @@ The following table gives the namespace to pod mapping for Palette Tenant Cluste | | kube-scheduler-vmdynamictest-cp-< random-hash > | | | | kube-vip-vmdynamictest-cp-< random-hash > | | | palette-system | palette-webhook-< random-hash > | Palette webhook that validates and defaults Palette custom resources. May be present depending on the installation type. | -| reach-system | reach-controller-manager-< random-hash > | Reach controller manager responsible for communication between Palette and workload clusters. | +| reach-system | reach-controller-manager-< random-hash > | Reach controller manager that operates as a mutating admission webhook, injecting proxy settings, environment variables, volumes, and volume mounts into matching pods through ClusterPodPreset and PodPreset resources. | | spectro-task-< random-hash > | crony-< random-hash > | Component responsible for OS upgrades and patching on cluster nodes. | ## Palette PCG NameSpaces with Pods @@ -85,8 +85,8 @@ The following table gives the namespace to pod mapping for Palette vSphere Gatew | | vsphere-csi-controller-< random-hash > | | | | vsphere-csi-node-< random-hash > | | | palette-system | palette-webhook-< random-hash > | Palette webhook that validates and defaults Palette custom resources. May be present depending on the installation type. | -| reach-system | reach-controller-manager-< random-hash > | Reach controller manager responsible for communication between Palette and workload clusters. | -| spectro-task-< random-hash > | crony-< random-hash > | Component responsible for OS upgrades and patching on cluster nodes. | +| reach-system | reach-controller-manager-< random-hash > | Reach controller manager that operates as a mutating admission webhook, injecting proxy settings, environment variables, volumes, and volume mounts into matching pods through ClusterPodPreset and PodPreset resources. | +| spectro-task-< random-hash > | crony-< random-hash > | Component responsible for OS upgrades and patching on cluster nodes. | ## Enterprise NameSpaces with Pods diff --git a/docs/docs-content/clusters/pcg/architecture.md b/docs/docs-content/clusters/pcg/architecture.md index b1682c5eb40..6963fd883d2 100644 --- a/docs/docs-content/clusters/pcg/architecture.md +++ b/docs/docs-content/clusters/pcg/architecture.md @@ -10,8 +10,8 @@ tags: ["pcg"] A Private Cloud Gateway (PCG) facilitates communication between Palette and your infrastructure environment. The PCG is necessary in environments where Palette does not have direct network access. Many infrastructure environments are placed -in a private network that blocks connections originating externally. The PCG connects to Palette, and acts as an -endpoint, allowing you to target the environment when deploying clusters in Palette. +in a private network that blocks connections that originate from external sources. The PCG connects to Palette and acts +as an endpoint, allowing you to target the environment when deploying clusters in Palette. When installed, the PCG registers with the self-hosted or SaaS Palette instance you specify, enabling secure communication between the Palette management plane and the private cloud environment. The PCG enables the deployment and @@ -62,28 +62,51 @@ proxy server details are saved as environment variables (`HTTPS_PROXY`, `HTTP_PR to all PCG cluster nodes, as well as the nodes of all workload clusters deployed with the PCG. The provided proxy servers are then used by the PCG and workload clusters to access the internet. -You can also provide Certificate Authority (CA) certificates for the proxy server during installation. However, proxy CA -certificates are only propagated to each PCG cluster node; they are not propagated to the nodes of tenant clusters. +You can also provide Certificate Authority (CA) certificates for the proxy server during installation. The certificate +file must be named `ca.crt` and be in PEM format. Proxy CA certificates are only propagated to PCG cluster nodes; they +are not propagated to workload cluster nodes. Proxy CA certificates must be added to workload clusters at either the tenant level or the cluster profile level in the -OS layer. +OS layer. To make the CA certificate available to pods inside workload clusters, you can use the `podMount` +configuration in the OS layer of your cluster profile. Refer to +[Configure Proxy CA Certificates for Workload Clusters](./manage-pcg/configure-proxy-ca-certs.md) for instructions on +configuring both node-level and pod-level CA certificate trust. -- If configured at the tenant level, _all_ workload clusters provisioned from the tenant, with the exception of managed - Kubernetes clusters (EKS, AKS, and GKE) and Edge clusters, will have the CA certificate injected into their cluster - nodes. - -- If configured at the cluster profile level, only workload clusters deployed using the cluster profile will be injected - with the CA certificate. - -For guidance on configuring proxy CA certificates, refer to appropriate Palette CLI PCG deployment guide for -[MAAS](./deploy-pcg/maas.md), [OpenStack](./deploy-pcg/openstack.md), [VMware vSphere](./deploy-pcg/vmware.md), or +For guidance on configuring proxy CA certificates during PCG installation, refer to the appropriate Palette CLI PCG +deployment guide for [MAAS](./deploy-pcg/maas.md), [VMware vSphere](./deploy-pcg/vmware.md), or [Apache CloudStack](./deploy-pcg/cloudstack.md). #### Existing Kubernetes Cluster A PCG installed onto an existing Kubernetes cluster will inherit the proxy server configuration from the underlying -Kubernetes cluster. Contact your Kubernetes administrator for the proxy server details and guidance on configuring the -underlying Kubernetes cluster to use a proxy server if needed. +Kubernetes cluster. To configure a proxy on a PCG installed on an existing Kubernetes cluster using the +[Reach system](#reach-system), refer to our [Deploy a PCG to an Existing Kubernetes Cluster](./deploy-pcg-k8s.md) guide. + +### Reach System + +The Reach system is a Kubernetes +[mutating admission webhook](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) +that automatically injects configuration into pods at creation time. It is deployed in the `reach-system` namespace on +every Palette CLI-provisioned PCG and can be manually installed on PCGs deployed to an existing Kubernetes cluster. When +deployed, the Reach system creates the following resources: + +- **Two Custom Resource Definitions (CRDs)** - _ClusterPodPreset_ (cluster-scoped) and _PodPreset_ (namespace-scoped). + These resources define the environment variables, volumes, and volume mounts to inject into matching pods. + +- **MutatingWebhookConfiguration** - Intercepts pod creation requests and applies the preset configurations. + +When Palette configures the Reach system with proxy settings, it creates ClusterPodPreset resources that inject the +`HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` environment variables into all matching pods across the cluster. +Namespace-scoped PodPreset resources are also created to append namespace-specific entries to the `NO_PROXY` list, so +that in-cluster service discovery traffic bypasses the proxy. + +The Reach system also supports injecting volumes and volume mounts into pods. This capability is used to mount +Certificate Authority (CA) certificates into pods so they can trust the proxy server. For more information about +configuring proxy CA certificates for workload clusters, refer to +[Configure Proxy CA Certificates for Workload Clusters](./manage-pcg/configure-proxy-ca-certs.md). + +For PCGs deployed to an existing Kubernetes cluster, you must manually install the Reach system using a Helm chart. +Refer to [Enable and Manage Proxy Configurations](./manage-pcg/configure-proxy.md) for instructions. ## Cluster Lifecycle Support diff --git a/docs/docs-content/clusters/pcg/deploy-pcg-k8s.md b/docs/docs-content/clusters/pcg/deploy-pcg-k8s.md index be684f76ae1..ad846294476 100644 --- a/docs/docs-content/clusters/pcg/deploy-pcg-k8s.md +++ b/docs/docs-content/clusters/pcg/deploy-pcg-k8s.md @@ -8,18 +8,26 @@ tags: ["pcg"] --- For infrastructure environments that the Palette CLI does not support, you can deploy a Private Cloud Gateway (PCG) to -an existing Kubernetes cluster. This guide provides the steps to deploy a PCG to an existing Kubernetes cluster. +an existing Kubernetes cluster. ## Supported Environments You can deploy a PCG onto an existing Kubernetes cluster that is not managed by Palette to the following environments: - AWS + - Azure -- Nutanix. Requires the registration of a Nutanix cloud. Refer to the - [Register Nutanix Cloud](../data-center/nutanix/register-nutanix-cloud.md) section for more information. -### PCG Sizing +- Nutanix + + :::info + + To select **Nutanix** as the PCG cloud type, you must + [Register Nutanix Cloud](../data-center/nutanix/register-nutanix-cloud.md) with Palette. + + ::: + +## PCG Sizing The following table provides the recommended sizing for the PCG based on the number of nodes, CPU, memory, storage, and the maximum concurrent cluster deployments. You can continue to deploy additional clusters once the current clusters @@ -28,7 +36,7 @@ deployment batch is complete. We recommend using a minimum of 3 nodes for production environments. Single node clusters are better suited for development and testing environments. -##### Single-Node Cluster +### Single-Node Cluster | **Size** | **Nodes** | **CPU** | **Memory** | **Storage** | **Maximum concurrent cluster deployments** | | -------- | --------- | ------- | ---------- | ----------- | ------------------------------------------ | @@ -36,7 +44,7 @@ development and testing environments. | Medium | 1 | 8 | 8 GB | 100 GB | 4-6 | | Large | 1 | 16 | 16 GB | 120 GB | 7-10 | -##### High-Availability (HA) Cluster +### High-Availability (HA) Cluster | **Size** | **Nodes** | **CPU** | **Memory** | **Storage** | **Maximum concurrent cluster deployments** | | -------- | --------- | ------- | ---------- | ----------- | ------------------------------------------ | @@ -58,7 +66,7 @@ development and testing environments. ::: -- You need sufficient permissions to deploy the PCG services in the cluster. The +- Sufficient permissions to deploy the PCG services in the cluster. The [default Kubernetes cluster roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings) _cluster-admin_ and _admin_ have the necessary permissions to deploy the PCG services. @@ -67,12 +75,12 @@ development and testing environments. - 4 vCPUs - 4 GB of memory - 60 GB of disk space - - 1 control plane node and 1 worker node. We recommend using a minimum of 3 nodes for production environments. Refer - to the [PCG Sizing](#pcg-sizing) section for more information. - - A Container Network Interface plugin installed. - - A Container Storage Interface plugin installed. - - The Kubernetes cluster must be set up on a version of Kubernetes that is compatible to your Palette version. Refer - to the [Kubernetes Requirements](./pcg.md#kubernetes-requirements) section to find the required Kubernetes version. + - 1 control plane node with worker capabilities or 3 nodes for production environments. Refer to the + [PCG Sizing](#pcg-sizing) section for more information. + - A Container Network Interface (CNI) plugin installed. + - A Container Storage Interface (CSI) plugin installed. + - The Kubernetes cluster must use a version of Kubernetes that is compatible to your Palette version. Refer to the + [Kubernetes Requirements](./pcg.md#kubernetes-requirements) section to find the required Kubernetes version. - PCG IP address requirements: @@ -85,7 +93,7 @@ development and testing environments. - If you want Palette to display cluster metrics, ensure that the [metrics server](https://github.com/kubernetes-sigs/metrics-server) is installed in the cluster. You can install the - metrics server by using the following command. + metrics server using the following command. ```shell kubectl apply --filename https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml @@ -95,26 +103,24 @@ development and testing environments. Use the following steps to install the PCG in your existing Kubernetes cluster. -1. Log in to [Palette](https://console.spectrocloud.com/). +1. Log in to [Palette](https://console.spectrocloud.com/) as a tenant admin. -2. From the left **Main Menu**, select **Tenant Settings**. +2. From the left main menu, select **Tenant Settings**. -3. Next, on the **Tenant Settings Menu**, select **Private Cloud Gateways** and click on **Add New Private Cloud - Gateway**. +3. From the **Tenant Settings** menu, select **Private Cloud Gateways > Add New Private Cloud Gateway**. -4. Select **Self Hosted** in the next window that Palette displays. +4. On the **Private Cloud Gateway installation instructions** dialog, select **Self Hosted**. -5. Provide a name for the PCG and use the **drop-down Menu** to select the target infrastructure provider. Click - **Confirm** to continue. You will be redirected to the Private Cloud Gateway Overview page. +5. Enter a **Private cloud gateway** name and use the **Cloud type** drop-down menu to select the target infrastructure + provider. **Create** your PCG when finished. You are redirected to the Private Cloud Gateway **Overview** page. -6. To install the Palette agent, copy the kubectl commands from the slide-out panel and execute them against your +6. To install the Palette agent, copy the kubectl commands from the side drawer and execute them against your self-hosted cluster. Issue the commands in the order they are listed. ![View of the cluster details page with the side drawer extended that contains the kubectl commands](/clusters_pcg_deploy-pcg-k8s_kubectl-cmds-view.webp) -7. Close the slide-out panel when you have copied and issued both commands. The PCG Overview page **Cluster Status** - field will display **Pending** while the PCG is deploying. The deployment is complete when the **Cluster Status** - field displays the status **Running**. +7. Close the side drawer once you issue both commands. The **Cluster Status** field displays **Pending** while the PCG + is deploying. The deployment is complete when the **Cluster Status** field displays the status **Running**. :::tip @@ -129,18 +135,17 @@ Use the following steps to install the PCG in your existing Kubernetes cluster. When deployed, the PCG registers itself with Palette. Use the steps below to verify if the PCG registration is successful. -1. Log in to [Palette](https://console.spectrocloud.com/). +1. Log in to [Palette](https://console.spectrocloud.com/) as a tenant admin. -2. Navigate to the **left Main Menu** and select **Tenant Settings**. +2. From the left main menu, select **Tenant Settings**. -3. Next, on the **Tenant Settings Menu**, select **Private Cloud Gateways**. +3. From the **Tenant Settings** menu, select **Private Cloud Gateways**, and choose your PCG. -4. Locate the PCG and verify it is installed and in the **Running** state. +4. On the Private Cloud Gateway **Overview** screen, verify the **Cluster Status** is **Running**. ## Next Steps -When the PCG is in the **Running** state, you can create a cloud account, toggle **Connect Private Cloud Gateway**, and -select the PCG you just deployed. The option to use the PCG you deployed on an existing cluster is only available to the -infrastructure provider you selected when you deployed the PCG. Clusters deployed to the cloud account with the -**Connect Private Cloud Gateway** enabled will use the PCG you deployed to support cluster deployment and removal -operations. +Once your PCG is in the **Running** state, you can add an [AWS](../public-cloud/aws/add-aws-accounts.md), +[Azure](../public-cloud/azure/azure-cloud.md), or [Nutanix](../data-center/nutanix/add-nutanix-cloud-account.md) cloud +account to Palette with the **Connect Private Cloud Gateway** option toggled. You can then use the PCG to deploy and +delete clusters in the cloud environment connected to the PCG. diff --git a/docs/docs-content/clusters/pcg/manage-pcg/configure-proxy-ca-certs.md b/docs/docs-content/clusters/pcg/manage-pcg/configure-proxy-ca-certs.md new file mode 100644 index 00000000000..e4c34099fe2 --- /dev/null +++ b/docs/docs-content/clusters/pcg/manage-pcg/configure-proxy-ca-certs.md @@ -0,0 +1,336 @@ +--- +sidebar_label: "Proxy CA Certificates for Workload Clusters" +title: "Configure Proxy CA Certificates for Workload Clusters" +description: + "Learn how to configure proxy CA certificates for workload clusters deployed through a Private Cloud Gateway (PCG)." +hide_table_of_contents: false +sidebar_position: 15 +tags: ["pcg", "proxy"] +--- + +When your proxy server uses a Certificate Authority (CA) certificate for TLS inspection, workload cluster nodes and pods +need access to the CA certificate to establish trusted connections through the proxy. The +[Reach system](../architecture.md#reach-system) handles injecting proxy environment variables into pods automatically, +but the CA certificate must be configured separately. + +Proxy CA certificates provided during PCG installation are propagated to the PCG cluster nodes but are not automatically +propagated to workload cluster nodes. You must configure the CA certificate for workload clusters using one of the +methods described in this guide. + +There are two levels of CA certificate trust to consider: + +- **Node-level trust** - The CA certificate is written to each workload cluster node's filesystem and added to the + operating system trust store. System services on the node, such as `containerd` and `kubelet`, will trust the proxy. + Both the [tenant-level](#tenant-level) and [cluster profile](#cluster-profile-os-layer) approaches provide node-level + trust. + +- **Pod-level trust** - The CA certificate is mounted into pods as a volume so that applications running inside + containers can trust the proxy. Pod-level trust requires the `podMount` configuration in the cluster profile OS layer + and is _only_ available through the [cluster profile](#cluster-profile-os-layer) approach. Certificates configured at + the tenant level are not mounted into pods. + +## Add CA Certificate to Workload Cluster Nodes + +To make the proxy CA certificate available to workload cluster nodes, configure it at either the tenant level or the +cluster profile level in the OS layer. + +- **Tenant level** - All workload clusters provisioned from the tenant, with the exception of managed Kubernetes + clusters (EKS, AKS, and GKE) and Edge clusters, have the CA certificate injected into their cluster nodes. + +- **Cluster profile OS level** - Only workload clusters deployed using the cluster profile have the CA certificate + injected. + +### Tenant Level + +Use this approach to propagate your proxy server CA certificate to _all_ workload cluster nodes provisioned from the +tenant, with the exception of managed Kubernetes clusters (EKS, AKS, and GKE) and Edge clusters. For managed Kubernetes +and Edge clusters, proceed to the [Cluster Profile OS Layer](#cluster-profile-os-layer) section. + +:::warning + +Tenant-level certificates provide node-level trust only. Applications running inside pods do not inherit the node trust +store and will not trust the proxy CA certificate. If your workloads need to make HTTPS requests through the proxy, use +the [Cluster Profile OS Layer](#cluster-profile-os-layer) approach with the `podMount` configuration to mount the +certificate into pods. + +::: + +#### Prerequisites + +- Palette tenant administrator access. + +- An active PCG cluster with proxy configured. Refer to the appropriate installation guide for your environment: + + - **Palette CLI** - [MAAS](../deploy-pcg/maas.md), [VMware vSphere](../deploy-pcg/vmware.md), + [OpenStack](../deploy-pcg/openstack.md), or [Apache CloudStack](../deploy-pcg/cloudstack.md). + - **Existing Kubernetes cluster** - [Enable and Manage Proxy Configurations](./configure-proxy.md). + +- The proxy CA certificate in Privacy-Enhanced Mail (PEM) format. The certificate file must be named `ca.crt`. + +#### Enablement + +1. Log in to [Palette](https://console.spectrocloud.com) as a tenant admin. + +2. From the left main menu, select **Tenant Settings**. + +3. From the **Tenant Settings** menu, below **Platform**, select **Certificates**. + +4. Select **Add A New Certificate**. + +5. In the **Add Certificate** dialog, enter the **Certificate Name** and **Certificate** value. + +6. **Confirm** your changes. + +#### Validate + +1. Log in to [Palette](https://console.spectrocloud.com). + +2. Deploy a workload cluster through using the PCG, or apply the updated cluster profile to an existing workload + cluster. + +3. SSH into a node in the workload cluster. Refer to the [SSH Keys](../../cluster-management/ssh/ssh-keys.md) and + [SSH Usernames](../../cluster-management/ssh/ssh-usernames.md) guides for information on accessing cluster nodes. + + ```shell + ssh -i @ + ``` + + ```shell hideClipboard title="Example command" + ssh -i ~/.ssh/spectro2026.pem spectro@10.10.149.37 + ``` + +4. Verify the CA certificate file is present on the node. + + ```shell + ls -l /usr/local/share/ca-certificates/spectro-ca.crt + ``` + + ```shell hideClipboard title="Example output" + -rw-r--r-- 1 root root 1049 Mar 6 13:46 /usr/local/share/ca-certificates/spectro-ca.crt + ``` + +5. Verify the certificate entered at either **Tenant Settings > Certificates** matches the CA certificate on the node. + + ```shell + cat /usr/local/share/ca-certificates/spectro-ca.crt + ``` + + ```shell hideClipboard title="Example output" + -----BEGIN CERTIFICATE----- + ************************** + -----END CERTIFICATE----- + ``` + +6. Verify the certificate is included in the system trust store. + + ```shell + openssl verify -CAfile /etc/ssl/certs/ca-certificates.crt /usr/local/share/ca-certificates/spectro-ca.crt + ``` + + ```shell hideClipboard title="Example output" + /usr/local/share/ca-certificates/spectro-ca.crt: OK + ``` + +### Cluster Profile OS Layer + +Use this approach to propagate proxy server CA certificates on a per-cluster basis. This is the only approach that +supports mounting the CA certificate into pods through the [Reach system](../architecture.md#reach-system). + +#### Prerequisites + +- An active PCG cluster with proxy configured. Refer to the appropriate installation guide for your environment: + + - **Palette CLI** - [MAAS](../deploy-pcg/maas.md), [VMware vSphere](../deploy-pcg/vmware.md), + [OpenStack](../deploy-pcg/openstack.md), or [Apache CloudStack](../deploy-pcg/cloudstack.md). + - **Existing Kubernetes cluster** - [Enable and Manage Proxy Configurations](./configure-proxy.md). + +- The proxy CA certificate in Privacy-Enhanced Mail (PEM) format. The certificate file must be named `ca.crt`. + +#### Enablement + +1. Log in to [Palette](https://console.spectrocloud.com). + +2. From the left main menu, select **Profiles**. + +3. Choose an existing cluster profile or create a new cluster profile. For more information, refer to the + [Cluster Profiles](../../../profiles/cluster-profiles/cluster-profiles.md) guide. + +4. In the OS layer of your cluster profile, add your CA certificate to the `content` field under `kubeadmconfig.files`. + The CA certificate file _must_ be named `ca.crt`. + + By default, adding a proxy CA certificate to the cluster profile OS layer places the certificate on each workload + cluster node and adds it to the node trust store. However, applications running inside pods do not automatically + inherit the node trust store. To make the CA certificate available inside pods, use the `podMount` configuration in + the `kubeadmconfig.files` section of the OS layer. The `podMount` configuration instructs Palette to mount the + specified host file into pods through the [Reach system](../architecture.md#reach-system). + + The following table describes the configuration fields. + + | Field | Description | + | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | `targetPath` | The path on the host node where the CA certificate file is written. The certificate file must be named `ca.crt`. | + | `targetOwner` | The file ownership in the format `user:group`. | + | `targetPermissions` | The file permissions in octal notation. | + | `content` | The PEM-encoded CA certificate content. | + | `podMount.allowed` | Set to `true` to enable mounting the host file into pods through the Reach system. | + | `podMount.targetPath` | The path inside pods where the file is mounted. Use `/etc/ssl/certs/ca-certificates.crt` to make the certificate available to most applications that use the system trust store. | + + ```yaml hideClipboard title="Example OS layer configuration" {17-19} + kubeadmconfig: + preKubeadmCommands: + - echo "Executing pre kube admin config commands" + - update-ca-certificates + - "systemctl restart containerd; sleep 3" + - 'while [ ! -S /var/run/containerd/containerd.sock ]; do echo "Waiting for containerd..."; sleep 1; done' + postKubeadmCommands: + - echo "Executing post kube admin config commands" + files: + - targetPath: /usr/local/share/ca-certificates/ca.crt + targetOwner: "root:root" + targetPermissions: "0644" + content: | + -----BEGIN CERTIFICATE----- + ************************* + -----END CERTIFICATE----- + podMount: + allowed: true + targetPath: /etc/ssl/certs/ca-certificates.crt + ``` + + :::warning + + When using `podMount`, the Reach system creates a host-path volume from the file at `targetPath` on the node and + mounts it at `podMount.targetPath` inside pods. If the individual proxy CA certificate file is mounted at + `/etc/ssl/certs/ca-certificates.crt`, it replaces the container's default CA bundle. This means pods will trust your + proxy CA but will not trust standard public Certificate Authorities. + + In most proxy environments this is acceptable because all outbound HTTPS traffic passes through the proxy, which + handles upstream TLS verification on behalf of the pod. However, if your pods make direct TLS connections to services + that bypass the proxy, such as internal services listed in `NO_PROXY`, those connections may fail because the + container no longer has the default public CA certificates. + + To preserve the container's default CA bundle, use a path other than `/etc/ssl/certs/ca-certificates.crt` for + `podMount.targetPath`. However, doing so requires individual configurations for each app to point to the custom + `podMount.targetPath`. + + ::: + +5. Select **Confirm Updates**. + +#### Validate Node-Level Trust + +1. Log in to [Palette](https://console.spectrocloud.com). + +2. Deploy a workload cluster through using the PCG, or apply the updated cluster profile to an existing workload + cluster. + +3. SSH into a node in the workload cluster. Refer to the [SSH Keys](../../cluster-management/ssh/ssh-keys.md) and + [SSH Usernames](../../cluster-management/ssh/ssh-usernames.md) guides for information on accessing cluster nodes. + + ```shell + ssh -i @ + ``` + + ```shell hideClipboard title="Example command" + ssh -i ~/.ssh/spectro2026.pem spectro@10.10.149.37 + ``` + +4. Verify the CA certificate file is present on the node. + + ```shell + ls -l /usr/local/share/ca-certificates/ca.crt + ``` + + ```shell hideClipboard title="Example output" + -rw-r--r-- 1 root root 1049 Mar 6 13:46 /usr/local/share/ca-certificates/ca.crt + ``` + +5. Verify the certificate entered in the cluster profile OS layer matches the CA certificate on the node. + + ```shell + cat /usr/local/share/ca-certificates/ca.crt + ``` + + ```shell hideClipboard title="Example output" + -----BEGIN CERTIFICATE----- + ************************** + -----END CERTIFICATE----- + ``` + +6. Verify the certificate is included in the system trust store. + + ```shell + openssl verify -CAfile /etc/ssl/certs/ca-certificates.crt /usr/local/share/ca-certificates/ca.crt + ``` + + ```shell hideClipboard title="Example output" + /usr/local/share/ca-certificates/ca.crt: OK + ``` + +#### Validate Pod-Level Trust + +1. Log in to [Palette](https://console.spectrocloud.com). + +2. Deploy a workload cluster through using the PCG, or apply the updated cluster profile to an existing workload + cluster. + +3. Download your workload cluster's kubeconfig and configure kubectl to access the cluster. Refer to our + [Access Cluster with CLI](../../cluster-management/palette-webctl.md) guide for more information. + +4. Verify the ClusterPodPreset contains volume and volumeMount entries for the CA certificate. + + ```shell + kubectl get clusterpodpreset proxy --output yaml | grep --after-context=10 volumeMounts + ``` + + The output should include `volumes` and `volumeMounts` entries referencing the CA certificate path. + + ```yaml hideClipboard title="Example output" {4,8} + volumeMounts: + - metadata: + merge_strategy: replace + mountPath: /etc/ssl/certs/ca-certificates.crt + name: ca-crt-11d45c1c + volumes: + - hostPath: + path: /usr/local/share/ca-certificates/ca.crt + type: File + metadata: + merge_strategy: replace + ``` + +5. Create a temporary namespace with the `privileged` Pod Security Standard. The Reach system injects `hostPath` volumes + into matching pods, which requires the `privileged` security level. + + ```shell + kubectl create namespace test-proxy-cert + kubectl label namespace test-proxy-cert pod-security.kubernetes.io/enforce=privileged + ``` + +6. Deploy a test pod with the `spectrocloud.com/connection: proxy` label. The Reach system only injects volumes into + pods that match the ClusterPodPreset's label selector. + + ```shell + kubectl run test-proxy-cert --namespace test-proxy-cert --image=busybox --restart=Never \ + --labels="spectrocloud.com/connection=proxy" \ + --command -- sleep 3600 + ``` + +7. Verify the certificate file is mounted inside the container. The certificate should match the content pasted in to + the cluster profile OS layer. + + ```shell + kubectl exec test-proxy-cert --namespace test-proxy-cert -- cat /etc/ssl/certs/ca-certificates.crt + ``` + + ```shell hideClipboard title="Example output" + -----BEGIN CERTIFICATE----- + ************************** + -----END CERTIFICATE----- + ``` + +8. Clean up the test namespace and pod. + + ```shell + kubectl delete namespace test-proxy-cert + ``` diff --git a/docs/docs-content/clusters/pcg/manage-pcg/configure-proxy.md b/docs/docs-content/clusters/pcg/manage-pcg/configure-proxy.md index c329df918ce..e975c24e7df 100644 --- a/docs/docs-content/clusters/pcg/manage-pcg/configure-proxy.md +++ b/docs/docs-content/clusters/pcg/manage-pcg/configure-proxy.md @@ -9,20 +9,28 @@ sidebar_position: 14 tags: ["pcg", "proxy"] --- -You can add and manage proxy configurations for a Private Cloud Gateway (PCG) deployed into an existing Kubernetes -cluster. By default, a PCG deployed to an existing Kubernetes cluster does not have a proxy configuration. If your -infrastructure environment requires a proxy configuration, use the instructions in this guide to add and manage proxy -configurations for a PCG deployed in a Kubernetes cluster and to ensure workload clusters deployed through Palette -inherit the proxy configuration from the PCG cluster. - -:::info - -Workload clusters deployed through Palette will inherit proxy configuration from the PCG cluster. The PCG is not used as -a network proxy for deployed workload clusters and does not provide internet connectivity for the workload clusters. -Individual workload clusters must have their own proxy configurations to access the internet. +You can add and manage proxy configurations for a Private Cloud Gateway (PCG) deployed into an +[existing Kubernetes cluster](../deploy-pcg-k8s.md). By default, a PCG deployed to an existing Kubernetes cluster does +not have a proxy configuration. If your infrastructure environment requires a proxy configuration, use the instructions +in this guide to add and manage proxy configurations for a PCG deployed in a Kubernetes cluster and to ensure workload +clusters deployed through Palette inherit the proxy configuration from the PCG cluster. The PCG is not used as a network +proxy for deployed workload clusters and does not provide internet connectivity for the workload clusters. Individual +workload clusters must have their own proxy configurations to access the internet. + +:::warning + +This guide applies only to PCGs deployed to an [existing Kubernetes cluster](../deploy-pcg-k8s.md). If you deployed a +PCG through the [Palette CLI](../pcg.md#supported-environments), the Reach system and `cert-manager` are automatically +installed during PCG provisioning. Refer to the respective platform installation guide for +[MAAS](../deploy-pcg/maas.md), [VMware vSphere](../deploy-pcg/vmware.md), or +[Apache CloudStack](../deploy-pcg/cloudstack.md) for instructions on how to configure proxy settings during the CLI +installation process. ::: +The Reach system is responsible for injecting proxy configuration into pods. To learn how the Reach system works, refer +to the [Reach System](../architecture.md#reach-system) section on the PCG Architecture page. + Use the following steps to add and manage proxy configurations for a PCG. ## Prerequisites @@ -30,14 +38,6 @@ Use the following steps to add and manage proxy configurations for a PCG. - A PCG is deployed into an active and healthy Kubernetes cluster. Refer to [Deploy a PCG to an Existing Kubernetes Cluster](../deploy-pcg-k8s.md) for additional guidance. - :::warning - - If you deployed a [PCG through the Palette CLI](../pcg.md#supported-environments), refer to the respective platform - installation guide for instructions on how to configure proxy settings during the installation process through the - CLI. - - ::: - - The kubeconfig file for the Kubernetes cluster where the PCG is deployed. The kubeconfig file is used to authenticate with the Kubernetes cluster and deploy the Reach service. @@ -69,6 +69,8 @@ Use the following steps to add and manage proxy configurations for a PCG. ::: +- If the proxy server uses a CA certificate, the certificate file must be named `ca.crt` and be in PEM format. + - Download the Reach Helm Chart provided by our support team. The Reach Helm Chart is used to deploy the Reach service into the Kubernetes cluster where the PCG is deployed. The Reach service is used to manage proxy configurations for the PCG. Contact our support team to obtain the Reach Helm Chart. @@ -178,6 +180,14 @@ Once the Reach service is deployed, the PCG will use the proxy configuration in account is configured to use the PCG, the proxy configuration will be inherited by the workload clusters deployed through Palette. +:::tip + +If your proxy server uses a CA certificate, workload cluster nodes and pods need separate configuration to trust the +proxy. Refer to [Configure Proxy CA Certificates for Workload Clusters](./configure-proxy-ca-certs.md) for instructions +on configuring node-level and pod-level CA certificate trust. + +::: + ## Validate Use the following steps to validate that the Reach service is deployed and that the proxy configuration is enabled for diff --git a/docs/docs-content/clusters/pcg/manage-pcg/manage-pcg.md b/docs/docs-content/clusters/pcg/manage-pcg/manage-pcg.md index b8a7b42c57c..7672362b81c 100644 --- a/docs/docs-content/clusters/pcg/manage-pcg/manage-pcg.md +++ b/docs/docs-content/clusters/pcg/manage-pcg/manage-pcg.md @@ -16,6 +16,8 @@ topics to learn more about managing a PCG. - [Enable and Manage Proxy Configurations](./configure-proxy.md) +- [Proxy CA Certificates for Workload Clusters](./configure-proxy-ca-certs.md) + - [Increase PCG Node Count](./scale-pcg-nodes.md) - [Delete a PCG](./remove-pcg.md) diff --git a/static/assets/docs/images/clusters_pcg_deploy-pcg-k8s_kubectl-cmds-view.webp b/static/assets/docs/images/clusters_pcg_deploy-pcg-k8s_kubectl-cmds-view.webp index a025104c1e3..4e629ff8b24 100644 Binary files a/static/assets/docs/images/clusters_pcg_deploy-pcg-k8s_kubectl-cmds-view.webp and b/static/assets/docs/images/clusters_pcg_deploy-pcg-k8s_kubectl-cmds-view.webp differ