|
| 1 | +## Baseline Components and Networking Considerations |
| 2 | + |
| 3 | +### 1. **Ingress Controllers** |
| 4 | + |
| 5 | +- **ingress-nginx** is deployed as the ingress controller. |
| 6 | +- It exposes internal Kubernetes services to external clients, typically via AWS Network Load Balancer (NLB) or cloud-specific load balancers. |
| 7 | +- It manages routing of HTTP/HTTPS traffic into the cluster, enforcing TLS, host/path rules, and sometimes source IP restrictions. |
| 8 | +- The choice of ingress controller and its configuration (e.g., annotations, load balancer type) directly affects how external traffic enters your cluster. |
| 9 | +- **Note:** |
| 10 | + - Ensure that your firewall and security groups allow inbound traffic to the load balancer and that DNS is configured to point to the ingress endpoint. |
| 11 | + - **Common endpoints that may need to be allowed:** |
| 12 | + - Ingress controller endpoints (HTTP/HTTPS) |
| 13 | + - Cloud load balancer endpoints (varies by provider) |
| 14 | + - **Container registries that may need to be allowlisted:** |
| 15 | + - quay.io |
| 16 | + |
| 17 | +### 2. **Load Balancers** |
| 18 | + |
| 19 | +- The baseline roles configure and deploy cloud-native load balancers (e.g., AWS NLB) via ingress controllers and service annotations. |
| 20 | +- These load balancers provide public or private endpoints for accessing cluster services. |
| 21 | +- Annotations are used to make/specify that the load balancer is either public or private. |
| 22 | +- **Note:** |
| 23 | + - Exposing services via public load balancers may have security implications. Restrict access as needed using security groups, firewall rules, or Kubernetes network policies. |
| 24 | + - **Common endpoints that may need to be allowed:** |
| 25 | + - Load balancer endpoints (public/private IPs, HTTP/HTTPS) |
| 26 | + - Health check endpoints (varies by provider) |
| 27 | + |
| 28 | +### 3. **Cluster Autoscaler** |
| 29 | + |
| 30 | +- The **cluster-autoscaler** works by scaling the number of nodes up or down, it can impact the availability of network endpoints and the distribution of pods across subnets. |
| 31 | +- **Note:** |
| 32 | + - Ensure that your cloud provider IAM roles and API access allow autoscaler operations, and that new nodes can join the cluster network without manual intervention. |
| 33 | + - **Common endpoints that may need to be allowed:** |
| 34 | + - Cloud provider APIs (for scaling, usually outbound HTTPS) |
| 35 | + - Kubernetes API server (internal cluster traffic) |
| 36 | + |
| 37 | +### 4. **Metrics Server, Cert-Manager, and Storage CSI Drivers** |
| 38 | + |
| 39 | +- **metrics-server** and **cert-manager** may require network access to the Kubernetes API and external endpoints (for certificate validation). |
| 40 | +- **CSI drivers** (such as NFS, EFS, etc.) may require network connectivity to storage backends (e.g., EFS, NFS servers). |
| 41 | +- **ebs-csi-driver** (AWS only) does not expose services externally, but requires network connectivity to AWS APIs and EBS endpoints for dynamic volume provisioning. This enables persistent storage for pods on AWS and may require outbound access to AWS services. |
| 42 | +- **Note:** |
| 43 | + - For NFS/EFS: Ensure that all cluster nodes have network access (NFS/TCP 2049) to the NFS or EFS server. Firewalls and security groups must allow this traffic. |
| 44 | + - For AWS EBS: Nodes must have outbound access to AWS APIs and the correct IAM permissions. |
| 45 | + - For cert-manager: If using ACME (Let's Encrypt), ensure outbound HTTPS access to the internet. |
| 46 | + - **Common endpoints that may need to be allowed:** |
| 47 | + - Kubernetes API server (internal cluster traffic) |
| 48 | + - NFS/EFS storage (TCP 2049) |
| 49 | + - AWS APIs (for EBS, outbound HTTPS) |
| 50 | + - Certificate authorities (e.g., Let's Encrypt ACME, outbound HTTPS) |
| 51 | + - **Container registries that may need to be allowlisted:** |
| 52 | + - metrics-server: registry.k8s.io |
| 53 | + - cert-manager: quay.io |
| 54 | + - csi-driver-nfs: registry.k8s.io, gcr.io |
| 55 | + - ebs-csi-driver: public.ecr.aws |
| 56 | + |
| 57 | +### 5. **Namespace and Resource Management** |
| 58 | + |
| 59 | +- The baseline roles create namespaces and manage resources, which can include network policies or service accounts that affect pod-to-pod communication and access to external resources. |
| 60 | +- **Note:** |
| 61 | + - If network policies are enabled, ensure that required inter-pod and pod-to-service communications are allowed. Review any default deny policies. |
| 62 | + - **Common endpoints that may need to be allowed:** |
| 63 | + - Pod-to-pod and pod-to-service communication (internal cluster traffic) |
| 64 | + - External services as required by workloads |
| 65 | + |
| 66 | +### 6. **Jump Server (Bastion Host) and SSH Access** |
| 67 | + |
| 68 | +- If a jump server is used, SSH access is required from the Ansible control node to the jump server, and from the jump server to the NFS server (if managing NFS exports or directories). |
| 69 | +- **Note:** |
| 70 | + - Ensure that SSH keys are properly configured and distributed. |
| 71 | + - Security groups/firewalls must allow SSH (typically TCP 22) from the control node to the jump server, and from the jump server to the NFS server. |
| 72 | + - The jump server must have the NFS share mounted and accessible at the configured path. |
| 73 | + - **Common endpoints that may need to be allowed:** |
| 74 | + - SSH (TCP 22) from control node to jump server |
| 75 | + - SSH (TCP 22) from jump server to NFS server |
| 76 | + - NFS (TCP 2049) from jump server to NFS server |
| 77 | + |
| 78 | +### 7. **Viya Deployment Manager (VDM)** |
| 79 | + |
| 80 | +- The Viya Deployment Manager (VDM) role orchestrates the deployment of core SAS Viya services and supporting infrastructure. It may create internal or external services (such as Postgres or Elasticsearch), configure ingress and TLS, expose endpoints (e.g., SAS/CONNECT, Consul UI), and manage storage overlays. VDM can also affect namespace isolation and network policies, especially in multi-tenant environments. Review VDM configuration and deployment options to ensure all required network access is permitted. |
| 81 | +- **Note:** |
| 82 | + - VDM may expose new endpoints or require connectivity to internal/external databases, storage, or certificate authorities. Ensure that firewalls, security groups, and network policies allow required traffic for all VDM-managed services and integrations, especially in multi-tenant or restricted environments. |
| 83 | + - **Common endpoints that may need to be allowed:** |
| 84 | + - Ingress controller endpoints (HTTP/HTTPS) |
| 85 | + - SAS/CONNECT load balancer endpoints |
| 86 | + - Consul UI (port 8500, if enabled) |
| 87 | + - Internal/external Postgres (default port 5432) |
| 88 | + - Internal Elasticsearch (default port 9200) |
| 89 | + - NFS/EFS storage (TCP 2049) |
| 90 | + - AWS APIs (for EBS, outbound HTTPS) |
| 91 | + - Certificate authorities (e.g., Let's Encrypt ACME, outbound HTTPS) |
| 92 | + - Container registries (for pulling images, outbound HTTPS) |
| 93 | + - **Container registries that may need to be allowlisted:** |
| 94 | + - quay.io |
| 95 | + - registry.k8s.io |
| 96 | + - gcr.io |
| 97 | + - mcr.microsoft.com |
| 98 | + - public.ecr.aws |
| 99 | + - (plus any additional registries for SAS Viya images and other required workloads) |
| 100 | + |
| 101 | +--- |
| 102 | + |
| 103 | +### Container Registries to Allowlist by Cloud Provider |
| 104 | + |
| 105 | +#### AWS |
| 106 | +- quay.io (ingress-nginx, cert-manager) |
| 107 | +- registry.k8s.io (metrics-server, csi-driver-nfs) |
| 108 | +- gcr.io (csi-driver-nfs) |
| 109 | +- public.ecr.aws (ebs-csi-driver) |
| 110 | + |
| 111 | +#### Azure |
| 112 | +- quay.io (ingress-nginx, cert-manager) |
| 113 | +- registry.k8s.io (csi-driver-nfs) |
| 114 | +- gcr.io (csi-driver-nfs) |
| 115 | + |
| 116 | +#### GCP |
| 117 | +- quay.io (ingress-nginx, cert-manager) |
| 118 | +- registry.k8s.io (csi-driver-nfs) |
| 119 | +- gcr.io (csi-driver-nfs) |
| 120 | + |
| 121 | +#### Generic K8s / NFS |
| 122 | +- quay.io (ingress-nginx, cert-manager) |
| 123 | +- registry.k8s.io (csi-driver-nfs) |
| 124 | +- gcr.io (csi-driver-nfs) |
| 125 | + |
| 126 | +**Notes:** |
| 127 | +- `metrics-server` and `ebs-csi-driver` are AWS only, so their registries are not needed for Azure or GCP. |
| 128 | +- If you are using only a specific cloud provider, you only need to allowlist the registries listed for that provider. |
| 129 | + |
| 130 | +--- |
| 131 | + |
| 132 | +## **Summary Table** |
| 133 | + |
| 134 | +|Component|Networking Considerations| |
| 135 | +|---|---| |
| 136 | +|ingress-nginx|Exposes services externally, manages HTTP/S routing, uses cloud load balancers| |
| 137 | +|Cluster Autoscaler|Indirectly affects networking by scaling nodes/pods| |
| 138 | +|metrics-server|Minimal, requires API access| |
| 139 | +|cert-manager|Minimal, may require outbound access for ACME| |
| 140 | +|CSI Drivers (NFS, EFS, etc.)|May require network access to storage backends| |
| 141 | +|ebs-csi-driver|Requires network connectivity to AWS APIs and EBS endpoints for dynamic volume provisioning; does not expose services externally but enables persistent storage for pods on AWS| |
| 142 | +|Jump Server|Requires SSH access from control node and to NFS server; must have NFS share mounted| |
| 143 | +|VDM (Viya Deployment Manager)|May create internal/external services (e.g., Postgres, Elasticsearch), configure ingress/TLS, expose endpoints (e.g., SAS/CONNECT, Consul UI), and require network access to storage backends and certificate authorities. Multi-tenancy may affect namespace isolation and network policies.| |
0 commit comments