Merge pull request #998 from stackhpc/secgroups-allow-all

✨ Implement allowAllInClusterTraffic flag

Author: k8s-ci-robot

api/v1alpha4/openstackcluster_types.go

@@ -75,14 +75,21 @@ type OpenStackClusterSpec struct {
 	// APIServerLoadBalancerAdditionalPorts adds additional ports to the APIServerLoadBalancer
 	APIServerLoadBalancerAdditionalPorts []int `json:"apiServerLoadBalancerAdditionalPorts,omitempty"`
 
-	// ManagedSecurityGroups defines that kubernetes manages the OpenStack security groups
-	// for now, that means that we'll create security group allows traffic to/from
-	// machines belonging to that group based on Calico CNI plugin default network
-	// requirements: BGP and IP-in-IP for master node(s) and worker node(s) respectively.
-	// In the future, we could make this more flexible.
+	// ManagedSecurityGroups determines whether OpenStack security groups for the cluster
+	// will be managed by the OpenStack provider or whether pre-existing security groups will
+	// be specified as part of the configuration.
+	// By default, the managed security groups have rules that allow the Kubelet, etcd, the
+	// Kubernetes API server and the Calico CNI plugin to function correctly.
 	// +optional
 	ManagedSecurityGroups bool `json:"managedSecurityGroups"`
 
+	// AllowAllInClusterTraffic is only used when managed security groups are in use.
+	// If set to true, the rules for the managed security groups are configured so that all
+	// ingress and egress between cluster nodes is permitted, allowing CNIs other than
+	// Calico to be used.
+	// +optional
+	AllowAllInClusterTraffic bool `json:"allowAllInClusterTraffic"`
+
 	// DisablePortSecurity disables the port security of the network created for the
 	// Kubernetes cluster, which also disables SecurityGroups
 	DisablePortSecurity bool `json:"disablePortSecurity,omitempty"`

config/crd/bases/infrastructure.cluster.x-k8s.io_openstackclusters.yaml

@@ -1072,6 +1072,12 @@ spec:
           spec:
             description: OpenStackClusterSpec defines the desired state of OpenStackCluster.
             properties:
+              allowAllInClusterTraffic:
+                description: AllowAllInClusterTraffic is only used when managed security
+                  groups are in use. If set to true, the rules for the managed security
+                  groups are configured so that all ingress and egress between cluster
+                  nodes is permitted, allowing CNIs other than Calico to be used.
+                type: boolean
               apiServerFloatingIP:
                 description: APIServerFloatingIP is the floatingIP which will be associated
                   to the APIServer. The floatingIP will be created if it not already
@@ -1553,12 +1559,12 @@ spec:
                   properties are mandatory: APIServerFloatingIP, APIServerPort'
                 type: boolean
               managedSecurityGroups:
-                description: 'ManagedSecurityGroups defines that kubernetes manages
-                  the OpenStack security groups for now, that means that we''ll create
-                  security group allows traffic to/from machines belonging to that
-                  group based on Calico CNI plugin default network requirements: BGP
-                  and IP-in-IP for master node(s) and worker node(s) respectively.
-                  In the future, we could make this more flexible.'
+                description: ManagedSecurityGroups determines whether OpenStack security
+                  groups for the cluster will be managed by the OpenStack provider
+                  or whether pre-existing security groups will be specified as part
+                  of the configuration. By default, the managed security groups have
+                  rules that allow the Kubelet, etcd, the Kubernetes API server and
+                  the Calico CNI plugin to function correctly.
                 type: boolean
               network:
                 description: If NodeCIDR cannot be set this can be used to detect

config/crd/bases/infrastructure.cluster.x-k8s.io_openstackclustertemplates.yaml

@@ -50,6 +50,13 @@ spec:
                     description: OpenStackClusterSpec defines the desired state of
                       OpenStackCluster.
                     properties:
+                      allowAllInClusterTraffic:
+                        description: AllowAllInClusterTraffic is only used when managed
+                          security groups are in use. If set to true, the rules for
+                          the managed security groups are configured so that all ingress
+                          and egress between cluster nodes is permitted, allowing
+                          CNIs other than Calico to be used.
+                        type: boolean
                       apiServerFloatingIP:
                         description: APIServerFloatingIP is the floatingIP which will
                           be associated to the APIServer. The floatingIP will be created
@@ -544,13 +551,13 @@ spec:
                           APIServerPort'
                         type: boolean
                       managedSecurityGroups:
-                        description: 'ManagedSecurityGroups defines that kubernetes
-                          manages the OpenStack security groups for now, that means
-                          that we''ll create security group allows traffic to/from
-                          machines belonging to that group based on Calico CNI plugin
-                          default network requirements: BGP and IP-in-IP for master
-                          node(s) and worker node(s) respectively. In the future,
-                          we could make this more flexible.'
+                        description: ManagedSecurityGroups determines whether OpenStack
+                          security groups for the cluster will be managed by the OpenStack
+                          provider or whether pre-existing security groups will be
+                          specified as part of the configuration. By default, the
+                          managed security groups have rules that allow the Kubelet,
+                          etcd, the Kubernetes API server and the Calico CNI plugin
+                          to function correctly.
                         type: boolean
                       network:
                         description: If NodeCIDR cannot be set this can be used to

docs/book/src/clusteropenstack/configuration.md

@@ -3,7 +3,7 @@
 **Table of Contents**  *generated with [DocToc](https://github.com/thlorenz/doctoc)*
 
 - [Required configuration](#required-configuration)
-  - [OpenStack Version](#openstack-version)
+  - [OpenStack version](#openstack-version)
   - [Operating system image](#operating-system-image)
   - [SSH key pair](#ssh-key-pair)
   - [OpenStack credential](#openstack-credential)
@@ -19,6 +19,7 @@
   - [Multiple Networks](#multiple-networks)
   - [Subnet Filters](#subnet-filters)
   - [Ports](#ports)
+  - [Security groups](#security-groups)
   - [Tagging](#tagging)
   - [Metadata](#metadata)
   - [Boot From Volume](#boot-from-volume)
@@ -77,19 +78,9 @@ openstack keypair create [--public-key <file> | --private-key <file>] <name>
 
 The key pair name must be exposed as an environment variable `OPENSTACK_SSH_KEY_NAME`.
 
-If you want to login to each machine by ssh,  you can [access nodes through the bastion host via SSH](#accessing-nodes-through-the-bastion-host-via-ssh). Otherwise you have to configure security groups. If `spec.managedSecurityGroups` of `OpenStackCluster` set to true, two security groups will be created and added to the instances. One is `k8s-cluster-${NAMESPACE}-${CLUSTER_NAME}-secgroup-controlplane`, another is `k8s-cluster-${NAMESPACE}-${CLUSTER_NAME}-secgroup-worker`. These security group rules include the kubeadm's [Check required ports](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#check-required-ports) so that each node can not be logged in through ssh by default. Please add pre-existing security group allowing ssh port to OpenStackMachineTemplate spec. Here is an example:
-
-```yaml
-apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
-kind: OpenStackMachineTemplate
-metadata:
-  name: ${CLUSTER_NAME}-control-plane
-spec:
-  template:
-    spec:
-      securityGroups:
-      - name: allow-ssh
-```
+In order to access cluster nodes via SSH, you must either
+[access nodes through the bastion host](#accessing-nodes-through-the-bastion-host-via-ssh)
+or [configure custom security groups](#security-groups) with rules allowing ingress for port 22.
 
 ## OpenStack credential
 
@@ -255,6 +246,47 @@ spec:
     ...
 ```
 
+## Security groups
+
+Security groups are used to determine which ports of the cluster nodes are accessible from where.
+
+If `spec.managedSecurityGroups` of `OpenStackCluster` is set to `true`, two security groups named
+`k8s-cluster-${NAMESPACE}-${CLUSTER_NAME}-secgroup-controlplane` and
+`k8s-cluster-${NAMESPACE}-${CLUSTER_NAME}-secgroup-worker` will be created and added to the control
+plane and worker nodes respectively.
+
+By default, these groups have rules that allow the following traffic:
+
+  * Control plane nodes 
+    * API server traffic from anywhere
+    * Etcd traffic from other control plane nodes
+    * Kubelet traffic from other cluster nodes
+    * Calico CNI traffic from other cluster nodes
+  * Worker nodes
+    * Node port traffic from anywhere
+    * Kubelet traffic from other cluster nodes
+    * Calico CNI traffic from other cluster nodes
+
+To use a CNI other than Calico, the flag `OpenStackCluster.spec.allowAllInClusterTraffic` can be
+set to `true`. With this flag set, the rules for the managed security groups permit all traffic
+between cluster nodes on all ports and protocols (API server and node port traffic is still
+permitted from anywhere, as with the default rules).
+
+If this is not flexible enough, pre-existing security groups can be added to the
+spec of an `OpenStackMachineTemplate`, e.g.:
+
+```yaml
+apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
+kind: OpenStackMachineTemplate
+metadata:
+  name: ${CLUSTER_NAME}-control-plane
+spec:
+  template:
+    spec:
+      securityGroups:
+      - name: allow-ssh
+```
+
 ## Tagging
 
 You have the ability to tag all resources created by the cluster in the `cluster.yaml` file. Here is an example how to configure tagging: