BREAKING CHANGE: switch all security list of worker nodes and load balancers to NSGs for greater flexibility and control on ports and cidr blocks (#399)

* BREAKING CHANGE: switch all security list of worker nodes and load balancers to NSGs for greater flexibility
Closes #398

Signed-off-by: Ali Mukadam <[email protected]>

* fix: fixed incorrect NSG rules, added separate NSG for WAF, made NSGs conditional, renamed variables

Signed-off-by: Ali Mukadam <[email protected]>

* fix: nsg rules attached in the wrong nsgs

corrected nsg rules to their nsgs, added rules for load balancer
healtchecks, outputs for default load balancer ngs, updated docs.

Signed-off-by: Ali Mukadam <[email protected]>

* fix: documented the use of waf NSG, tightened security rules.

Signed-off-by: Ali Mukadam <[email protected]>

* fix: Updated changelog, health check security rules

Signed-off-by: Ali Mukadam <[email protected]>

* fix: added port range for internal load balancer ports ingress and egress

Signed-off-by: Ali Mukadam <[email protected]>

* fix: readded 1 rule for control plane which was mistakenly deleted.

Signed-off-by: Ali Mukadam <[email protected]>

* fix: added worker_nsgs in terraform.tfvars.example file

Signed-off-by: Ali Mukadam <[email protected]>

Author: hyder

CHANGELOG.adoc

@@ -7,8 +7,8 @@ All notable changes to this project are documented in this file.
 
 The format is based on {uri-changelog}[Keep a Changelog].
 
-=== Unreleased
-# Breaking changes
+= Unreleased
+=== Breaking changes
 * Set minimum version to Terraform 1.0.0
 * Removed base module and use vcn, bastion and operator modules directly
 * Renamed and standardized all control variables
@@ -20,29 +20,48 @@ The format is based on {uri-changelog}[Keep a Changelog].
 * Moved dynamic group and policy for kms into oke module
 * Added a 30s delay between policy creation for kms and cluster creation to allow for global propagation
 * Added a home provider in oke module for dynamic group and policy creation
+* Changed from security list to NSGs for better flexibility
 
-# Changes
+=== Changes
 * Changed default Kubernetes version to v1.20.8 and removed v1.16.8, v1.17.9 from docs.
-* Bug fix: Use correct calico file to install calico for networking policy only (#307)
 * Added support for GPU and ARM shapes (#302)
 * VCN module upgraded to VCN 3.0.0. This allows supporting multiple cidr blocks (#360)
+* Bastion and operator sub-modules upgraded to 3.0.0 (#183)
 * kubeconfig on operator always uses PRIVATE_ENDPOINT (#358)
 * Documented providers in quickstart (#355)
 * Renamed tags to freeform_tags in line with other modules (#364)
 * Added validation on some variables (#370)
 
-# New Features
+=== New Features
 * Added OCI Bastion Service as option to access operator or control plane
 * Added support for reserved public IP address for NAT gateway (#311)
 * Added LPGs for hub and spoke deployment model (#295)
 * Allow access to operator via OCI Bastion service (#352)
-* Added support for using NSGs for cluster endpoint (#343 )
+* Added support for using NSGs for cluster endpoint (#343)
 * Added option to disable worker node access to Internet. Users can only pull images from OCIR (#331)
 * Added ability to specify api and private ssh keys using heredoc format with a variable (#375) 
 
-# Bug fixes
+=== Bug fixes
 * Added home region to update dynamic group script for cases when actual region is different from tenancy home region (#347)
-* Added 1 missing rule for operator to access control plane (#349 )
+* Added 1 missing rule for operator to access control plane (#349)
+
+=== Known issues
+* Enabling WAF has to be done in 2 stages:
+** Create the cluster along with the VCN and other resources without WAF enabled by setting `enable_waf=false`
+** Subsequently enable WAF by setting `enable_waf=true`
+
+= 3.3.0
+
+=== Additions
+* Support for using reserved public IP address for NAT Gateway (#311) with new parameter nat_gateway_public_ip_id
+* Support for GPU and ARM Shapes (#302 )
+* Conditional checks for WAF CIDR block data source
+
+=== Changes
+* Default Kubernetes version changed to v1.20.8
+
+=== Bug fixes
+* Use correct manifest to install Calico as policy (#306)
 
 = 3.2.0 (May 12, 2021)
 

docs/instructions.adoc

@@ -299,10 +299,11 @@ service_account_cluster_role_binding = ""
 
 You can monitor and protect the load balancers created by OKE using {uri-oci-waf}[OCI Web Application Firewall].
 
-If you would like to monitor and protect your application with OCI Web Application firewall, set *_enable_waf = true_*.
+If you would like to monitor and protect your application with OCI Web Application firewall, set `enable_waf = true` *_after_* the cluster has been created. In other words, you need to run `terraform apply` twice. In the first `terraform apply`, `enable_waf` should be set to `false` while the VCN and other resources are created. You can then set `enable_waf=true` and run `terraform apply` again.
 
-You'll then need to:
+You will then need to:
 
+. add the WAF NSG to the load balancer. Refer to the example in {uri-topology}#using-public-load-balancers[Topology - Using Public Load Balancers] for an example
 . {uri-oci-waf-policy}[create a WAF Policy]
 . {uri-oci-waf-dns}[Update your DNS records to enable WAF]
 

docs/terraformoptions.adoc

@@ -543,19 +543,19 @@ EOT
 |
 |oke
 
-|control_plane_access
+|control_plane_type
 |Whether to allow public or private access to the control plane endpoint.
 |public/private
 |public
 
-|control_plane_access_source
+|control_plane_allowed_cidrs
 |The list of CIDR blocks from which the control plane can be accessed.
 |`[0.0.0.0/0]`
 |`[]`
 
 |control_plane_nsgs
-|A list of the network security groups (NSGs) ids to apply to the cluster endpoint.
-|[]
+|An additional list of network security groups (NSG) ids for the cluster endpoint that can be created subsequently.
+|["ocid1.networksecuritygroup.oc1....","ocid1.networksecuritygroup.oc1...."]
 |[]
 
 |`dashboard_enabled`
@@ -657,7 +657,12 @@ node_pools = {
 |
 |7.9
 
-|`worker_mode`
+|`worker_nsgs`
+|An additional list of network security groups (NSG) ids for the worker nodes that can be created subsequently.
+|["ocid1.networksecuritygroup.oc1....","ocid1.networksecuritygroup.oc1...."]
+|[]
+
+|`worker_type`
 |Whether the worker nodes should be public or private. Private requires NAT gateway.
 |private/public
 |private
@@ -704,17 +709,17 @@ node_pools = {
 |Values
 |Default
 
-|lb_subnet_type
+|load_balancers
 |The type of load balancer subnets to create. 
 
-Even if you set the load balancer subnets to be internal, you still need to set the correct {uri-oci-loadbalancer-annotations}[annotations] when creating internal load balancers. Just setting the subnet to be private is *_not_* sufficient.
+Even if you set the load balancer subnets to be internal, you still need to set the correct {uri-oci-loadbalancer-annotations}[annotations] when creating internal load balancers. Just setting this value to internal is *_not_* sufficient.
 
 Refer to {uri-topology}[topology] for more thorough examples.
 |both, internal, public
 |public
 
-|preferred_lb_subnet_type
-|The preferred load balancer subnets that OKE will automatically choose when creating load balancers. If 'public' is chosen, the value for lb_subnet_type must be either 'public' or 'both'. If 'private' is chosen, the value for lb_subnet_type must be either 'internal' or 'both'.
+|preferred_load_balancer
+|The preferred load balancer subnets that OKE will automatically choose when creating load balancers. If 'public' is chosen, the value for load_balancers must be either 'public' or 'both'. If 'private' is chosen, the value for load_balancers must be either 'internal' or 'both'.
 
 Even if you set the load balancer subnets to be internal, you still need to set the correct {uri-oci-loadbalancer-annotations}[annotations] when creating internal load balancers. Just setting the subnet to be private is *_not_* sufficient.
 
@@ -723,11 +728,26 @@ Refer to {uri-topology}[topology] for more thorough examples.
 |internal/public
 |public
 
-|public_lb_ports
-|The List of allowed ports for public load balancers. Use a string if specifying a range.
-|`e.g.: [80,443,8080]` or [80,443, "7001-7010"]
+|internal_lb_allowed_cidrs
+|The list of CIDR blocks from which the internal load balancer can be accessed.
+|`e.g.: ["0.0.0.0/0"]
+|`["0.0.0.0/0"]`
+
+|internal_lb_allowed_ports
+|List of allowed ports for internal load balancers.
+|e.g.: `[80, 443]`
 |`[80, 443]`
 
+|public_lb_allowed_cidrs
+|The list of CIDR blocks from which the public load balancer can be accessed.
+|e.g.: `["0.0.0.0/0"]`
+|`["0.0.0.0/0"]`
+
+|public_lb_allowed_ports
+|List of allowed ports for public load balancers.
+|e.g.: `[443]`
+|`[443]`
+
 |===
 
 == OCIR