Merge pull request #305 from sassoftware/staging

6.5.0 - April 20, 2023

Author: riragh

TODO.md

@@ -4,7 +4,6 @@
 
 ## Changes for SAS locked down
 - make var for setting outbound_type. Needing for locked down accounts where creating routing tables is not permitted
-- add Private access (VNet Integration) for flexible postgres
 
 ## Update docs
 - Add this line back into CONFIG-VARS.md @ 122

docs/CONFIG-VARS.md

@@ -85,7 +85,7 @@ You can use `default_public_access_cidrs` to set a default range for all created
 | subnets | Subnets to be created and their settings | map(object) | *check below* | This variable is ignored when subnet_names is set (AKA bring your own subnets). All defined subnets must exist within the vnet address space. |
 | cluster_egress_type | The outbound (egress) routing method to be used for this Kubernetes Cluster | string | "loadBalancer" | Possible values: <ul><li>`loadBalancer`<li>`userDefinedRouting`</ul> By default, AKS will create and use a [loadbalancer](https://docs.microsoft.com/en-us/azure/aks/load-balancer-standard) for outgoing connections.<p>Set to `userDefinedRouting` when using your own network [egress](https://docs.microsoft.com/en-us/azure/aks/egress-outboundtype).|
 | aks_network_plugin | Network plugin to use for networking. Currently supported values are `azure` and `kubenet`| string | `kubenet`| For details see Azure's documentation on: [configure kubenet](https://docs.microsoft.com/en-us/azure/aks/configure-kubenet), [Configure Azure CNI](https://learn.microsoft.com/en-us/azure/aks/configure-azure-cni).<br>**Note**: To support Azure CNI your Subnet must be large enough to accommodate the nodes, pods, and all Kubernetes and Azure resources that might be provisioned in your cluster.<br>To calculate the minimum subnet size including an additional node for upgrade operations use formula: `(number of nodes + 1) + ((number of nodes + 1) * maximum pods per node that you configure)` <br>Example for a 5 node cluster: `(5) + (5 * 110) = 555 (/22 or larger)`|
-| aks_network_policy | Sets up network policy to be used with Azure CNI. Network policy allows to control the traffic flow between pods. Currently supported values are `calico` and `azure`.| string | `azure`| Network policy can only be used when `aks_network_plugin` is set to `azure`. |
+| aks_network_policy | Sets up network policy to be used with Azure CNI. Network policy allows to control the traffic flow between pods. Currently supported values are `calico` and `azure`.| string | `azure`| Network policy `azure` is only supported for `aks_network_plugin = azure` and network policy `calico` is supported for both `aks_network_plugin` values `azure` and `kubenet`. |
 
 
 The default values for the `subnets` variable are as follows:
@@ -340,6 +340,7 @@ Each server element, like `foo = {}`, can contain none, some, or all of the para
 | administrator_password | The Password associated with the administrator_login for the PostgreSQL Flexible Server | string | "my$up3rS3cretPassw0rd" | The password must contain between 8 and 128 characters and must contain characters from three of the following categories: English uppercase letters, English lowercase letters, numbers (0 through 9), and non-alphanumeric characters (!, $, #, %, etc.). |
 | server_version | The version of the PostgreSQL Flexible server instance | string | "13" | Refer to the [SAS Viya Platform Administration Guide](https://go.documentation.sas.com/doc/en/sasadmincdc/default/itopssr/p05lfgkwib3zxbn1t6nyihexp12n.htm?fromDefault=#p1wq8ouke3c6ixn1la636df9oa1u) for the supported versions of PostgreSQL for the SAS Viya platform. |
 | ssl_enforcement_enabled | Enforce SSL on connection to the Azure Database for PostgreSQL Flexible server instance | bool | true | |
+| connectivity_method | Network connectivity option to connect to your flexible server. There are two connectivity options available: Public access (allowed IP addresses) and Private access (VNet Integration). Defaults to public access with firewall rules enabled.| string | "public" | Valid options are `public` and `private`. See details [here](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-networking) |
 | postgresql_configurations | Sets a PostgreSQL Configuration value on a Azure PostgreSQL Flexible Server | list(object) | [] | More details can be found [here](https://docs.microsoft.com/en-us/azure/postgresql/flexible-server/howto-configure-server-parameters-using-cli) |
 
 Here is a sample of the `postgres_servers` variable with the `default` entry only overriding the `administrator_password` parameter and the `cps` entry overriding all of the parameters:
@@ -358,6 +359,7 @@ postgres_servers = {
     administrator_password       = "1tsAB3aut1fulDay"
     server_version               = "13"
     ssl_enforcement_enabled      = true
+    connectivity_method          = "public"
     postgresql_configurations    = [
        {
          name  = "azure.extensions"

docs/user/PostgreSQLPrivateAccess.md

@@ -0,0 +1,45 @@
+# Azure Database for PostgreSQL Flexible Server with the Private access (VNet Integration)
+
+Azure Database for PostgreSQL Flexible Server is a managed service that you can use to run, manage, and scale highly available PostgreSQL servers in the cloud. Azure Database for PostgreSQL - Flexible Server supports two types of mutually exclusive network connectivity methods to connect to your flexible server. The two options are:
+
+* Public access (allowed IP addresses)
+* Private access (VNet Integration)
+
+In this document, we will focus on PostgreSQL server with Private access (VNet Integration).
+
+You can deploy a flexible server into your Azure virtual network (VNet). Azure virtual networks provide private and secure network communication. Resources in a virtual network can communicate through private IP addresses that were assigned on this network. In Private access, the connections to the PostgreSQL server are restricted to only within your virtual network. To learn more about it, refer to [Private access (VNet Integration)](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-networking#private-access-vnet-integration).
+
+To create PostgreSQL Flexible Server with the private access connectivity method use the example file provided [here](../../examples/sample-input-ppg.tfvars).
+
+# Connect Azure Database for PostgreSQL Flexible Server with the private access connectivity method
+
+Since the PostgreSQL Flexible server is in a virtual network, you can only connect to the server from other Azure services in the same virtual network as the server. The virtual machine must be created in the same region and same subscription. The Linux virtual machine can be used as an SSH tunnel to manage your database server. To connect and manage the server, you can either create a separate Linux virtual machine or use the jump server that was created with your cluster. Below we will see the steps to connect to the jump server and access the PostgreSQL Flexible Server.
+
+## Connect to jump server
+
+Create an SSH connection with the VM using Bash or PowerShell. At your prompt, open an SSH connection to your virtual machine. Replace the IP address with the one from your VM, and replace SSH user's private key used during cluster creation.
+
+```bash
+ssh -i <path_to_jump_svr_private_key> [email protected]
+```
+
+## Install PostgreSQL client tools
+
+You need to install the postgresql-client tool to be able to connect to the server.
+
+```bash
+sudo apt-get update
+sudo apt-get install postgresql-client
+```
+
+Connections to the database are enforced with SSL, hence you need to download the public SSL certificate.
+
+```bash
+wget --no-check-certificate https://dl.cacerts.digicert.com/DigiCertGlobalRootCA.crt.pem
+```
+
+With the psql client tool installed, we can now connect to the server from your local environment.
+
+```bash
+psql --host=mydemoserver-pg.postgres.database.azure.com --port=5432 --username=myadmin --dbname=postgres --set=sslmode=require --set=sslrootcert=DigiCertGlobalRootCA.crt.pem
+```

examples/sample-input-ppg.tfvars

@@ -19,13 +19,67 @@ ssh_public_key              = "~/.ssh/id_rsa.pub"
 # Tags can be specified matching your tagging strategy.
 tags = {} # for example: { "owner|email" = "<you>@<domain>.<com>", "key1" = "value1", "key2" = "value2" }
 
-# Postgres config - By having this entry a database server is created. If you do not
-#                   need an external database server remove the 'postgres_servers'
+# PostgreSQL
+
+# Postgres config - By having this entry a database server is created. 
+#                   Default networking option: Public access (allowed IP addresses) is enabled
+#                   If you do not need an external database server remove the 'postgres_servers'
 #                   block below.
 postgres_servers = {
   default = {},
 }
 
+# To use Private access (VNet Integration) remove the 'postgres_servers' block above and uncomment the blocks below:
+# postgres_servers and subnets
+
+# postgres_servers = {
+#   default = {
+#     connectivity_method = "private"
+#   }
+# }
+#
+# # Subnet for PostgreSQL
+# subnets = {
+#   aks = {
+#     "prefixes" : ["192.168.0.0/23"],
+#     "service_endpoints" : ["Microsoft.Sql"],
+#     "private_endpoint_network_policies_enabled" : true,
+#     "private_link_service_network_policies_enabled" : false,
+#     "service_delegations" : {},
+#   }
+#   misc = {
+#     "prefixes" : ["192.168.2.0/24"],
+#     "service_endpoints" : ["Microsoft.Sql"],
+#     "private_endpoint_network_policies_enabled" : true,
+#     "private_link_service_network_policies_enabled" : false,
+#     "service_delegations" : {},
+#   }
+#   netapp = {
+#     "prefixes" : ["192.168.3.0/24"],
+#     "service_endpoints" : [],
+#     "private_endpoint_network_policies_enabled" : false,
+#     "private_link_service_network_policies_enabled" : false,
+#     "service_delegations" : {
+#       netapp = {
+#         "name" : "Microsoft.Netapp/volumes"
+#         "actions" : ["Microsoft.Network/networkinterfaces/*", "Microsoft.Network/virtualNetworks/subnets/join/action"]
+#       }
+#     }
+#   }
+#   postgresql = {
+#     "prefixes": ["192.168.4.0/24"],
+#     "service_endpoints": ["Microsoft.Sql"],
+#     "private_endpoint_network_policies_enabled": true,
+#     "private_link_service_network_policies_enabled": false,
+#     "service_delegations": {
+#       flexpostgres = {
+#         "name"    : "Microsoft.DBforPostgreSQL/flexibleServers"
+#         "actions" : ["Microsoft.Network/virtualNetworks/subnets/join/action"]
+#       }
+#     }
+#   }
+# }
+
 # Azure Container Registry config
 create_container_registry           = false
 container_registry_sku              = "Standard"

main.tf

@@ -225,6 +225,9 @@ module "flex_postgresql" {
   server_version               = each.value.server_version
   firewall_rule_prefix         = "${var.prefix}-${each.key}-postgres-firewall-"
   firewall_rules               = local.postgres_firewall_rules
+  connectivity_method          = each.value.connectivity_method
+  virtual_network_id           = each.value.connectivity_method == "private" ? module.vnet.id : null
+  delegated_subnet_id          = each.value.connectivity_method == "private" ? module.vnet.subnets["postgresql"].id : null
   postgresql_configurations = each.value.ssl_enforcement_enabled ? concat(each.value.postgresql_configurations, local.default_postgres_configuration) : concat(
   each.value.postgresql_configurations, [{ name : "require_secure_transport", value : "OFF" }], local.default_postgres_configuration)
   tags = var.tags

modules/azure_aks/main.tf

@@ -20,7 +20,8 @@ resource "azurerm_kubernetes_cluster" "aks" {
 
   network_profile {
     network_plugin = var.aks_network_plugin
-    network_policy = var.aks_network_plugin == "azure" ? var.aks_network_policy : null
+    network_policy = var.aks_network_plugin == "kubenet" && var.aks_network_policy == "azure" ? null : var.aks_network_policy
+
     # Docs on AKS Advanced Networking config
     # https://docs.microsoft.com/en-us/azure/architecture/aws-professional/networking
     # https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-vnet-plan-design-arm

modules/azurerm_postgresql_flex/main.tf

@@ -5,7 +5,24 @@
 ### Managed PostgreSQL Flexible server on Azure ###
 ###################################################
 
-# https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/postgresql_flexible_server
+## https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/postgresql_flexible_server
+
+resource "azurerm_private_dns_zone" "flexpsql" {
+  count = var.connectivity_method == "private" ? 1 : 0
+
+  name                = "${var.server_name}.postgres.database.azure.com"
+  resource_group_name = var.resource_group_name
+}
+
+resource "azurerm_private_dns_zone_virtual_network_link" "flexpsql" {
+  count = var.connectivity_method == "private" ? 1 : 0
+
+  name                  = var.server_name
+  private_dns_zone_name = azurerm_private_dns_zone.flexpsql.0.name
+  virtual_network_id    = var.virtual_network_id
+  resource_group_name   = var.resource_group_name
+}
+
 resource "azurerm_postgresql_flexible_server" "flexpsql" {
   name                         = "${var.server_name}-flexpsql"
   location                     = var.location
@@ -18,12 +35,16 @@ resource "azurerm_postgresql_flexible_server" "flexpsql" {
   administrator_password       = var.administrator_password
   version                      = var.server_version
   tags                         = var.tags
+  delegated_subnet_id          = var.delegated_subnet_id
+  private_dns_zone_id          = try(azurerm_private_dns_zone.flexpsql.0.id, null)
+
+  depends_on = [azurerm_private_dns_zone_virtual_network_link.flexpsql]
 
   lifecycle {
-    ignore_changes = [ 
+    ignore_changes = [
       # Ignore changes to zone on updates after intial creation
       zone
-    ]  
+    ]
   }
 }
 
@@ -39,8 +60,8 @@ resource "azurerm_postgresql_flexible_server_configuration" "flexpsql" {
 }
 
 resource "azurerm_postgresql_flexible_server_firewall_rule" "flexpsql" {
-  count = var.public_network_access_enabled ? length(var.firewall_rules) : 0
-  
+  count = var.connectivity_method == "public" ? length(var.firewall_rules) : 0
+
   name             = format("%s%s", var.firewall_rule_prefix, lookup(var.firewall_rules[count.index], "name", count.index))
   server_id        = azurerm_postgresql_flexible_server.flexpsql.id
   start_ip_address = var.firewall_rules[count.index]["start_ip"]
@@ -49,7 +70,7 @@ resource "azurerm_postgresql_flexible_server_firewall_rule" "flexpsql" {
 
 # NOTE: This firewall rule enables the flag - "Allow public access from any Azure service within Azure to this server"
 resource "azurerm_postgresql_flexible_server_firewall_rule" "azure_public" {
-  count = var.public_network_access_enabled ? 1 : 0
+  count = var.connectivity_method == "public" ? 1 : 0
 
   name             = "Allow-public-access-from-any-Azure-service"
   server_id        = azurerm_postgresql_flexible_server.flexpsql.id

modules/azurerm_postgresql_flex/variables.tf

@@ -56,10 +56,10 @@ variable "server_version" {
   default     = "13"
 }
 
-variable "public_network_access_enabled" {
-  description = "Whether or not public network access is allowed for this server. Defaults to true"
-  type        = bool
-  default     = true
+variable "connectivity_method" {
+  description = "Network connectivity options to connect to your flexible server. Valid options are 'public' and 'private'. Defaults to public"
+  type        = string
+  default     = "public"
 }
 
 variable "firewall_rule_prefix" {
@@ -88,3 +88,19 @@ variable "postgresql_configurations" {
   }))
   default = []
 }
+
+variable "virtual_network_id" {
+  description = "The ID of the Virtual Network that should be linked to the DNS Zone. Changing this forces a new resource to be created."
+  type        = string
+}
+
+variable "delegated_subnet_id" {
+  description = "The ID of the virtual network subnet to create the PostgreSQL Flexible Server. The provided subnet should not have any other resource deployed in it and this subnet will be delegated to the PostgreSQL Flexible Server, if not already delegated. Changing this forces a new PostgreSQL Flexible Server to be created."
+  type        = string
+}
+
+variable "private_dns_zone_id" {
+  description = "The ID of the private DNS zone to create the PostgreSQL Flexible Server. Changing this forces a new PostgreSQL Flexible Server to be created."
+  type        = string
+  default     = null
+}

variables.tf

@@ -268,6 +268,7 @@ variable "postgres_server_defaults" {
     administrator_password       = "my$up3rS3cretPassw0rd"
     server_version               = "13"
     ssl_enforcement_enabled      = true
+    connectivity_method          = "public"
     postgresql_configurations    = []
   }
 }

versions.tf

@@ -4,7 +4,7 @@
 terraform {
 
   required_version = ">= 1.0.0"
-  
+
   required_providers {
     azurerm = {
       source  = "hashicorp/azurerm"