Skip to content

Repository files navigation

terraform-google-dr-infra

Terraform module to create Google Cloud infrastructure resources required to run DataRobot.

Usage

module "datarobot_infra" {
  source = "datarobot-oss/dr-infra/google"

  name              = "datarobot"
  google_project_id = "your-google-project-id"
  region            = "us-west1"
  domain_name       = "datarobot.example.com"

  cert_manager_letsencrypt_email_address = "you@example.com"

  tags = {
    environment = "dev"
    managed-by  = "terraform"
  }
}

Examples

  • Complete - Demonstrates all available input variables.
  • Public - Minimal configuration for a publicly accessible deployment (internet-facing load balancer, public GKE API endpoint).
  • Private - Minimal configuration for a private deployment (internal load balancer, private-only GKE API endpoint, existing VPC).

Using an example directly from source

  1. Clone the repo
git clone https://github.com/datarobot-oss/terraform-google-dr-infra.git
  1. Change directories into the example that best aligns with your use-case.
cd terraform-google-dr-infra/examples/public
  1. Modify main.tf to suit your specific use-case.
  2. Run terraform.
terraform init
terraform apply

Module Descriptions

Network

Toggle

  • create_network to create a new Google VPC
  • existing_vpc_name, existing_kubernetes_nodes_subnet, and existing_kubernetes_pods_range_name to use an existing VPC and subnet

Description

Create a new Google VPC with one subnet using a /20 slice of network_address_space and a NAT gateway attached.

kubernetes_pod_cidr and kubernetes_service_cidr are secondary ranges within the subnet which will be used for the Kubernetes pod and service IPs, respectively.

Only the priamry the kubernetes_pod_cidr IPs are attached to the Cloud NAT gateway.

Permissions

TBD

DNS

Toggle

  • create_dns_zone to create a new Google Cloud DNS managed zone
  • existing_dns_zone_name to use an existing Google Cloud DNS managed zone

Description

Creates a new Cloud DNS managed zone for domain_name. When dns_zone_public is true (the default) a public zone is created; when false a private zone is created for the given VPC.

The Cloud DNS zone is used by external_dns to manage DNS records for the resources created by the DataRobot application. When the zone is public it is also used for DNS validation when using cert_manager and cert_manager_letsencrypt_clusterissuers.

Permissions

TBD

Storage

Toggle

  • create_storage to create a new Google Cloud Storage Bucket
  • existing_gcs_bucket_name to use an existing Google Cloud Storage Bucket

Description

Create a new GCS Bucket with prefix name and name datarobot.

The DataRobot application will use this storage account for persistent file storage.

Permissions

TBD

Container Registry

Toggle

  • create_container_registry to create a new Google Artifact Registry Repository
  • existing_artifact_registry_repo_id to use an existing Google Artifact Registry Repository

Description

Create a new GAR repository with name name.

The DataRobot application will use this registry to host custom images created by various services.

Permissions

TBD

Kubernetes

Toggle

  • create_kubernetes_cluster to create a new Google Kubernetes Engine Cluster
  • existing_gke_cluster_name to use an existing GKE cluster

Description

Create a new GKE cluster to host the DataRobot application and any other helm charts installed by this module.

By default, the Kubernetes cluster API endpoint is accessible both via a private endpoint created within the same VPC as well as publicly over the internet. GKE nodes always communicate with the control plane using the private IP address. Public endpoint access can be restricted using the kubernetes_cluster_endpoint_access_list variable or disabled completely by setting kubernetes_cluster_endpoint_public_access to false.

When kubernetes_cluster_endpoint_public_access is false, Kubernetes management operations such as kubectl and helm commands (including the Helm chart installs performed by this Terraform module) must be run from a host which can access the Kubernetes cluster API private endpoint. By default, any host within the GKE nodes subnet has access but this can be extended using the kubernetes_cluster_endpoint_access_list variable. This can be helpful when running this Terraform module from a host that resides within the same VPC as the GKE cluster but in a different subnet than the GKE nodes.

Two node groups are created:

  • A drcpu node group intended to host the majority of the DataRobot pods
  • A drgpu node group intended to host GPU workload pods containing the label datarobot.com/node-capability: gpu and taint nvidia.com/gpu:NoSchedule

By default, slices of network_address_space will be used for the cluster nodes and control plane private endpoint IPs. It is best to use a separate address space for kubernetes_pod_cidr as it is a secondary (aliased) range.

Permissions

TBD

App Identity

Toggle

  • create_app_identity to create a new Google Service account to represent the DataRobot application

Description

Create a new GKE Service Account with roles/storage.admin access to the Google Cloud Storage bucket and roles/artifactregistry.writer access to the Google Artifact Registry Repository.

Workload identities are created for each datarobot_service_accounts within the datarobot_namespace and attached to this Service Account. This allows those pods running with those service accounts to access file storage and the artifact registry.

Permissions

TBD

Postgres

Toggle

  • create_postgres to create a new Cloud SQL for PostgreSQL instance

Description

Create an Cloud SQL for PostgreSQL connected to via service networking.

Permissions

TBD

Redis

Toggle

  • create_redis to create a new Memorystore Redis instance

Description

Create a Memorystore Redis instance connected to via service networking.

Permissions

TBD

MongoDB

Toggle

  • create_mongodb to create a new MongoDB Atlas cluster

Description

Create a MongoDB Atlas project and cluster for use by the DataRobot application.

Permissions

TBD

Helm Chart - ingress-nginx

Toggle

  • ingress_nginx to install the ingress-nginx helm chart

Description

Uses the terraform-helm-release module to install the https://kubernetes.github.io/ingress-nginx/ingress-nginx helm chart into the ingress-nginx namespace.

The ingress-nginx helm chart will trigger the deployment of an Google Network Load Balancer directing traffic to the ingress-nginx-controller Kubernetes services.

Values passed to the helm chart can be overridden by passing a custom values file via the ingress_nginx_values variable as demonstrated in the complete example.

Permissions

Not required

Helm Chart - cert-manager

Toggle

  • cert_manager to install the cert-manager helm chart

Description

Uses the terraform-helm-release module to install the https://charts.jetstack.io/cert-manager helm chart into the cert-manager namespace.

A Google Service Account is created for the cert-manager Kubernetes service account running in the cert-manager namespace that allows the creation of DNS resources within the specified DNS zone.

cert-manager can be used by the DataRobot application to create and manage various certificates including the application.

When cert_manager_letsencrypt_clusterissuers is enabled, letsencrypt-staging and letsencrypt-prod ClusterIssuers will be created which can be used by the datarobot-google umbrella chart to issue certificates used by the DataRobot application. The default values in that helm chart (as of version 10.2) have global.ingress.tls.enabled, global.ingress.tls.certmanager, and global.ingress.tls.issuer as letsencrypt-prod which will use the letsencrypt-prod ClusterIssuer to issue a public ACME certificate as the TLS certificate used by the Kubernetes ingress resources.

Values passed to the helm chart can be overridden by passing a custom values file via the cert_manager_values variable as demonstrated in the complete example.

Permissions

TBD

Helm Chart - external-dns

Toggle

  • external_dns to install the external-dns helm chart

Description

Uses the terraform-helm-release module to install the https://charts.bitnami.com/bitnami/external-dns helm chart into the external-dns namespace.

A Google Service Account is created for the external-dns Kubernetes service account running in the external-dns namespace that allows the creation of DNS resources within the specified DNS zone.

external-dns is used to automatically create DNS records for ingress resources in the Kubernetes cluster. When the DataRobot application is installed and the ingress resources are created, external-dns will automatically create a DNS record pointing at the ingress resource.

Values passed to the helm chart can be overridden by passing a custom values file via the external_dns_values variable as demonstrated in the complete example.

Permissions

TBD

Helm Chart - nvidia-device-plugin

Toggle

  • nvidia_device_plugin to install the nvidia-device-plugin helm chart

Description

Uses the terraform-helm-release module to install the https://nvidia.github.io/k8s-device-plugin/nvidia-device-plugin helm chart into the nvidia-device-plugin namespace.

Values passed to the helm chart can be overridden by passing a custom values file via the nvidia_device_plugin_values variable as demonstrated in the complete example.

Permissions

Not required

Helm Chart - descheduler

Toggle

  • descheduler to install the descheduler helm chart

Description

Uses the terraform-helm-release module to install the descheduler helm chart from the https://kubernetes-sigs.github.io/descheduler/ helm repo into the kube-system namespace.

This helm chart allows for automatic rescheduling of pods for optimizing resource consumption.

Permissions

Not required

Comprehensive Required Permissions

TBD

DataRobot versions

Currently the only thing coupling a release of this module to a DataRobot Enterprise Release is the default list of datarobot_service_accounts. Technically, this module can be used with any DataRobot version if the user specifies the correct list of datarobot_service_accounts for that version.

The default installation supports DataRobot versions >= 10.0.

Requirements

Name Version
terraform >= 1.3.5
google >= 6.6.0
helm >= 3.0.2
kubectl ~> 1.19
mongodbatlas ~> 2.0

Providers

Name Version
google >= 6.6.0

Modules

Name Source Version
app_identity terraform-google-modules/service-accounts/google ~> 4.0
cert_manager ./modules/cert-manager n/a
cloud_router terraform-google-modules/cloud-router/google ~> 6.1
custom_endpoints ./modules/custom-private-endpoints n/a
descheduler ./modules/descheduler n/a
dns terraform-google-modules/cloud-dns/google ~> 5.0
external_dns ./modules/external-dns n/a
ingress_nginx ./modules/ingress-nginx n/a
kubernetes terraform-google-modules/kubernetes-engine/google//modules/private-cluster ~> 39.0
mongodb ./modules/mongodb n/a
network terraform-google-modules/network/google ~> 9.0
nvidia_device_plugin ./modules/nvidia-device-plugin n/a
observability ./modules/observability n/a
postgres terraform-google-modules/sql-db/google//modules/postgresql ~> 26.0
private_link_service ./modules/private-link-service n/a
redis terraform-google-modules/memorystore/google ~> 15.0
storage terraform-google-modules/cloud-storage/google ~> 8.0

Resources

Name Type
google_artifact_registry_repository.this resource
google_artifact_registry_repository_iam_member.datarobot resource
google_compute_global_address.postgres resource
google_compute_global_address.redis resource
google_service_account_iam_member.datarobot resource
google_service_networking_connection.this resource
google_storage_bucket_iam_member.datarobot resource
google_client_config.default data source
google_compute_network.existing data source
google_compute_subnetwork.existing_ingress_psc data source
google_compute_subnetwork.existing_kubernetes_nodes data source
google_compute_subnetwork.existing_mongodb data source
google_container_cluster.existing data source
google_dns_managed_zone.existing data source

Inputs

Name Description Type Default Required
allow_psc_global_access Whether to allow global access for Private Service Connect bool false no
cert_manager Install the cert-manager helm chart. All other cert_manager variables are ignored if this variable is false. bool true no
cert_manager_letsencrypt_clusterissuers Whether to create letsencrypt-prod and letsencrypt-staging ClusterIssuers. This will only work if the DNS zone is public. bool true no
cert_manager_letsencrypt_email_address Email address for the certificate owner. Let's Encrypt will use this to contact you about expiring certificates, and issues related to your account. Only required if cert_manager_letsencrypt_clusterissuers is true. string "user@example.com" no
cert_manager_values_overrides Values in raw yaml format to pass to helm. string null no
create_app_identity Create a new user assigned identity for the DataRobot application bool true no
create_container_registry Create a new Google Container Registry. Ignored if an existing existing_artifact_registry_repo_id is specified. bool true no
create_dns_zone Create a Cloud DNS managed zone for domain_name. Ignored if existing_dns_zone_name is specified. bool true no
create_ingress_psc Expose the internal LB created by the ingress-nginx controller as a Google Private Service Connection. Only applies if internet_facing_ingress_lb is false. bool false no
create_ingress_psc_namespace Whether to create a namespace for the ingress private service connect bool false no
create_kubernetes_cluster Create a new Google Kubernetes Engine cluster. All kubernetes and helm chart variables are ignored if this variable is false. bool true no
create_mongodb Whether to create a MongoDB Atlas instance bool false no
create_network Create a new Google VPC. Ignored if an existing existing_vpc_id is specified. bool true no
create_observability Whether to provision observability resources bool false no
create_postgres Whether to create a CloudSQL for PostgreSQL instance bool false no
create_redis Whether to create a Google Memorystore Redis instance bool false no
create_storage Create a new Google Storage Bucket to use for DataRobot file storage. Ignored if an existing_gcs_bucket_name is specified. bool true no
custom_private_endpoints Configuration for the specific endpoint
list(object({
service_name = string
private_dns_zone = optional(string, "")
private_dns_name = optional(string, "")
}))
[] no
datarobot_namespace Kubernetes namespace in which the DataRobot application will be installed string "dr-app" no
datarobot_service_accounts Kubernetes service accounts in the datarobot_namespace to provide with Storage Blob Data Contributor and AcrPush access set(string)
[
"datarobot-storage-sa",
"dynamic-worker",
"kubeworker-sa",
"prediction-server-sa",
"internal-api-sa",
"build-service",
"tileservergl-sa",
"nbx-notebook-revisions-account",
"buzok-account",
"exec-manager-qw",
"exec-manager-wrangling",
"lrs-job-manager",
"blob-view-service"
]
no
descheduler Install the descheduler helm chart to enable rescheduling of pods. All other descheduler variables are ignored if this variable is false bool true no
descheduler_values_overrides Values in raw yaml format to pass to helm. string null no
dns_zone_force_destroy Force destroy the Cloud DNS managed zone. Ignored if an existing_dns_zone_name is specified or create_dns_zone is false. bool false no
dns_zone_public Create a public Cloud DNS managed zone. When false, a private zone will be created for the given VPC. bool true no
domain_name Name of the domain to use for the DataRobot application. If create_dns_zone is true then a zone will be created for this domain. It is also used by the cert-manager helm chart for DNS validation and as a domain filter by the external-dns helm chart. string "" no
existing_artifact_registry_repo_id ID of existing artifact registry repository to use string null no
existing_dns_zone_name Name of an existing Cloud DNS managed zone to use. When specified, all other DNS variables will be ignored. string null no
existing_gcs_bucket_name ID of existing Google Storage Bucket to use for DataRobot file storage. When specified, all other storage variables will be ignored. string null no
existing_gke_cluster_name Name of existing GKE cluster to use. When specified, all other kubernetes variables will be ignored. string null no
existing_ingress_pcs_subnet_name Name of an existing subnet to use for the Private Service Connection used by the ingress-nginx controller. Required when an existing_vpc_name is specified and create_ingress_psc is true. Ignored if no existing_vpc_name is specified or create_ingress_psc is false. string null no
existing_kubernetes_nodes_subnet Name of an existing subnet to use for the GKE node pools and control plane private endpoint. Required when create_kubernetes_cluster is true and an existing_vpc_name is specified. Ignored if no existing_vpc_name is specified. string null no
existing_kubernetes_pods_range_name Name of an secondary IP range within subnet defined by existing_kubernetes_nodes_subnet_name to use for the Kubernetes pods. Required when an existing_vpc_name is specified. Ignored if no existing_vpc_name is specified. string null no
existing_mongodb_subnet_name Name of an existing subnet to use for MongoDB Atlas VPC Peering. Required when an existing_vpc_name is specified. Ignored if no existing_vpc_name is specified. string null no
existing_vpc_name Name of an existing Google VPC to use. When specified, other network variables are ignored. string null no
external_dns Install the external_dns helm chart to manage DNS records for resources created by the application. All other external_dns variables are ignored if this variable is false. bool true no
external_dns_values_overrides Values in raw yaml format to pass to helm. string null no
gcr_registry_name Name of the Artifact Registry repository. Defaults to the name if not specified. string null no
google_project_id The ID of the Google Project where these resources will be created string n/a yes
ingress_nginx Install the ingress-nginx helm chart to use as the ingress controller for the GKE cluster. All other ingress_nginx variables are ignored if this variable is false. bool true no
ingress_nginx_values_overrides Values in raw yaml format to pass to helm. string null no
ingress_psc_consumer_projects The list of consumer project IDs that are allowed to connect to the ServiceAttachment. This field can only be used when connectionPreference is ACCEPT_MANUAL. list(string) [] no
ingress_psc_subnet_cidr CIDR range to use for the Private Service Connection used by the ingress-nginx controller. Only used when create_network is true and an existing_vpc_name is not specified. string null no
ingress_service_name The name of the ingress service to attach the private link to. string "ingress-nginx-controller" no
install_helm_charts Whether to install helm charts into the target EKS cluster. All other helm chart variables are ignored if this is false. bool true no
internet_facing_ingress_lb Determines the type of Load Balancer created for GKE ingress. If true, an external Load Balancer will be created. If false, an internal Load Balancer will be created. bool true no
kubernetes_cluster_deletion_protection Enable deletion protection on the GKE cluster bool true no
kubernetes_cluster_endpoint_access_list List of CIDRs allowed to access the Kubernetes cluster API endpoint. When kubernetes_cluster_endpoint_public_access is true, these CIDRs specify which public IP addresses are allowed to access the Kubernetes cluster API external endpoint. When kubernetes_cluster_endpoint_public_access is false, these CIDRs specify which private IP addresses are allowed to access the Kubernetes cluster API internal endpoint. By default, only hosts within the kubernetes nodes subnet are allowed to access the Kubernetes cluster API internal endpoint. list(string) [] no
kubernetes_cluster_endpoint_public_access Whether the Kubernetes cluster API endpoint can be accessed via an external IP address bool true no
kubernetes_cluster_grant_registry_access Grants created cluster-specific service account storage.objectViewer and artifactregistry.reader roles bool true no
kubernetes_cluster_version GKE cluster version string "latest" no
kubernetes_master_ipv4_cidr The IP range in CIDR notation to use for the hosted master network including the Kubernetes control plane. If you use this flag, GKE creates a new subnet that uses the values you defined in master-ipv4-cidr and uses the new subnet to provision the internal IP address for the control plane. string null no
kubernetes_node_pools Map of GKE node pools any
{
"drcpu": {
"disk_size_gb": 200,
"machine_type": "e2-standard-32",
"max_count": 10,
"min_count": 1,
"name": "drcpu",
"node_count": 1,
"node_labels": {
"datarobot.com/node-capability": "cpu"
},
"node_taints": []
},
"drgpu": {
"max_count": 10,
"min_count": 0,
"name": "drgpu",
"node_count": 0,
"node_labels": {
"datarobot.com/node-capability": "gpu"
},
"node_taints": [
{
"effect": "NO_SCHEDULE",
"key": "nvidia.com/gpu",
"value": "true"
}
],
"vm_size": "n1-highmem-4"
}
}
no
kubernetes_nodes_cidr The CIDR to use for Kubernetes nodes IP addresses. This is used as the primary IP range for the Kubernetes nodes subnet. string null no
kubernetes_pod_cidr The CIDR to use for Kubernetes pod IP addresses. This is used as a secondary IP range within the Kubernetes nodes subnet. string "172.16.0.0/15" no
mongodb_admin_username MongoDB admin username string "pcs-mongodb" no
mongodb_atlas_auto_scaling_disk_gb_enabled Enable Atlas disk size autoscaling bool true no
mongodb_atlas_disk_size Starting atlas disk size string "20" no
mongodb_atlas_instance_type atlas instance type string "M30" no
mongodb_atlas_org_id Atlas organization ID string null no
mongodb_atlas_private_key Private API key for Mongo Atlas string "" no
mongodb_atlas_public_key Public API key for Mongo Atlas string "" no
mongodb_audit_enable Enable database auditing for production instances only(cost incurred 10%) bool false no
mongodb_backup_schedule Configuration for the MongoDB Atlas cloud backup schedule policy items and cross-region copy settings
object({
policy_item_hourly = optional(object({
frequency_interval = optional(number, 6)
retention_unit = optional(string, "days")
retention_value = optional(number, 7)
}), {})
policy_item_daily = optional(object({
frequency_interval = optional(number, 1)
retention_unit = optional(string, "days")
retention_value = optional(number, 30)
}), {})
policy_item_weekly = optional(object({
frequency_interval = optional(number, 6)
retention_unit = optional(string, "days")
retention_value = optional(number, 30)
}), {})
policy_item_monthly = optional(object({
frequency_interval = optional(number, 1)
retention_unit = optional(string, "months")
retention_value = optional(number, 1)
}), {})
copy_settings = optional(object({
enabled = optional(bool, true)
cloud_provider = optional(string, "GCP")
frequencies = optional(list(string), ["DAILY"])
should_copy_oplogs = optional(bool, false)
}), {})
})
{} no
mongodb_enable_slack_alerts Enable alert notifications to a Slack channel. When true, slack_api_token and slack_notification_channel must be set. string false no
mongodb_network_reservation_ip_offset Value to offset the network reservation IP number 2 no
mongodb_slack_api_token Slack API token to use for alert notifications. Required when enable_slack_alerts is true. string null no
mongodb_slack_notification_channel Slack channel to send alert notifications to. Required when enable_slack_alerts is true. string null no
mongodb_subnet_cidr CIDR range to use for MongoDB Atlas VPC Peering. Only used when create_network is true and an existing_vpc_name is not specified. string null no
mongodb_termination_protection_enabled Enable protection to avoid accidental production cluster termination bool false no
mongodb_version MongoDB version string "7.0" no
name Name to use as a prefix for created resources string n/a yes
network_address_space The CIDR to use for the VPC and subnets created by this module string "10.0.0.0/20" no
nvidia_device_plugin Install the nvidia-device-plugin helm chart to expose node GPU resources to the GKE cluster. All other nvidia_device_plugin variables are ignored if this variable is false. bool true no
nvidia_device_plugin_values_overrides Values in raw yaml format to pass to helm. string null no
postgres_availability_type The availability type of the Cloud SQL instance, high availability (REGIONAL) or single zone (ZONAL) string "REGIONAL" no
postgres_cidr CIDR range to use for PostgreSQL private IP string null no
postgres_database_flags The database flags for the Cloud SQL instance.
list(object({
name = string
value = string
}))
[
{
"name": "password_encryption",
"value": "scram-sha-256"
}
]
no
postgres_database_version The PostgreSQL version to use string "POSTGRES_13" no
postgres_deletion_protection Whether Terraform will be prevented from destroying the instance. When the field is set to true or unset in Terraform state, a terraform apply or terraform destroy that would delete the instance will fail. When the field is set to false, deleting the instance is allowed bool false no
postgres_disk_autoresize_limit The maximum size to which storage capacity can be automatically increased. The default value is 0, which specifies that there is no limit. number 0 no
postgres_disk_size The size of data disk, in GB. Size of a running instance cannot be reduced but can be increased number 20 no
postgres_disk_type The type of data disk: PD_SSD, PD_HDD, or HYPERDISK_BALANCED string "PD_SSD" no
postgres_tier The machine type to use. See tiers for more details and supported versions. Postgres supports only shared-core machine types, and custom machine types such as db-custom-2-13312. string "db-custom-4-16384" no
redis_cidr CIDR range to use for Redis private IP string null no
redis_memory_size_gb Redis memory size in GiB. Defaulted to 1 GiB number 8 no
redis_tier The service tier of the instance: BASIC or STANDARD_HA string "STANDARD_HA" no
redis_transit_encryption_mode The TLS mode of the Redis instance, If not provided, TLS is enabled for the instance. Possible values are: SERVER_AUTHENTICATION, DISABLED. string "SERVER_AUTHENTICATION" no
region Google region to create the resources in string n/a yes
release_channel The release channel of this cluster. Accepted values are UNSPECIFIED, RAPID, REGULAR and STABLE. Defaults to STABLE. string "STABLE" no
storage_force_destroy Force destroy the Google Storage Bucket when terminating, deleting all objects it contains. Ignored if an existing_gcs_bucket_name is specified or create_storage is false. bool false no
tags A map of tags to add to all created resources map(string)
{
"managed-by": "terraform"
}
no

Outputs

Name Description
artifact_registry_repo_id ID of the Artifact Registry repository
artifact_registry_repo_path Path to the Artifact Registry repository
datarobot_service_account_email Email of the DataRobot service account
datarobot_service_account_key DataRobot service account key
dns_zone_name Name of the DNS zone
dns_zone_name_servers Name servers of the DNS zone
gke_cluster_name Name of the GKE cluster
mongodb_endpoint MongoDB endpoint
mongodb_password MongoDB admin password
postgres_endpoint PostgreSQL endpoint
postgres_password PostgreSQL admin password
redis_endpoint Google Memorystore Redis endpoint
redis_password Google Memorystore Redis instance primary access key
redis_port Google Memorystore Redis port
storage_bucket_name Name of the storage bucket
vpc_name Name of the VPC

About

No description or website provided.

Topics

Resources

Code of conduct

Contributing

Stars

3 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages