[Part 1/3] Upgrade terraform from 0.11.14 to 1.2.5

Why:
- terraform rotated their signing [certificate](https://discuss.hashicorp.com/t/terraform-updates-for-hcsec-2021-12/23570). As a result `terraform init` fails. Instead, a user will need to use `terraform init -verify-plugins=false` if they want to maintain using 0.11.14. This is not safe. Version 0.11.15 has the new certificate. But the other reasons prevented from using that.
- The providers versions are not pinned. As a result, it pulls the latest that it can. Not sure if it was the one used initially.
- Terraform 0.11.x is end of life [1](https://discuss.hashicorp.com/t/when-specifically-is-terraform-0-11-deprecated/3996/4) [2](https://www.hashicorp.com/blog/deprecating-terraform-0-11-support-in-terraform-providers)
- Resources used previously were in the google-beta provider. Now they
  are in the google stable provider.

Changes:
- Add .terraform.lock.hcl. This file is a lock file for terraform
  providers that this repo depends on. Only available in newer versions
of terraform
- Syntax changes. Version 0.12 and onwards has a different syntax than
  0.11. Example:
  - `ports = "${local.forwarded_ports}` -> `ports =
    local.forwarded_ports`
  - These changes occured automatically leveraging [terraform
    0.12upgrade](https://www.terraform.io/language/upgrade-guides/0-12#upgrading-terraform-configuration)
- various `versions.tf` files. These were generated when running
  [terraform
0.13upgrade](https://www.terraform.io/language/upgrade-guides/0-13#explicit-provider-source-locations)
- compute.tf: This file was added by me. Previously, this repo used two
  external modules: github.com/dcaba/terraform-google-managed-instance-group and github.com/ecosystem-infra/terraform-google-multi-port-managed-instance-group. However, these modules are not compatible with the new versions of terraform. One repo is archived and the other has not been touched in awhile. Those modules are using beta features which are now in the main google provider now. We can now just use the out of the box Google provider to build the same infrastructure.
- Upgrade the terraform-google-modules/container-vm/google module
  reference to latest.
- Removed references to the `google-beta` provider since it does not
  exist anymore
- Added placeholder "cos_image_name" which pins the OS image currently
  used. Otherwise, it will pick the latest and cause a difference
  to be detected. This will be removed in Part 3.

How were these changes tested:

Along with the changes in Part 2/3, running `terraform plan` yielded no
changes in the infrastructure after the terraform upgrade

Outstanding changes:
- terraform.tfstate has changed the format over the versions. In order
  to make this PR readable, I separated that change into Part 2/3

Author: jcscottiii

.github/workflows/lint.yml

@@ -21,6 +21,6 @@ jobs:
     - name: Set up Terraform
       uses: hashicorp/setup-terraform@v1
       with:
-        terraform_version: '0.11.14'
+        terraform_version: '1.2.5'
     - name: terraform
       run: terraform fmt --check

.terraform.lock.hcl

@@ -139,7 +139,7 @@ Requirements:
 
 - [Python 3](https://python.org)
 - [Pipenv](https://pipenv.pypa.io/)
-- [Terraform](https://www.terraform.io/) version 0.11.14
+- [Terraform](https://www.terraform.io/) version 1.2.5
 
 The following commands will run the lints:
 
@@ -153,7 +153,7 @@ Requirements:
 
 - [Docker](https://www.docker.com/)
 - [GNU Make](https://www.gnu.org/software/make/)
-- [Terraform](https://www.terraform.io/) version 0.11.14
+- [Terraform](https://www.terraform.io/) version 1.2.5
 - [Python 3](https://python.org)
 - access credentials to the Google Cloud Platform project, saved to a file named
   `google-cloud-platform-credentials.json` in the root pf this repository

README.md

@@ -1,11 +1,11 @@
 variable "registry" {
   description = "Host name of the Docker registry from which the image identifier should be retirieved"
-  type        = "string"
+  type        = string
 }
 
 variable "image" {
   description = "Name of the Docker image whose identifier should be retrieved"
-  type        = "string"
+  type        = string
 }
 
 output "identifier" {
@@ -17,8 +17,8 @@ data "external" "image" {
     "python3",
     "${path.module}/latest-image.py",
     "--registry",
-    "${var.registry}",
+    var.registry,
     "--image",
-    "${var.image}",
+    var.image,
   ]
 }

infrastructure/docker-image/main.tf

@@ -0,0 +1,9 @@
+
+terraform {
+  required_version = "~> 1.2.5"
+  required_providers {
+    external = {
+      source = "hashicorp/external"
+    }
+  }
+}

infrastructure/docker-image/versions.tf

@@ -0,0 +1,291 @@
+# Contains the configurations for the Compute Engine section of Google Cloud
+# These configurations come from modules that are now archived:
+# - Cert Renewer used: github.com/dcaba/terraform-google-managed-instance-group
+# - WPT Server used: github.com/ecosystem-infra/terraform-google-multi-port-managed-instance-group
+# Most hardcoded defaults come from those aforementioned modules.
+
+########################################
+# WPT Server
+# These configurations come from: github.com/ecosystem-infra/terraform-google-multi-port-managed-instance-group
+# More information about how it was used previously: https://github.com/web-platform-tests/wpt.live/blob/67dc5976ccce2e64483f2028a35659d4d6e58891/infrastructure/web-platform-tests/main.tf#L69-L137
+########################################
+
+resource "google_compute_health_check" "wpt_health_check" {
+  name = "${var.name}-wpt-servers"
+
+  check_interval_sec  = 10
+  timeout_sec         = 10
+  healthy_threshold   = 3
+  unhealthy_threshold = 6
+
+  https_health_check {
+    port = "443"
+    # A query parameter is used to distinguish the health check in the server's
+    # request logs.
+    request_path = "/?gcp-health-check"
+  }
+}
+
+resource "google_compute_instance_group_manager" "wpt_servers" {
+  name               = "${var.name}-wpt-servers"
+  zone               = var.zone
+  description        = "compute VM Instance Group"
+  wait_for_instances = false
+  base_instance_name = "${var.name}-wpt-servers"
+  version {
+    name              = "${var.name}-wpt-servers-default"
+    instance_template = google_compute_instance_template.wpt_server.self_link
+  }
+  update_policy {
+    type                  = local.update_policy.type
+    minimal_action        = local.update_policy.minimal_action
+    max_unavailable_fixed = local.update_policy.max_unavailable_fixed
+  }
+  target_pools = [google_compute_target_pool.default.self_link]
+  target_size  = 2
+
+  dynamic "named_port" {
+    for_each = var.wpt_server_ports
+    content {
+      name = named_port.value["name"]
+      port = named_port.value["port"]
+    }
+  }
+
+  auto_healing_policies {
+    health_check      = google_compute_health_check.wpt_health_check.self_link
+    initial_delay_sec = 30
+  }
+}
+
+resource "google_compute_firewall" "wpt-server-mig-health-check" {
+  name    = "${var.name}-wpt-servers-vm-hc"
+  network = var.network_name
+
+  allow {
+    protocol = "tcp"
+    # https port
+    ports = [var.wpt_server_ports[2].port]
+  }
+
+  # This range comes from this module that was used previously:
+  # https://github.com/Ecosystem-Infra/terraform-google-multi-port-managed-instance-group/blob/master/main.tf#L347
+  source_ranges = ["130.211.0.0/22", "35.191.0.0/16"]
+  target_tags   = ["${var.name}-allow"]
+}
+
+resource "google_compute_firewall" "wpt-servers-default-ssh" {
+  name    = "${var.name}-wpt-servers-vm-ssh"
+  network = var.network_name
+
+  allow {
+    protocol = "tcp"
+    ports    = ["22"]
+  }
+
+  source_ranges = ["0.0.0.0/0"]
+  target_tags   = ["allow-ssh"]
+}
+
+resource "google_compute_instance_template" "wpt_server" {
+  name_prefix = "default-"
+
+  tags = ["allow-ssh", "${var.name}-allow"]
+
+  # As of 2020-06-17, we were running into OOM issues with the 1.7 GB
+  # "g1-small" instance[1]. This was suspected to be due to 'git gc' needing
+  # more memory, so we upgraded to "e2-medium" (4 GB of RAM).
+  #
+  # [1] https://github.com/web-platform-tests/wpt.live/issues/30
+  machine_type = "e2-medium"
+
+  # The "google-logging-enabled" metadata is undocumented, but it is apparently
+  # necessary to enable the capture of logs from the Docker image.
+  #
+  # https://github.com/GoogleCloudPlatform/konlet/issues/56
+  labels = {
+    "${module.wpt-server-container.vm_container_label_key}" = module.wpt-server-container.vm_container_label
+  }
+
+  network_interface {
+    network    = var.network_name
+    subnetwork = var.subnetwork_name
+    access_config {
+      network_tier = "PREMIUM"
+    }
+  }
+
+  can_ip_forward = false
+
+  // Create a new boot disk from an image
+  disk {
+    auto_delete  = true
+    boot         = true
+    source_image = module.wpt-server-container.source_image
+    type         = "PERSISTENT"
+    disk_type    = "pd-ssd"
+    disk_size_gb = var.wpt_server_disk_size
+    mode         = "READ_WRITE"
+  }
+
+  service_account {
+    email  = "default"
+    scopes = ["storage-ro", "logging-write"]
+  }
+
+  scheduling {
+    automatic_restart   = true
+    on_host_maintenance = "MIGRATE"
+  }
+
+  # startup-script and tf_depends_id comes from the module previously used for wpt-server. (see link at top)
+  # TODO: evaluate if those two should be removed.
+  metadata = {
+    # "${module.wpt-server-container.metadata_key}" = module.wpt-server-container.metadata_value
+    # The value for ${module.wpt-server-container.metadata_key} is temporary. During the upgrade, the metadata rendering changes.
+    # More info: https://github.com/terraform-google-modules/terraform-google-container-vm/blob/master/docs/upgrading_to_v2.0.md
+    # Clarification to the linked docs, metadata changes will destroy the old template and create a new one.
+    # In order to make this as smooth as possible, we will hardcode this.
+    # When ready, remove this temporary metadata and the one on cert-renewer. And uncomment the line above.
+    "${module.wpt-server-container.metadata_key}" = <<-EOT
+              ---
+              spec:
+                containers:
+                - env:
+                  - name: WPT_HOST
+                    value: wpt.live
+                  - name: WPT_ALT_HOST
+                    value: not-wpt.live
+                  - name: WPT_BUCKET
+                    value: wpt-tot-certificates
+                  image: gcr.io/wpt-live/wpt-live-wpt-server-tot@sha256:5d7a3d7a5ca0ba4ca7f6e56ad62aa6342c9ab92d41eea24cc6ce4a9b1e2a6afe
+                restartPolicy: Always
+                volumes: []
+              EOT
+    "startup-script"                              = ""
+    "tf_depends_id"                               = ""
+    "google-logging-enabled"                      = "true"
+  }
+
+  lifecycle {
+    create_before_destroy = true
+  }
+}
+
+########################################
+# Cert Renewers
+# These configurations come from: github.com/dcaba/terraform-google-managed-instance-group
+# More information about how it was used previously: https://github.com/web-platform-tests/wpt.live/blob/67dc5976ccce2e64483f2028a35659d4d6e58891/infrastructure/web-platform-tests/main.tf#L139-L178
+########################################
+
+resource "google_compute_instance_template" "cert_renewers" {
+  name_prefix = "default-"
+
+  machine_type = "f1-micro"
+
+  region = var.region
+
+  tags = ["allow-ssh", "${var.name}-allow"]
+
+  labels = {
+    "${module.cert-renewer-container.vm_container_label_key}" = module.cert-renewer-container.vm_container_label
+  }
+
+  network_interface {
+    network    = var.network_name
+    subnetwork = var.subnetwork_name
+    network_ip = ""
+    access_config {
+      network_tier = "PREMIUM"
+    }
+  }
+
+  can_ip_forward = false
+
+  disk {
+    auto_delete  = true
+    boot         = true
+    source_image = module.cert-renewer-container.source_image
+    type         = "PERSISTENT"
+    disk_type    = "pd-ssd"
+    mode         = "READ_WRITE"
+  }
+
+  service_account {
+    email  = "default"
+    scopes = ["cloud-platform"]
+  }
+
+  # startup-script and tf_depends_id comes from the module previously used for cert renewer. (see link at top)
+  # TODO: evaluate if those two should be removed.
+  metadata = {
+    # "${module.cert-renewer-container.metadata_key}" = module.cert-renewer-container.metadata_value
+    # The value for ${module.cert-renewer-container.metadata_key} is temporary. During the upgrade, the metadata rendering changes.
+    # More info: https://github.com/terraform-google-modules/terraform-google-container-vm/blob/master/docs/upgrading_to_v2.0.md
+    # Clarification to the linked docs, metadata changes will destroy the old template and create a new one.
+    # In order to make this as smooth as possible, we will hardcode this.
+    # When ready, remove this temporary metadata and the one on wpt-server. And uncomment the line above.
+    "${module.cert-renewer-container.metadata_key}" = <<-EOT
+              ---
+              spec:
+                containers:
+                - env:
+                  - name: WPT_HOST
+                    value: wpt.live
+                  - name: WPT_ALT_HOST
+                    value: not-wpt.live
+                  - name: WPT_BUCKET
+                    value: wpt-tot-certificates
+                  image: gcr.io/wpt-live/wpt-live-cert-renewer@sha256:5b3c0a3a2b0d7e2a0e1c0303874d09bb3214aa93dec55ac245cf1c81e7d117d5
+                restartPolicy: Always
+                volumes: []
+              EOT
+    "startup-script"                                = ""
+    "tf_depends_id"                                 = ""
+    "google-logging-enabled"                        = "true"
+  }
+
+  scheduling {
+    preemptible         = false
+    automatic_restart   = true
+    on_host_maintenance = "MIGRATE"
+  }
+
+  lifecycle {
+    create_before_destroy = true
+  }
+}
+
+resource "google_compute_instance_group_manager" "cert_renewers" {
+  name               = "${var.name}-cert-renewers"
+  description        = "compute VM Instance Group"
+  wait_for_instances = false
+
+  base_instance_name = "${var.name}-cert-renewers"
+
+  version {
+    instance_template = google_compute_instance_template.cert_renewers.self_link
+  }
+
+  zone = var.zone
+
+  update_policy {
+    # The type is different from wpt servers's update policy.
+    # TODO: Evaluate why
+    type                  = "OPPORTUNISTIC"
+    minimal_action        = local.update_policy.minimal_action
+    max_unavailable_fixed = local.update_policy.max_unavailable_fixed
+  }
+
+  target_pools = []
+
+  target_size = 1
+
+  dynamic "named_port" {
+    for_each = var.cert_renewer_ports
+    content {
+      name = named_port.value["name"]
+      port = named_port.value["port"]
+    }
+  }
+}