diff --git a/0-bootstrap/README-GitHub.md b/0-bootstrap/README-GitHub.md index 8b424e319..db5be2592 100644 --- a/0-bootstrap/README-GitHub.md +++ b/0-bootstrap/README-GitHub.md @@ -215,7 +215,7 @@ export the GitHub fine grained access token as an environment variable: terraform apply bootstrap.tfplan ``` -1. Run `terraform output` to get the email address of the terraform service accounts that will be used to run manual steps for `shared` environments in steps `3-networks-dual-svpc`, `3-networks-hub-and-spoke`, and `4-projects`. +1. Run `terraform output` to get the email address of the terraform service accounts that will be used to run manual steps for `shared` environments in steps `3-networks-svpc`, `3-networks-hub-and-spoke`, and `4-projects`. ```bash export network_step_sa=$(terraform output -raw networks_step_terraform_service_account_email) @@ -508,7 +508,7 @@ See any of the envs folder [README.md](../2-environments/envs/production/README. 1. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/actions under `tf-apply`. 1. You can now move to the instructions in the network stage. -To use the [Dual Shared VPC](https://cloud.google.com/architecture/security-foundations/networking#vpcsharedvpc-id7-1-shared-vpc-) network mode go to [Deploying step 3-networks-dual-svpc](#deploying-step-3-networks-dual-svpc), +To use the [Shared VPC](https://cloud.google.com/architecture/security-foundations/networking#vpcsharedvpc-id7-1-shared-vpc-) network mode go to [Deploying step 3-networks-svpc](#deploying-step-3-networks-svpc), or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hub-and-spoke) to use the [Hub and Spoke](https://cloud.google.com/architecture/security-foundations/networking#hub-and-spoke) network mode. 1. Before moving to the next step, go back to the parent directory. @@ -517,9 +517,9 @@ or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hu cd .. ``` -## Deploying step 3-networks-dual-svpc +## Deploying step 3-networks-svpc -1. Clone the repository you created to host the `3-networks-dual-svpc` terraform configuration at the same level of the `terraform-example-foundation` folder. +1. Clone the repository you created to host the `3-networks-svpc` terraform configuration at the same level of the `terraform-example-foundation` folder. ```bash git clone git@github.com:/.git gcp-networks @@ -557,7 +557,7 @@ or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hu 1. Copy contents of foundation to new repo. ```bash - cp -RT ../terraform-example-foundation/3-networks-dual-svpc/ . + cp -RT ../terraform-example-foundation/3-networks-svpc/ . cp -RT ../terraform-example-foundation/policy-library/ ./policy-library mkdir -p .github/workflows cp ../terraform-example-foundation/build/github-tf-* ./.github/workflows/ @@ -587,8 +587,8 @@ or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hu ``` 1. Update `common.auto.tfvars` file with values from your GCP environment. -See any of the envs folder [README.md](../3-networks-dual-svpc/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file. -1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the restricted project. +See any of the envs folder [README.md](../3-networks-svpc/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file. +1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the project. 1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap in the `common.auto.tfvars` file. ```bash @@ -643,24 +643,48 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th ./tf-wrapper.sh apply shared ``` -1. Push your plan branch. +1. You must manually plan and apply the `production` environment since the `development`, `nonproduction` and `plan` environments depend on it. ```bash - git push --set-upstream origin plan + git checkout production + git merge plan ``` -1. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/plan from the `plan` branch to the `production` branch and review the output. +1. Run `init` and `plan` and review output for environment production. - > NOTE: Development and Non-production branches depends on the production branch to be deployed first, for the `3-networks-dual-svpc`. + ```bash + ./tf-wrapper.sh init production + ./tf-wrapper.sh plan production + ``` -1. The Pull request will trigger a GitHub Action that will run Terraform `init`/`plan`/`validate` in the `production` environment. -1. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under `tf-pull-request`. -1. If the GitHub action is successful, merge the pull request in to the `production` branch. -1. The merge will trigger a GitHub Action that will apply the terraform configuration for the `production` environment. -1. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under `tf-apply`. -1. If the GitHub action is successful, apply the next environment. +1. Run `apply` production. -1. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/plan from the `plan` branch to the `development` branch and review the output. + ```bash + ./tf-wrapper.sh apply production + ``` + +1. Push your production branch since development and nonproduction depends it. + +*Note:** The Production envrionment must be the first branch to be pushed as it includes the DNS Hub communication that will be used by other environments. + + ```bash + git add . + git commit -m 'Initialize networks repo - production' + git push --set-upstream origin production + ``` + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `production` branch to the `plan` branch and review the output. + +> NOTE: Development and Non-production branches depends on the production branch to be deployed first, for the `3-networks-dual-svpc`. + +1. Push your plan branch. + + ```bash + git checkout plan --set-upstream origin plan + git push + ``` + +1. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/plan from the `production` branch to the `development` branch and review the output. 1. The Pull request will trigger a GitHub Action that will run Terraform `init`/`plan`/`validate` in the `development` environment. 1. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under `tf-pull-request`. 1. If the GitHub action is successful, merge the pull request in to the `development` branch. @@ -748,7 +772,7 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th 1. Update `common.auto.tfvars` file with values from your GCP environment. See any of the envs folder [README.md](../3-networks-hub-and-spoke/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file. -1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the restricted project. +1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the project. 1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap in the `common.auto.tfvars` file. ```bash diff --git a/0-bootstrap/README-GitLab.md b/0-bootstrap/README-GitLab.md index 6117b9577..ff3207693 100644 --- a/0-bootstrap/README-GitLab.md +++ b/0-bootstrap/README-GitLab.md @@ -184,7 +184,7 @@ Run the `0-bootstrap/scripts/git_create_branches_helper.sh` script to create the 1. Go to https://gitlab.com/GITLAB-OWNER/GITLAB-RUNNER-REPO/-/settings/ci_cd#js-token-access 1. Add all the repositories: Bootstrap, Organization, Environments, Networks, and Projects to the allow list tha allow access to the CI/CD runner image. -1. In "Allow CI job tokens from the following projects to access this project" add the other projects/repositories. Format is / +1. In "Allow CI job tokens from the following projects to access this project" add the other projects/repositories. Format is `/` ### Deploying step 0-bootstrap @@ -283,7 +283,7 @@ export the GitLab personal or group access token as an environment variable: terraform apply bootstrap.tfplan ``` -1. Run `terraform output` to get the email address of the terraform service accounts that will be used to run manual steps for `shared` environments in steps `3-networks-dual-svpc`, `3-networks-hub-and-spoke`, and `4-projects`. +1. Run `terraform output` to get the email address of the terraform service accounts that will be used to run manual steps for `shared` environments in steps `3-networks-svpc`, `3-networks-hub-and-spoke`, and `4-projects`. ```bash export network_step_sa=$(terraform output -raw networks_step_terraform_service_account_email) @@ -539,10 +539,10 @@ See any of the envs folder [README.md](../2-environments/envs/production/README. ``` 1. You can now move to the instructions in the network stage. -To use the [Dual Shared VPC](https://cloud.google.com/architecture/security-foundations/networking#vpcsharedvpc-id7-1-shared-vpc-) network mode go to [Deploying step 3-networks-dual-svpc](#deploying-step-3-networks-dual-svpc), +To use the [Dual Shared VPC](https://cloud.google.com/architecture/security-foundations/networking#vpcsharedvpc-id7-1-shared-vpc-) network mode go to [Deploying step 3-networks-svpc](#deploying-step-3-networks-svpc), or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hub-and-spoke) to use the [Hub and Spoke](https://cloud.google.com/architecture/security-foundations/networking#hub-and-spoke) network mode. -## Deploying step 3-networks-dual-svpc +## Deploying step 3-networks-svpc 1. Navigate into the repo. All subsequent steps assume you are running them from the `gcp-networks` directory. If you run them from another directory, adjust your copy paths accordingly. @@ -560,7 +560,7 @@ or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hu 1. Copy contents of foundation to new repo. ```bash - cp -RT ../terraform-example-foundation/3-networks-dual-svpc/ . + cp -RT ../terraform-example-foundation/3-networks-svpc/ . cp -RT ../terraform-example-foundation/policy-library/ ./policy-library cp ../terraform-example-foundation/build/gitlab-ci.yml ./.gitlab-ci.yml cp ../terraform-example-foundation/build/run_gcp_auth.sh . @@ -590,8 +590,8 @@ or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hu ``` 1. Update `common.auto.tfvars` file with values from your GCP environment. -See any of the envs folder [README.md](../3-networks-dual-svpc/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file. -1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the restricted project. +See any of the envs folder [README.md](../3-networks-svpc/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file. +1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the project. 1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap in the `common.auto.tfvars` file. ```bash @@ -646,37 +646,61 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th ./tf-wrapper.sh apply shared ``` +1. You must manually plan and apply the `production` environment since the `development`, `nonproduction` and `plan` environments depend on it. + + ```bash + git checkout production + git merge plan + ``` + +1. Run `init` and `plan` and review output for environment production. + + ```bash + ./tf-wrapper.sh init production + ./tf-wrapper.sh plan production + ``` + +1. Run `apply` production. + + ```bash + ./tf-wrapper.sh apply production + ``` + + 1. Push your production branch since development and nonproduction depends it. + +*Note:** The Production envrionment must be the first branch to be pushed as it includes the DNS Hub communication that will be used by other environments. + + ```bash + git add . + git commit -m 'Initialize networks repo - production' + git push + ``` + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `production` branch to the `plan` branch and review the output. + 1. Push your plan branch. ```bash + git checkout plan git push ``` -1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `plan` branch to the `production` branch and review the output. +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `production` branch to the `development` branch and review the output. > NOTE: Development and Non-production branches depends on the production branch to be deployed first, for the `3-networks-dual-svpc`. -1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `production` environment. -1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines. -1. If the GitLab pipelines is successful, merge the merge request in to the `production` branch. -1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `production` environment. -1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines under `tf-apply`. - -1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `plan` branch to the `development` branch and review the output. 1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `development` environment. 1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines. 1. If the GitLab pipelines is successful, merge the merge request in to the `development` branch. 1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `development` environment. 1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines under `tf-apply`. -1. If the GitLab pipelines is successful, apply the next environment. -1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `development` branch to the `nonproduction` branch and review the output. +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `production` branch to the `nonproduction` branch and review the output. 1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `nonproduction` environment. 1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines. 1. If the GitLab pipelines is successful, merge the merge request in to the `nonproduction` branch. 1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `nonproduction` environment. 1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines under `tf-apply`. -1. If the GitLab pipelines is successful, apply the next environment. 1. Before executing the next steps, unset the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` environment variable. @@ -728,7 +752,7 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th 1. Update `common.auto.tfvars` file with values from your GCP environment. See any of the envs folder [README.md](../3-networks-hub-and-spoke/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file. -1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the restricted project. +1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the project. 1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap in the `common.auto.tfvars` file. ```bash diff --git a/0-bootstrap/gitlab.tf.example b/0-bootstrap/gitlab.tf.example index afd5fd77c..aa68c4237 100644 --- a/0-bootstrap/gitlab.tf.example +++ b/0-bootstrap/gitlab.tf.example @@ -112,6 +112,7 @@ module "gitlab_oidc" { pool_id = "foundation-pool" provider_id = "foundation-gl-provider" sa_mapping = local.sa_mapping + attribute_condition = "assertion.user_login=='${var.gl_repos.owner}'" } resource "gitlab_project_variable" "variables" { @@ -174,3 +175,20 @@ resource "google_service_account_iam_member" "self_impersonate" { role = "roles/iam.serviceAccountTokenCreator" member = "serviceAccount:${google_service_account.terraform-env-sa[each.key].email}" } + +module "gcp_projects_state_bucket" { + source = "terraform-google-modules/cloud-storage/google//modules/simple_bucket" + version = "~> 9.0" + + name = "${var.bucket_prefix}-${module.seed_bootstrap.seed_project_id}-gcp-projects-tfstate" + project_id = module.seed_bootstrap.seed_project_id + location = var.default_region + force_destroy = var.bucket_force_destroy + + encryption = { + default_kms_key_name = local.state_bucket_kms_key + } + + depends_on = [module.seed_bootstrap.gcs_bucket_tfstate] +} +