diff --git a/0-bootstrap/README-GitLab.md b/0-bootstrap/README-GitLab.md index 146cd695e..4e66b2021 100644 --- a/0-bootstrap/README-GitLab.md +++ b/0-bootstrap/README-GitLab.md @@ -13,19 +13,20 @@ To run the instructions described in this document, install the following: - [Google Cloud SDK](https://cloud.google.com/sdk/install) version 393.0.0 or later - [terraform-tools](https://cloud.google.com/docs/terraform/policy-validation/validate-policies#install) component - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) version 2.28.0 or later -- [GLab](https://gitlab.com/gitlab-org/cli) version 1.32.0 or later -- [Terraform](https://www.terraform.io/downloads.html) version 1.3.0 or later +- [Terraform](https://www.terraform.io/downloads.html) version 1.3.0 or later +- [jq](https://jqlang.github.io/jq/) version 1.6 or later. Also make sure that you have the following: - A [GitLab](https://docs.gitlab.com/ee/user/profile/account/create_accounts.html) account for your User or Group. -- A private GitLab project (repository) for each one of the stages of Foundation: +- A private GitLab project (repository) for each one of the stages of Foundation and one for the GutLab runner Image: - Bootstrap - Organization - Environments - Networks - Projects -A [Personal](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) access token or a [Group](https://docs.gitlab.com/ee/user/group/settings/group_access_tokens.html) access token configured with the following [scopes](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#personal-access-token-scopes): + - CI/CD Runner +- A [Personal](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) access token or a [Group](https://docs.gitlab.com/ee/user/group/settings/group_access_tokens.html) access token configured with the following [scopes](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#personal-access-token-scopes): - read_api - create_runner - read_repository @@ -59,41 +60,6 @@ that are created, see the organization bootstrap module git clone https://github.com/terraform-google-modules/terraform-example-foundation.git ``` -### Build CI/CD runner image - -1. Clone the private project you created to host the docker configuration for the CI/CD runner at the same level of the `terraform-example-foundation` folder. -You must have [SSH keys](https://docs.gitlab.com/ee/user/ssh.html) configured with GitLab. - - ```bash - git clone git@gitlab.com:/.git gcp-cicd-runner - ``` - -1. Navigate into the repo. All subsequent - steps assume you are running them from the `gcp-cicd-runner` directory. - If you run them from another directory, adjust your copy paths accordingly. - - ```bash - cd gcp-cicd-runner - ``` - -1. Copy contents of foundation to cloned project (modify accordingly based on your current directory). - - ```bash - cp ../terraform-example-foundation/build/gitlab/gitlab-build-image-ci.yml ./.gitlab-ci.yml - cp ../terraform-example-foundation/build/gitlab/Dockerfile ./Dockerfile - ``` - -1. Save the CI/CD runner configuration to `gcp-cicd-runner` GitLab project: - - ```bash - git add . - git commit -m 'Initialize CI/CD runner project' - git push --set-upstream origin main - ``` - -1. Review the CI/CD Job output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-RUNNER-REPO/-/jobs under name `build-image`. -1. If the CI/CD Job is successful proceed with the next steps - ### Deploying step 0-bootstrap 1. Clone the private project you created to host the `0-bootstrap` terraform configuration at the same level of the `terraform-example-foundation` folder. @@ -107,7 +73,6 @@ You must have [SSH keys](https://docs.gitlab.com/ee/user/ssh.html) configured wi ```bash gcp-bootstrap/ - gcp-cicd-runner/ terraform-example-foundation/ ``` @@ -133,10 +98,9 @@ You must have [SSH keys](https://docs.gitlab.com/ee/user/ssh.html) configured wi cp -RT ../terraform-example-foundation/0-bootstrap/ ./envs/shared cp -RT ../terraform-example-foundation/policy-library/ ./policy-library cp ../terraform-example-foundation/build/gitlab-ci.yml ./.gitlab-ci.yml - cp ../terraform-example-foundation/build/run_gcp_sts.sh . - chmod 755 ./run_gcp_sts.sh + cp ../terraform-example-foundation/build/run_gcp_auth.sh . cp ../terraform-example-foundation/build/tf-wrapper.sh . - chmod 755 ./tf-wrapper.sh + chmod 755 ./*.sh cd ./envs/shared ``` @@ -170,11 +134,10 @@ export the GitLab personal or group access token as an environment variable: ```bash export TF_VAR_gitlab_token="YOUR-PERSONAL-OR-GROUP-ACCESS-TOKEN" ``` - 1. Use the helper script [validate-requirements.sh](../scripts/validate-requirements.sh) to validate your environment: ```bash - ../../../terraform-example-foundation/scripts/validate-requirements.sh -o -b -u -t GitHub + ../../../terraform-example-foundation/scripts/validate-requirements.sh -o -b -u ``` **Note:** The script is not able to validate if the user is in a Cloud Identity or Google Workspace group with the required roles. @@ -255,6 +218,91 @@ export the GitLab personal or group access token as an environment variable: git push --set-upstream origin plan ``` +### Build CI/CD runner image + +1. Clone the private project you created to host the docker configuration for the CI/CD runner at the same level of the `terraform-example-foundation` folder. +You must have [SSH keys](https://docs.gitlab.com/ee/user/ssh.html) configured with GitLab. + + ```bash + git clone git@gitlab.com:/.git gcp-cicd-runner + ``` + +1. Navigate into the repo. All subsequent steps assume you are running them from the `gcp-cicd-runner` directory. + If you run them from another directory, adjust your copy paths accordingly. + + ```bash + cd gcp-cicd-runner + ``` + +1. Create a new branch called `image` which will contains the Docker image using in the Gitlab Pipelines. + + ```bash + git checkout -b image + ``` + +1. Copy contents of foundation to cloned project (modify accordingly based on your current directory). + + ```bash + cp ../terraform-example-foundation/build/gitlab-ci.yml ./.gitlab-ci.yml + cp ../terraform-example-foundation/0-bootstrap/Dockerfile ./Dockerfile + ``` +1. Create a runner for your GitLab CICD Repository in [Settings -> CICD -> Runners](https://gitlab.com/GITLAB-ACCOUNT/BOOTSTRAP-REPOSITORY/-/settings/ci_cd) called `gl_runner` and also set the tag name for `gl_runner`. + +1. To execute the next step, you need to allow the SSH access for your Gitlab instance created in the CICD Project from 0-bootstrap step running the command below: + +```bash +gcloud compute firewall-rules create allow-ssh --allow tcp:22 --network gl-runner-network --project +``` + +1. Once you have access to your GitLab runner instance, you need to register your runner in the GitLab using the command below. Remember to change the token `glrt-xxx` for the token generated in the GitLab interface. + +```bash +sudo gitlab-runner register --non-interactive --url https://gitlab.com --name gl_runner --executor docker --docker-image docker:dind --docker-privileged=true --token glrt-xxx +``` + +1. Edit the `./.gitlab-ci.yml` file and update the paramenter the following line: + +```bash +image: + name: registry.gitlab.com/GITLAB-ACCOUNT/UPDATE_ME/terraform-gcloud:latest +``` + +To: + +```bash +image: + name: registry.gitlab.com/GITLAB-ACCOUNT/GITLAB-IMAGE_REPOSITORY/terraform-gcloud:latest +``` + +1. Save the CI/CD runner configuration to `gcp-cicd-runner` GitLab project: + + ```bash + git add . + git commit -m 'Initialize CI/CD runner project' + git push --set-upstream origin image + ``` + +1. Review the CI/CD Job output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-RUNNER-REPO/-/jobs under name `build-image`. + +1. If the CI/CD Job is successful proceed with the next steps. + +1. Copy the file `gitlab-ci.yml` to the 0-bootstrap directory. + +```bash +cp .gitlab-ci.yml ../gcp-bootstrap/.gitlab-ci.yml +``` + +1. Make sure you have enabled the Gitlab Runner to the bootstrap project. You can do this in `https://gitlab.com/YOUR-GITLAB-USER/gcp-cicd-runner/-/settings/ci_cd#js-runners-settings`. + +1. Push the `gitlab-ci.yml` to the 0-bootsrap repository. + + ```bash + cd ../gcp-bootstrap + git add . + git commit -m 'Initialize CICD to the bootstrap project' + git push --set-upstream origin plan + ``` + TODO **Note 1:** The stages after `0-bootstrap` use `terraform_remote_state` data source to read common configuration like the organization ID from the output of the `0-bootstrap` stage. @@ -263,3 +311,699 @@ They will [fail](../docs/TROUBLESHOOTING.md#error-unsupported-attribute) if the **Note 2:** After the deploy, to prevent the project quota error described in the [Troubleshooting guide](../docs/TROUBLESHOOTING.md#project-quota-exceeded), we recommend that you request 50 additional projects for the **projects step service account** created in this step. +**Note 3:** In case GitLab variables are not being created on Gitlab repositories, it may be related to the existence of Access Token in both User/Group profile and in the repo settings. The deploy will [fail](../docs/TROUBLESHOOTING.md#error-repository-not-found). + +**Note 4:** If the GitLab pipelines fail in 0-bootstrep or next steps, you may need to disable `Limit access to this project` in the CICD Runner repository. For more details see [fail](../docs/TROUBLESHOOTING.md#gitlab-pipelines-access-denied) + + +## Deploying step 1-org + +1. Clone the repository you created to host the `1-org` terraform configuration at the same level of the `terraform-example-foundation` folder. + + ```bash + git clone git@gilab.com:/.git gcp-org + ``` + +1. Navigate into the repo. All subsequent steps assume you are running them from the `gcp-org` directory. + If you run them from another directory, adjust your copy paths accordingly. + + ```bash + cd gcp-org + ``` + +1. Seed the repository if it has not been initialized yet. + + ```bash + git commit --allow-empty -m 'repository seed' + git push --set-upstream origin main + + git checkout -b production + git push --set-upstream origin production + ``` + +1. change to a non-production branch. + + ```bash + git checkout -b plan + ``` + +1. Copy contents of foundation to new repo. + + ```bash + cp -RT ../terraform-example-foundation/1-org/ . + cp -RT ../terraform-example-foundation/policy-library/ ./policy-library + cp ../terraform-example-foundation/build/gitlab-ci.yml ./.gitlab-ci.yml + cp ../terraform-example-foundation/build/run_gcp_auth.sh . + cp ../terraform-example-foundation/build/tf-wrapper.sh . + chmod 755 ./*.sh + ``` + +1. Rename `./envs/shared/terraform.example.tfvars` to `./envs/shared/terraform.tfvars` + + ```bash + mv ./envs/shared/terraform.example.tfvars ./envs/shared/terraform.tfvars + ``` + +1. Update the file `envs/shared/terraform.tfvars` with values from your GCP environment. +See the shared folder [README.md](../1-org/envs/shared/README.md#inputs) for additional information on the values in the `terraform.tfvars` file. + +1. Un-comment the variable `create_access_context_manager_access_policy = false` if your organization already has an Access Context Manager Policy. + + ```bash + export ORGANIZATION_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -json common_config | jq '.org_id' --raw-output) + + export ACCESS_CONTEXT_MANAGER_ID=$(gcloud access-context-manager policies list --organization ${ORGANIZATION_ID} --format="value(name)") + + echo "access_context_manager_policy_id = ${ACCESS_CONTEXT_MANAGER_ID}" + + if [ ! -z "${ACCESS_CONTEXT_MANAGER_ID}" ]; then sed -i "s=//create_access_context_manager_access_policy=create_access_context_manager_access_policy=" ./envs/shared/terraform.tfvars; fi + ``` + +1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap. + + ```bash + export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) + + echo "remote_state_bucket = ${backend_bucket}" + + sed -i "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./envs/shared/terraform.tfvars + ``` + +1. Check if a Security Command Center Notification with the default name, **scc-notify**, already exists in your organization. + + ```bash + export ORG_STEP_SA=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw organization_step_terraform_service_account_email) + + gcloud scc notifications describe "scc-notify" --format="value(name)" --organization=${ORGANIZATION_ID} --impersonate-service-account=${ORG_STEP_SA} + ``` + +1. If the notification exists the output will be: + + ```text + organizations/ORGANIZATION_ID/notificationConfigs/scc-notify + ``` + +1. If the notification does not exist the output will be: + + ```text + ERROR: (gcloud.scc.notifications.describe) NOT_FOUND: Requested entity was not found. + ``` + +1. If the notification exists, choose a different value for the `scc_notification_name` variable in the `./envs/shared/terraform.tfvars` file. + +1. Commit changes. + + ```bash + git add . + git commit -m 'Initialize org repo' + ``` + +1. Push your plan branch. + + ```bash + git push --set-upstream origin plan + ``` + +**Note 1:** Before open a merge request, make sure the option `Delete source branch` is unchecked. + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/merge_requests?scope=all&state=opened from the `plan` branch to the `production` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `production` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `production` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `production` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. +1. If the GitLab pipelines is successful, apply the next environment. + + +## Deploying step 2-environments + +1. Clone the repository you created to host the `2-environments` terraform configuration at the same level of the `terraform-example-foundation` folder. + + ```bash + git clone git@gitlab.com:/.git gcp-environments + ``` + +1. Navigate into the repo. All subsequent + steps assume you are running them from the `gcp-environments` directory. + If you run them from another directory, adjust your copy paths accordingly. + + ```bash + cd gcp-environments + ``` + +1. Seed the repository if it has not been initialized yet. + + ```bash + git commit --allow-empty -m 'repository seed' + git push --set-upstream origin main + + git checkout -b production + git push --set-upstream origin production + + git checkout -b non-production + git push --set-upstream origin non-production + + git checkout -b development + git push --set-upstream origin development + ``` + +1. change to a non-production branch. + + ```bash + git checkout -b plan + ``` + +1. Copy contents of foundation to new repo. + + ```bash + cp -RT ../terraform-example-foundation/2-environments/ . + cp -RT ../terraform-example-foundation/policy-library/ ./policy-library + cp ../terraform-example-foundation/build/gitlab-ci.yml ./.gitlab-ci.yml + cp ../terraform-example-foundation/build/run_gcp_auth.sh . + cp ../terraform-example-foundation/build/tf-wrapper.sh . + chmod 755 ./*.sh + ``` + +1. Rename `terraform.example.tfvars` to `terraform.tfvars`. + + ```bash + mv terraform.example.tfvars terraform.tfvars + ``` + +1. Update the file with values from your GCP environment. +See any of the envs folder [README.md](../2-environments/envs/production/README.md#inputs) files for additional information on the values in the `terraform.tfvars` file. + +1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap. + + ```bash + export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate) + echo "remote_state_bucket = ${backend_bucket}" + + sed -i "s/REMOTE_STATE_BUCKET/${backend_bucket}/" terraform.tfvars + ``` + +1. Commit changes. + + ```bash + git add . + git commit -m 'Initialize environments repo' + ``` + +1. Push your plan branch. + + ```bash + git push --set-upstream origin plan + ``` + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `plan` branch to the `development` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `development` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `development` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `development` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. +1. If the Gitlab pipelines is successful, apply the next environment. + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `development` branch to the `non-production` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `non-production` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `non-production` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `non-production` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. +1. If the GitLab pipelines is successful, apply the next environment. + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `non-production` branch to the `production` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `production` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `production` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `production` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. + +1. You can now move to the instructions in the network stage. +To use the [Dual Shared VPC](https://cloud.google.com/architecture/security-foundations/networking#vpcsharedvpc-id7-1-shared-vpc-) network mode go to [Deploying step 3-networks-dual-svpc](#deploying-step-3-networks-dual-svpc), +or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hub-and-spoke) to use the [Hub and Spoke](https://cloud.google.com/architecture/security-foundations/networking#hub-and-spoke) network mode. + +## Deploying step 3-networks-dual-svpc + +1. Clone the repository you created to host the `3-networks-dual-svpc` terraform configuration at the same level of the `terraform-example-foundation` folder. + + ```bash + git clone git@gitlab.com:/.git gcp-networks + ``` + +1. Navigate into the repo. All subsequent steps assume you are running them from the `gcp-networks` directory. + If you run them from another directory, adjust your copy paths accordingly. + + ```bash + cd gcp-networks + ``` + +1. Seed the repository if it has not been initialized yet. + + ```bash + git commit --allow-empty -m 'repository seed' + git push --set-upstream origin main + + git checkout -b production + git push --set-upstream origin production + + git checkout -b non-production + git push --set-upstream origin non-production + + git checkout -b development + git push --set-upstream origin development + ``` + +1. change to a non-production branch. + + ```bash + git checkout -b plan + ``` + +1. Copy contents of foundation to new repo. + + ```bash + cp -RT ../terraform-example-foundation/3-networks-dual-svpc/ . + cp -RT ../terraform-example-foundation/policy-library/ ./policy-library + cp ../terraform-example-foundation/build/gitlab-ci.yml ./.gitlab-ci.yml + cp ../terraform-example-foundation/build/run_gcp_auth.sh . + cp ../terraform-example-foundation/build/tf-wrapper.sh . + chmod 755 ./*.sh + ``` + +1. Rename `common.auto.example.tfvars` to `common.auto.tfvars`, rename `shared.auto.example.tfvars` to `shared.auto.tfvars` and rename `access_context.auto.example.tfvars` to `access_context.auto.tfvars`. + + ```bash + mv common.auto.example.tfvars common.auto.tfvars + mv shared.auto.example.tfvars shared.auto.tfvars + mv access_context.auto.example.tfvars access_context.auto.tfvars + ``` + +1. Update the file `shared.auto.tfvars` with the values for the `target_name_server_addresses`. +1. Update the file `access_context.auto.tfvars` with the organization's `access_context_manager_policy_id`. + + ```bash + export ORGANIZATION_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -json common_config | jq '.org_id' --raw-output) + + export ACCESS_CONTEXT_MANAGER_ID=$(gcloud access-context-manager policies list --organization ${ORGANIZATION_ID} --format="value(name)") + + echo "access_context_manager_policy_id = ${ACCESS_CONTEXT_MANAGER_ID}" + + sed -i "s/ACCESS_CONTEXT_MANAGER_ID/${ACCESS_CONTEXT_MANAGER_ID}/" ./access_context.auto.tfvars + ``` + +1. Update `common.auto.tfvars` file with values from your GCP environment. +See any of the envs folder [README.md](../3-networks-dual-svpc/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file. +1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the restricted project. +1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap in the `common.auto.tfvars` file. + + ```bash + export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw gcs_bucket_tfstate) + + echo "remote_state_bucket = ${backend_bucket}" + + sed -i "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./common.auto.tfvars + ``` + +1. Commit changes + + ```bash + git add . + git commit -m 'Initialize networks repo' + ``` + +1. You must manually plan and apply the `shared` environment (only once) since the `development`, `non-production` and `production` environments depend on it. +1. Use `terraform output` to get the CI/CD project ID and the networks step Terraform Service Account from gcp-bootstrap output. +1. The CI/CD project ID will be used in the [validation](https://cloud.google.com/docs/terraform/policy-validation/quickstart) of the Terraform configuration + + ```bash + export CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw cicd_project_id) + echo ${CICD_PROJECT_ID} + ``` + +1. The networks step Terraform Service Account will be used for [Service Account impersonation](https://cloud.google.com/docs/authentication/use-service-account-impersonation) in the following steps. +An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with the Terraform Service Account to enable impersonation. + + ```bash + export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw networks_step_terraform_service_account_email) + echo ${GOOGLE_IMPERSONATE_SERVICE_ACCOUNT} + ``` + +1. Run `init` and `plan` and review output for environment shared. + + ```bash + ./tf-wrapper.sh init shared + ./tf-wrapper.sh plan shared + ``` + +1. To use the `validate` option of the `tf-wrapper.sh` script, please follow the [instructions](https://cloud.google.com/docs/terraform/policy-validation/validate-policies#install) to install the terraform-tools component. +1. Run `validate` and check for violations. + + ```bash + ./tf-wrapper.sh validate shared $(pwd)/policy-library ${CICD_PROJECT_ID} + ``` + +1. Run `apply` shared. + + ```bash + ./tf-wrapper.sh apply shared + ``` + +1. Push your plan branch. + + ```bash + git push --set-upstream origin plan + ``` + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `plan` branch to the `development` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `development` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `development` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `development` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. +1. If the Gitlab pipelines is successful, apply the next environment. + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `development` branch to the `non-production` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `non-production` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `non-production` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `non-production` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. +1. If the GitLab pipelines is successful, apply the next environment. + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `non-production` branch to the `production` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `production` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `production` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `production` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. + +1. Before executing the next steps, unset the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` environment variable. + + ```bash + unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT + ``` + +1. You can now move to the instructions in the [4-projects](#deploying-step-4-projects) stage. + +## Deploying step 3-networks-hub-and-spoke + +1. Clone the repository you created to host the `3-networks-hub-and-spoke` terraform configuration at the same level of the `terraform-example-foundation` folder. + + ```bash + git clone git@gitlab.com:/.git gcp-networks + ``` + +1. Navigate into the repo. All subsequent steps assume you are running them from the `gcp-networks` directory. + If you run them from another directory, adjust your copy paths accordingly. + + ```bash + cd gcp-networks + ``` + +1. Seed the repository if it has not been initialized yet. + + ```bash + git commit --allow-empty -m 'repository seed' + git push --set-upstream origin main + + git checkout -b production + git push --set-upstream origin production + + git checkout -b non-production + git push --set-upstream origin non-production + + git checkout -b development + git push --set-upstream origin development + ``` + +1. change to a non-production branch. + + ```bash + git checkout -b plan + ``` + +1. Copy contents of foundation to new repo. + + ```bash + cp -RT ../terraform-example-foundation/3-networks-hub-and-spoke/ . + cp -RT ../terraform-example-foundation/policy-library/ ./policy-library + cp ../terraform-example-foundation/build/gitlab-ci.yml ./.gitlab-ci.yml + cp ../terraform-example-foundation/build/run_gcp_auth.sh . + cp ../terraform-example-foundation/build/tf-wrapper.sh . + chmod 755 ./*.sh + ``` + +1. Rename `common.auto.example.tfvars` to `common.auto.tfvars`, rename `shared.auto.example.tfvars` to `shared.auto.tfvars` and rename `access_context.auto.example.tfvars` to `access_context.auto.tfvars`. + + ```bash + mv common.auto.example.tfvars common.auto.tfvars + mv shared.auto.example.tfvars shared.auto.tfvars + mv access_context.auto.example.tfvars access_context.auto.tfvars + ``` + +1. Update `common.auto.tfvars` file with values from your GCP environment. +See any of the envs folder [README.md](../3-networks-hub-and-spoke/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file. +1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the restricted project. +1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap in the `common.auto.tfvars` file. + + ```bash + export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw gcs_bucket_tfstate) + + echo "remote_state_bucket = ${backend_bucket}" + + sed -i "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./common.auto.tfvars + ``` + +1. Commit changes + + ```bash + git add . + git commit -m 'Initialize networks repo' + ``` + +1. You must manually plan and apply the `shared` environment (only once) since the `development`, `non-production` and `production` environments depend on it. +1. Use `terraform output` to get the CI/CD project ID and the networks step Terraform Service Account from gcp-bootstrap output. +1. The CI/CD project ID will be used in the [validation](https://cloud.google.com/docs/terraform/policy-validation/quickstart) of the Terraform configuration + + ```bash + export CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw cicd_project_id) + echo ${CICD_PROJECT_ID} + ``` + +1. The networks step Terraform Service Account will be used for [Service Account impersonation](https://cloud.google.com/docs/authentication/use-service-account-impersonation) in the following steps. +An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with the Terraform Service Account to enable impersonation. + + ```bash + export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw networks_step_terraform_service_account_email) + echo ${GOOGLE_IMPERSONATE_SERVICE_ACCOUNT} + ``` + +1. Run `init` and `plan` and review output for environment shared. + + ```bash + ./tf-wrapper.sh init shared + ./tf-wrapper.sh plan shared + ``` + +1. To use the `validate` option of the `tf-wrapper.sh` script, please follow the [instructions](https://cloud.google.com/docs/terraform/policy-validation/validate-policies#install) to install the terraform-tools component. +1. Run `validate` and check for violations. + + ```bash + ./tf-wrapper.sh validate shared $(pwd)/policy-library ${CICD_PROJECT_ID} + ``` + +1. Run `apply` shared. + + ```bash + ./tf-wrapper.sh apply shared + ``` + +1. Push your plan branch. + + ```bash + git push --set-upstream origin plan + ``` + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `plan` branch to the `development` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `development` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `development` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `development` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. +1. If the Gitlab pipelines is successful, apply the next environment. + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `development` branch to the `non-production` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `non-production` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `non-production` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `non-production` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. +1. If the GitLab pipelines is successful, apply the next environment. + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `non-production` branch to the `production` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `production` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `production` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `production` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. + + +1. Before executing the next steps, unset the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` environment variable. + + ```bash + unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT + ``` + +1. You can now move to the instructions in the [4-projects](#deploying-step-4-projects) stage. + +## Deploying step 4-projects + +1. Clone the repository you created to host the `4-projects` terraform configuration at the same level of the `terraform-example-foundation` folder. + + ```bash + git clone git@gitlab.com:/.git gcp-projects + ``` + +1. Navigate into the repo. All subsequent + steps assume you are running them from the `gcp-projects` directory. + If you run them from another directory, adjust your copy paths accordingly. + + ```bash + cd gcp-projects + ``` + +1. Seed the repository if it has not been initialized yet. + + ```bash + git commit --allow-empty -m 'repository seed' + git push --set-upstream origin main + + git checkout -b production + git push --set-upstream origin production + + git checkout -b non-production + git push --set-upstream origin non-production + + git checkout -b development + git push --set-upstream origin development + ``` + +1. change to a non-production branch. + + ```bash + git checkout -b plan + ``` + +1. Copy contents of foundation to new repo. + + ```bash + cp -RT ../terraform-example-foundation/4-projects/ . + cp -RT ../terraform-example-foundation/policy-library/ ./policy-library + cp ../terraform-example-foundation/build/gitlab-ci.yml ./.gitlab-ci.yml + cp ../terraform-example-foundation/build/run_gcp_auth.sh . + cp ../terraform-example-foundation/build/tf-wrapper.sh . + chmod 755 ./*.sh + ``` + +1. Rename `auto.example.tfvars` files to `auto.tfvars`. + + ```bash + mv common.auto.example.tfvars common.auto.tfvars + mv shared.auto.example.tfvars shared.auto.tfvars + mv development.auto.example.tfvars development.auto.tfvars + mv non-production.auto.example.tfvars non-production.auto.tfvars + mv production.auto.example.tfvars production.auto.tfvars + ``` + +1. See any of the envs folder [README.md](../4-projects/business_unit_1/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars`, `development.auto.tfvars`, `non-production.auto.tfvars`, and `production.auto.tfvars` files. +1. See any of the shared folder [README.md](../4-projects/business_unit_1/shared/README.md#inputs) files for additional information on the values in the `shared.auto.tfvars` file. + +1. Use `terraform output` to get the backend bucket value from bootstrap output. + + ```bash + export remote_state_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw gcs_bucket_tfstate) + echo "remote_state_bucket = ${remote_state_bucket}" + + sed -i "s/REMOTE_STATE_BUCKET/${remote_state_bucket}/" ./common.auto.tfvars + ``` + +1. Commit changes. + + ```bash + git add . + git commit -m 'Initialize projects repo' + ``` + +1. You need to manually plan and apply only once the `business_unit_1/shared` and `business_unit_2/shared` environments since `development`, `non-production`, and `production` depend on them. + +1. Use `terraform output` to get the CI/CD project ID and the projects step Terraform Service Account from gcp-bootstrap output. +1. The CI/CD project ID will be used in the [validation](https://cloud.google.com/docs/terraform/policy-validation/quickstart) of the Terraform configuration + + ```bash + export CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw cicd_project_id) + echo ${CICD_PROJECT_ID} + ``` + +1. The projects step Terraform Service Account will be used for [Service Account impersonation](https://cloud.google.com/docs/authentication/use-service-account-impersonation) in the following steps. +An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with the Terraform Service Account to enable impersonation. + + ```bash + export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw projects_step_terraform_service_account_email) + echo ${GOOGLE_IMPERSONATE_SERVICE_ACCOUNT} + ``` + +1. Run `init` and `plan` and review output for environment shared. + + ```bash + ./tf-wrapper.sh init shared + ./tf-wrapper.sh plan shared + ``` + +1. To use the `validate` option of the `tf-wrapper.sh` script, please follow the [instructions](https://cloud.google.com/docs/terraform/policy-validation/validate-policies#install) to install the terraform-tools component. +1. Run `validate` and check for violations. + + ```bash + ./tf-wrapper.sh validate shared $(pwd)/policy-library ${CICD_PROJECT_ID} + ``` + +1. Run `apply` shared. + + ```bash + ./tf-wrapper.sh apply shared + ``` + +1. Push your plan branch. + + ```bash + git push --set-upstream origin plan + ``` + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `plan` branch to the `development` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `development` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `development` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `development` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. +1. If the Gitlab pipelines is successful, apply the next environment. + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `development` branch to the `non-production` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `non-production` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `non-production` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `non-production` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. +1. If the GitLab pipelines is successful, apply the next environment. + +1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/merge_requests?scope=all&state=opened from the `non-production` branch to the `production` branch and review the output. +1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `production` environment. +1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ENVIRONMENTS-REPO/-/pipelines. +1. If the GitLab pipelines is successful, merge the merge request in to the `production` branch. +1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `production` environment. +1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-ORGANIZATION-REPO/-/pipelines under `tf-apply`. + +1. Unset the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` environment variable. + + ```bash + unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT + ``` + diff --git a/0-bootstrap/README.md b/0-bootstrap/README.md index 27937a2e9..7ad41c055 100644 --- a/0-bootstrap/README.md +++ b/0-bootstrap/README.md @@ -282,6 +282,7 @@ Each step has instructions for this change. | billing\_account | The ID of the billing account to associate projects with. | `string` | n/a | yes | | bucket\_force\_destroy | When deleting a bucket, this boolean option will delete all contained objects. If false, Terraform will fail to delete buckets which contain objects. | `bool` | `false` | no | | bucket\_prefix | Name prefix to use for state bucket created. | `string` | `"bkt"` | no | +| create\_network | When set to true, VPC,router and NAT will be auto created | `bool` | `true` | no | | default\_region | Default region to create resources where applicable. | `string` | `"us-central1"` | no | | folder\_prefix | Name prefix to use for folders created. Should be the same in all steps. | `string` | `"fldr"` | no | | gitlab\_token | A GitLab personal access token or group access token.
See:
https://docs.gitlab.com/ee/user/group/settings/group_access_tokens.html
https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html | `string` | n/a | yes | @@ -290,11 +291,16 @@ Each step has instructions for this change. | group\_org\_admins | Google Group for GCP Organization Administrators | `string` | n/a | yes | | groups | Contain the details of the Groups to be created. |
object({
create_groups = bool
billing_project = string
required_groups = object({
group_org_admins = string
group_billing_admins = string
billing_data_users = string
audit_data_users = string
monitoring_workspace_users = string
})
optional_groups = object({
gcp_platform_viewer = string
gcp_security_reviewer = string
gcp_network_viewer = string
gcp_scc_admin = string
gcp_global_secrets_admin = string
gcp_audit_viewer = string
})
})
|
{
"billing_project": "",
"create_groups": false,
"optional_groups": {
"gcp_audit_viewer": "",
"gcp_global_secrets_admin": "",
"gcp_network_viewer": "",
"gcp_platform_viewer": "",
"gcp_scc_admin": "",
"gcp_security_reviewer": ""
},
"required_groups": {
"audit_data_users": "",
"billing_data_users": "",
"group_billing_admins": "",
"group_org_admins": "",
"monitoring_workspace_users": ""
}
}
| no | | initial\_group\_config | Define the group configuration when it is initialized. Valid values are: WITH\_INITIAL\_OWNER, EMPTY and INITIAL\_GROUP\_CONFIG\_UNSPECIFIED. | `string` | `"WITH_INITIAL_OWNER"` | no | +| network\_name | Name for the VPC network | `string` | `"gl-runner-network"` | no | | org\_id | GCP Organization ID | `string` | n/a | yes | | org\_policy\_admin\_role | Additional Org Policy Admin role for admin group. You can use this for testing purposes. | `bool` | `false` | no | | org\_project\_creators | Additional list of members to have project creator role across the organization. Prefix of group: user: or serviceAccount: is required. | `list(string)` | `[]` | no | | parent\_folder | Optional - for an organization with existing projects or for development/validation. It will place all the example foundation resources under the provided folder instead of the root organization. The value is the numeric folder ID. The folder must already exist. | `string` | `""` | no | | project\_prefix | Name prefix to use for projects created. Should be the same in all steps. Max size is 3 characters. | `string` | `"prj"` | no | +| repo\_owner | The owner of Gitlab repository. | `string` | n/a | yes | +| subnet\_ip | IP range for the subnet | `string` | `"10.10.10.0/24"` | no | +| subnet\_name | Name for the subnet | `string` | `"gl-runner-subnet"` | no | +| subnetwork\_project | The ID of the project in which the subnetwork belongs. If it is not provided, the project\_id is used. | `string` | n/a | yes | ## Outputs diff --git a/0-bootstrap/cb.tf.example b/0-bootstrap/cb.tf similarity index 100% rename from 0-bootstrap/cb.tf.example rename to 0-bootstrap/cb.tf diff --git a/0-bootstrap/gitlab.tf b/0-bootstrap/gitlab.tf.example similarity index 93% rename from 0-bootstrap/gitlab.tf rename to 0-bootstrap/gitlab.tf.example index b8fa1b614..588f2e16c 100644 --- a/0-bootstrap/gitlab.tf +++ b/0-bootstrap/gitlab.tf.example @@ -91,6 +91,9 @@ module "gitlab_cicd" { "cloudresourcemanager.googleapis.com", "iamcredentials.googleapis.com", "sts.googleapis.com", + "dns.googleapis.com", + "secretmanager.googleapis.com", + ] } @@ -122,3 +125,11 @@ module "cicd_project_wif_iam_member" { parent_id = local.cicd_project_id roles = each.value } + +module "gitlab_runner" { + source = "./modules/gitlab-runner" + + project_id = local.cicd_project_id + #service_account = google_service_account.terraform-env-sa["bootstrap"].email +} + diff --git a/0-bootstrap/modules/gitlab-runner/files/startup_script.sh b/0-bootstrap/modules/gitlab-runner/files/startup_script.sh new file mode 100755 index 000000000..4d3861f8c --- /dev/null +++ b/0-bootstrap/modules/gitlab-runner/files/startup_script.sh @@ -0,0 +1,18 @@ +#!/bin/bash +# Copyright 2023 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash +sudo apt-get -y install gitlab-runner +sudo apt install docker.io -y diff --git a/0-bootstrap/modules/gitlab-runner/main.tf b/0-bootstrap/modules/gitlab-runner/main.tf new file mode 100644 index 000000000..1ef3bca5d --- /dev/null +++ b/0-bootstrap/modules/gitlab-runner/main.tf @@ -0,0 +1,215 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +locals { + service_account = var.service_account == "" ? google_service_account.runner_service_account[0].email : var.service_account + private_googleapis_cidr = module.private_service_connect.private_service_connect_ip +} + +/***************************************** + Optional Runner Networking + *****************************************/ +module "vpc_network" { + source = "terraform-google-modules/network/google" + version = "~> 7.0" + + project_id = var.project_id + network_name = var.network_name + + subnets = [ + { + subnet_name = var.subnet_name + subnet_ip = var.subnet_ip + subnet_region = var.region + subnet_private_access = "true" + }, + ] +} + +resource "google_compute_router" "default" { + name = "${var.network_name}-router" + network = module.vpc_network.network_self_link + region = var.region + project = var.project_id +} + +// Nat is being used here since internet access is required for the Runner Network. Other internet access can be setup instead of NAT resource (e.g: Secure Web Proxy) +resource "google_compute_router_nat" "nat" { + project = var.project_id + name = "${var.network_name}-nat" + router = google_compute_router.default.name + region = google_compute_router.default.region + nat_ip_allocate_option = "AUTO_ONLY" + source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES" +} + +resource "google_dns_policy" "default_policy" { + project = var.project_id + name = "dns-gl-runner-default-policy" + enable_inbound_forwarding = true + enable_logging = true + + networks { + network_url = module.vpc_network.network_self_link + } +} + +/******************************************* + Private service connect and firewall rule + *******************************************/ +resource "google_compute_firewall" "allow_private_api_egress" { + name = "fw-${module.vpc_network.network_name}-65430-e-a-allow-google-apis-all-tcp-443" + network = module.vpc_network.network_name + project = var.project_id + direction = "EGRESS" + priority = 65430 + + dynamic "log_config" { + for_each = var.firewall_enable_logging == true ? [{ + metadata = "INCLUDE_ALL_METADATA" + }] : [] + + content { + metadata = log_config.value.metadata + } + } + + allow { + protocol = "tcp" + ports = ["443"] + } + + destination_ranges = [local.private_googleapis_cidr] + + target_tags = ["gl-runner-vm"] +} + +module "private_service_connect" { + source = "terraform-google-modules/network/google//modules/private-service-connect" + version = "~> 5.2" + + project_id = var.project_id + dns_code = "dz-${module.vpc_network.network_name}" + network_self_link = module.vpc_network.network_self_link + private_service_connect_ip = var.private_service_connect_ip + forwarding_rule_target = "all-apis" +} + +/***************************************** + IAM Bindings GCE SVC + *****************************************/ +resource "google_service_account" "runner_service_account" { + count = var.service_account == "" ? 1 : 0 + + project = var.project_id + account_id = "runner-service-account" + display_name = "GitLab Runner GCE Service Account" +} + +# allow GCE to pull images from GCR +resource "google_project_iam_binding" "gce" { + project = var.project_id + role = "roles/storage.objectViewer" + members = [ + "serviceAccount:${local.service_account}", + ] +} + +/***************************************** + Runner GCE Instance Template + *****************************************/ +module "mig_template" { + source = "terraform-google-modules/vm/google//modules/instance_template" + version = "~> 7.0" + + project_id = var.project_id + machine_type = var.machine_type + network_ip = var.network_ip + network = module.vpc_network.network_name + subnetwork = module.vpc_network.subnets_names[0] + region = var.region + subnetwork_project = var.project_id + service_account = { + email = local.service_account + scopes = [ + "https://www.googleapis.com/auth/cloud-platform", + ] + } + disk_size_gb = 100 + disk_type = "pd-ssd" + auto_delete = true + name_prefix = "gl-runner" + source_image_family = var.source_image_family + source_image_project = var.source_image_project + startup_script = file("${abspath(path.module)}/files/startup_script.sh") + source_image = var.source_image + metadata = var.custom_metadata + tags = [ + "gl-runner-vm" + ] +} + +/***************************************** + Runner MIG + *****************************************/ +module "mig" { + source = "terraform-google-modules/vm/google//modules/mig" + version = "~> 7.0" + + project_id = var.project_id + subnetwork_project = var.project_id + hostname = var.instance_name + region = var.region + instance_template = module.mig_template.self_link + + /* autoscaler */ + autoscaling_enabled = true + min_replicas = var.min_replicas + max_replicas = var.max_replicas + cooldown_period = var.cooldown_period +} + + +# resource "google_compute_instance" "gitlab_runner" { +# name = "gl-runner-instance" +# project = var.project_id +# zone = "us-central1-a" +# machine_type = "e2-medium" +# can_ip_forward = true + +# boot_disk { +# initialize_params { +# image = "debian-cloud/debian-11" +# } +# } +# tags = ["https-server", "gl-runner-vm"] +# metadata_startup_script = file("${abspath(path.module)}/files/startup_script.sh") + +# network_interface { +# subnetwork = module.vpc_network.subnets_names[0] +# network_ip = "10.10.10.8" +# subnetwork_project = var.project_id +# } + +# service_account { +# email = local.service_account +# scopes = ["cloud-platform"] +# } + +# depends_on = [ +# module.vpc_network +# ] +# } diff --git a/0-bootstrap/modules/gitlab-runner/outputs.tf b/0-bootstrap/modules/gitlab-runner/outputs.tf new file mode 100644 index 000000000..5af1b03e9 --- /dev/null +++ b/0-bootstrap/modules/gitlab-runner/outputs.tf @@ -0,0 +1,30 @@ +/** + * Copyright 2020 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +output "mig_instance_group" { + description = "The instance group url of the created MIG" + value = module.mig.instance_group +} + +output "mig_instance_template" { + description = "The name of the MIG Instance Template" + value = module.mig_template.name +} + +output "service_account" { + description = "Service account email for GCE" + value = local.service_account +} diff --git a/0-bootstrap/modules/gitlab-runner/variables.tf b/0-bootstrap/modules/gitlab-runner/variables.tf new file mode 100644 index 000000000..edb1181a4 --- /dev/null +++ b/0-bootstrap/modules/gitlab-runner/variables.tf @@ -0,0 +1,150 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +variable "region" { + type = string + description = "The GCP region to deploy instances into" + default = "us-central1" +} + +variable "network_name" { + type = string + description = "Name for the VPC network" + default = "gl-runner-network" +} + +variable "subnetwork_project" { + type = string + description = "The ID of the project in which the subnetwork belongs. If it is not provided, the project_id is used." + default = "" +} + +variable "subnet_ip" { + type = string + description = "IP range for the subnet" + default = "10.10.10.0/24" +} + +variable "create_subnetwork" { + type = bool + description = "Whether to create subnetwork or use the one provided via subnet_name" + default = true +} + +variable "subnet_name" { + type = string + description = "Name for the subnet" + default = "gl-runner-subnet" +} + +variable "gl_runner_labels" { + type = set(string) + description = "GitLab runner labels to attach to the runners." + default = [] +} + +variable "min_replicas" { + type = number + description = "Minimum number of runner instances" + default = 2 +} + +variable "max_replicas" { + type = number + default = 10 + description = "Maximum number of runner instances" +} + +variable "service_account" { + description = "Service account email address" + type = string + default = "" +} + +variable "instance_name" { + description = "Instance name used by Gitlab Runner" + type = string + default = "gl-runner-vm" +} + +variable "network_ip" { + description = "Private IP address to assign to the instance" + type = string + default = "10.10.10.8" +} + +variable "machine_type" { + type = string + description = "The GCP machine type to deploy" + default = "n1-standard-1" +} + +variable "source_image_family" { + type = string + description = "Source image family. If neither source_image nor source_image_family is specified, defaults to the latest public Ubuntu image." + default = "ubuntu-2204-lts" +} + +variable "source_image_project" { + type = string + description = "Project where the source image comes from" + default = "ubuntu-os-cloud" +} + +variable "source_image" { + type = string + description = "Source disk image. If neither source_image nor source_image_family is specified, defaults to the latest public CentOS image." + default = "" +} + +variable "startup_script" { + type = string + description = "User startup script to run when instances spin up" + default = "" +} + +variable "shutdown_script" { + type = string + description = "User shutdown script to run when instances shutdown" + default = "" +} + +variable "custom_metadata" { + type = map(any) + description = "User provided custom metadata" + default = {} +} + +variable "cooldown_period" { + description = "The number of seconds that the autoscaler should wait before it starts collecting information from a new instance." + type = number + default = 60 +} + +variable "project_id" { + type = string + description = "ID of the project where the Gitlab runner will be created." +} + +variable "private_service_connect_ip" { + type = string + default = "10.10.64.5" +} + +variable "firewall_enable_logging" { + type = bool + default = true +} diff --git a/build/gitlab/main.tf b/0-bootstrap/modules/gitlab-runner/versions.tf similarity index 67% rename from build/gitlab/main.tf rename to 0-bootstrap/modules/gitlab-runner/versions.tf index a43039d56..a7ceb97f0 100644 --- a/build/gitlab/main.tf +++ b/0-bootstrap/modules/gitlab-runner/versions.tf @@ -15,26 +15,13 @@ */ terraform { + required_version = ">= 0.13" required_providers { - gitlab = { - source = "gitlabhq/gitlab" - version = "~> 16.2.0" + + google = { + source = "hashicorp/google" + version = ">= 3.64, < 5.0.0" } } -} - -variable "gitlab_token" { - type = string -} -provider "gitlab" { - token = var.gitlab_token -} - -provider "null" {} - -resource "null_resource" "gcloud_auth_list" { - provisioner "local-exec" { - command = "gcloud auth list" - } } diff --git a/0-bootstrap/outputs.tf b/0-bootstrap/outputs.tf index 2a75b8d52..0248e8e03 100644 --- a/0-bootstrap/outputs.tf +++ b/0-bootstrap/outputs.tf @@ -87,62 +87,61 @@ output "optional_groups" { Specific to cloudbuild_module ---------------------------------------- */ # Comment-out the cloudbuild_bootstrap module and its outputs if you want to use Jenkins instead of Cloud Build -# output "cloudbuild_project_id" { -# description = "Project where Cloud Build configuration and terraform container image will reside." -# value = module.tf_source.cloudbuild_project_id -# } - -# output "gcs_bucket_cloudbuild_artifacts" { -# description = "Bucket used to store Cloud Build artifacts in cicd project." -# value = { for key, value in module.tf_workspace : key => replace(value.artifacts_bucket, local.bucket_self_link_prefix, "") } -# } +output "cloudbuild_project_id" { + description = "Project where Cloud Build configuration and terraform container image will reside." + value = module.tf_source.cloudbuild_project_id +} -# output "gcs_bucket_cloudbuild_logs" { -# description = "Bucket used to store Cloud Build logs in cicd project." -# value = { for key, value in module.tf_workspace : key => replace(value.logs_bucket, local.bucket_self_link_prefix, "") } -# } +output "gcs_bucket_cloudbuild_artifacts" { + description = "Bucket used to store Cloud Build artifacts in cicd project." + value = { for key, value in module.tf_workspace : key => replace(value.artifacts_bucket, local.bucket_self_link_prefix, "") } +} -# output "projects_gcs_bucket_tfstate" { -# description = "Bucket used for storing terraform state for stage 4-projects foundations pipelines in seed project." -# value = module.gcp_projects_state_bucket.bucket.name -# } +output "gcs_bucket_cloudbuild_logs" { + description = "Bucket used to store Cloud Build logs in cicd project." + value = { for key, value in module.tf_workspace : key => replace(value.logs_bucket, local.bucket_self_link_prefix, "") } +} -# output "cloud_builder_artifact_repo" { -# description = "Artifact Registry (AR) Repository created to store TF Cloud Builder images." -# value = "projects/${module.tf_source.cloudbuild_project_id}/locations/${var.default_region}/repositories/${module.tf_cloud_builder.artifact_repo}" -# } +output "projects_gcs_bucket_tfstate" { + description = "Bucket used for storing terraform state for stage 4-projects foundations pipelines in seed project." + value = module.gcp_projects_state_bucket.bucket.name +} -# output "csr_repos" { -# description = "List of Cloud Source Repos created by the module, linked to Cloud Build triggers." -# value = { for k, v in module.tf_source.csr_repos : k => { -# "id" = v.id, -# "name" = v.name, -# "project" = v.project, -# "url" = v.url, -# } -# } -# } +output "cloud_builder_artifact_repo" { + description = "Artifact Registry (AR) Repository created to store TF Cloud Builder images." + value = "projects/${module.tf_source.cloudbuild_project_id}/locations/${var.default_region}/repositories/${module.tf_cloud_builder.artifact_repo}" +} -# output "cloud_build_private_worker_pool_id" { -# description = "ID of the Cloud Build private worker pool." -# value = module.tf_private_pool.private_worker_pool_id +output "csr_repos" { + description = "List of Cloud Source Repos created by the module, linked to Cloud Build triggers." + value = { for k, v in module.tf_source.csr_repos : k => { + "id" = v.id, + "name" = v.name, + "project" = v.project, + "url" = v.url, + } + } +} -# } +output "cloud_build_private_worker_pool_id" { + description = "ID of the Cloud Build private worker pool." + value = module.tf_private_pool.private_worker_pool_id +} -# output "cloud_build_worker_range_id" { -# description = "The Cloud Build private worker IP range ID." -# value = module.tf_private_pool.worker_range_id -# } +output "cloud_build_worker_range_id" { + description = "The Cloud Build private worker IP range ID." + value = module.tf_private_pool.worker_range_id +} -# output "cloud_build_worker_peered_ip_range" { -# description = "The IP range of the peered service network." -# value = module.tf_private_pool.worker_peered_ip_range -# } +output "cloud_build_worker_peered_ip_range" { + description = "The IP range of the peered service network." + value = module.tf_private_pool.worker_peered_ip_range +} -# output "cloud_build_peered_network_id" { -# description = "The ID of the Cloud Build peered network." -# value = module.tf_private_pool.peered_network_id -# } +output "cloud_build_peered_network_id" { + description = "The ID of the Cloud Build peered network." + value = module.tf_private_pool.peered_network_id +} /* ---------------------------------------- Specific to jenkins_bootstrap module @@ -187,12 +186,13 @@ output "optional_groups" { Specific to gitlab_bootstrap ---------------------------------------- */ # Un-comment gitlab_bootstrap and its outputs if you want to use GitLab CI/CD instead of Cloud Build -output "cicd_project_id" { - description = "Project where the CI/CD infrastructure for GitLab CI/CD resides." - value = module.gitlab_cicd.project_id -} +# output "cicd_project_id" { +# description = "Project where the CI/CD infrastructure for GitLab CI/CD resides." +# value = module.gitlab_cicd.project_id +# } + +# output "projects_gcs_bucket_tfstate" { +# description = "Bucket used for storing terraform state for stage 4-projects foundations pipelines in seed project." +# value = module.seed_bootstrap.gcs_bucket_tfstate +# } -output "projects_gcs_bucket_tfstate" { - description = "Bucket used for storing terraform state for stage 4-projects foundations pipelines in seed project." - value = module.seed_bootstrap.gcs_bucket_tfstate -} diff --git a/0-bootstrap/terraform.example.tfvars b/0-bootstrap/terraform.example.tfvars deleted file mode 100644 index 447c2ad57..000000000 --- a/0-bootstrap/terraform.example.tfvars +++ /dev/null @@ -1,113 +0,0 @@ -/** - * Copyright 2021 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -org_id = "REPLACE_ME" # format "000000000000" - -billing_account = "REPLACE_ME" # format "000000-000000-000000" - -group_org_admins = "REPLACE_ME" - -group_billing_admins = "REPLACE_ME" - -# Example of values for the groups -# group_org_admins = "gcp-organization-admins@example.com" -# group_billing_admins = "gcp-billing-admins@example.com" - -default_region = "us-central1" - -// Optional - for an organization with existing projects or for development/validation. -// Uncomment this variable to place all the example foundation resources under -// the provided folder instead of the root organization. -// The variable value is the numeric folder ID -// The folder must already exist. -//parent_folder = "01234567890" - -// Optional - for enabling the automatic groups creation, uncoment the groups -// variable and update the values with the desired group names -//groups = { -// create_groups = true, -// billing_project = "billing-project", -// required_groups = { -// group_org_admins = "group_org_admins_local_test@example.com" -// group_billing_admins = "group_billing_admins_local_test@example.com" -// billing_data_users = "billing_data_users_local_test@example.com" -// audit_data_users = "audit_data_users_local_test@example.com" -// monitoring_workspace_users = "monitoring_workspace_users_local_test@example.com" -// }, -// optional_groups = { -// gcp_platform_viewer = "gcp_platform_viewer_local_test@example.com" -// gcp_security_reviewer = "gcp_security_reviewer_local_test@example.com" -// gcp_network_viewer = "gcp_network_viewer_local_test@example.com" -// gcp_scc_admin = "gcp_scc_admin_local_test@example.com" -// gcp_global_secrets_admin = "gcp_global_secrets_admin_local_test@example.com" -// gcp_audit_viewer = "gcp_audit_viewer_local_test@example.com" -// } -//} -// - - -/* ---------------------------------------- - Specific to jenkins_bootstrap module - ---------------------------------------- */ -// Un-comment the jenkins_bootstrap module and its outputs if you want to use Jenkins instead of Cloud Build -//jenkins_agent_gce_subnetwork_cidr_range = "172.16.1.0/24" -// -//jenkins_agent_gce_private_ip_address = "172.16.1.6" -// -//jenkins_agent_gce_ssh_pub_key = "ssh-rsa [KEY_VALUE] [USERNAME]" -// -//jenkins_agent_sa_email = "jenkins-agent-gce" # service_account_prefix will be added -// -//jenkins_controller_subnetwork_cidr_range = ["10.1.0.6/32"] -// -//nat_bgp_asn = "64514" -// -//vpn_shared_secret = "shared_secret" -// -//on_prem_vpn_public_ip_address = "" -// -//on_prem_vpn_public_ip_address2 = "" -// -//router_asn = "64515" -// -//bgp_peer_asn = "64513" -// -//tunnel0_bgp_peer_address = "169.254.1.1" -// -//tunnel0_bgp_session_range = "169.254.1.2/30" -// -//tunnel1_bgp_peer_address = "169.254.2.1" -// -//tunnel1_bgp_session_range = "169.254.2.2/30" - -/* ---------------------------------------- - Specific to gitlab_bootstrap - ---------------------------------------- */ -# Un-comment gitlab_bootstrap and its outputs if you want to use GitLab CI/CD instead of Cloud Build -# gl_repos = { -# owner = "YOUR-GITLAB-USER-OR-GROUP", -# bootstrap = "YOUR-BOOTSTRAP-REPOSITORY", -# organization = "YOUR-ORGANIZATION-REPOSITORY", -# environments = "YOUR-ENVIRONMENTS-REPOSITORY", -# networks = "YOUR-NETWORKS-REPOSITORY", -# projects = "YOUR-PROJECTS-REPOSITORY", -# } -# -# to prevent saving the `gitlab_token` in plain text in this file, -# export the GitLab access token in the command line -# as an environment variable before running terraform. -# Run the following commnad in your shell: -# export TF_VAR_gitlab_token="YOUR-ACCESS-TOKEN" diff --git a/0-bootstrap/variables.tf b/0-bootstrap/variables.tf index 0a028d576..e97fae4a1 100644 --- a/0-bootstrap/variables.tf +++ b/0-bootstrap/variables.tf @@ -247,37 +247,72 @@ variable "initial_group_config" { # } /* ---------------------------------------- - Specific to github_bootstrap + Specific to gitlab_bootstrap ---------------------------------------- */ -# Un-comment github_bootstrap and its outputs if you want to use GitHub Actions instead of Cloud Build -variable "gl_repos" { - description = < project -> settings -> cicd -> token access -> limit access to this project image: - name: $CI_REGISTRY/terraform-gcloud:latest + name: registry.gitlab.com/GITLAB-ACCOUNT/GITLAB-REPOSITORY/terraform-gcloud:latest entrypoint: [""] +terraform-plan: + stage: validate + tags: + - gl_runner + id_tokens: + GITLAB_OIDC_TOKEN: + aud: "//iam.googleapis.com/${WIF_PROVIDER_NAME}" + variables: + GOOGLE_IMPERSONATE_SERVICE_ACCOUNT: "$SERVICE_ACCOUNT_EMAIL" + script: + - bash run_gcp_auth.sh "${GITLAB_OIDC_TOKEN}" "${WIF_PROVIDER_NAME}" "${SERVICE_ACCOUNT_EMAIL}" `pwd` + - gcloud config set project "$PROJECT_ID" + - bash tf-wrapper.sh plan_validate_all $CI_COMMIT_REF_NAME $CI_PROJECT_DIR/policy-library $PROJECT_ID "FILESYSTEM" terraform-plan: stage: validate + tags: + - gl_runner id_tokens: GITLAB_OIDC_TOKEN: aud: "//iam.googleapis.com/${WIF_PROVIDER_NAME}" variables: GOOGLE_IMPERSONATE_SERVICE_ACCOUNT: "$SERVICE_ACCOUNT_EMAIL" + script: - bash run_gcp_auth.sh "${GITLAB_OIDC_TOKEN}" "${WIF_PROVIDER_NAME}" "${SERVICE_ACCOUNT_EMAIL}" `pwd` - gcloud config set project "$PROJECT_ID" @@ -41,6 +82,8 @@ terraform-plan: terraform-apply: stage: apply + tags: + - gl_runner id_tokens: GITLAB_OIDC_TOKEN: aud: "//iam.googleapis.com/${WIF_PROVIDER_NAME}" @@ -58,3 +101,5 @@ terraform-apply: - development - production - non-production + when: manual + diff --git a/build/gitlab/Dockerfile b/build/gitlab/Dockerfile deleted file mode 100644 index 9b3025035..000000000 --- a/build/gitlab/Dockerfile +++ /dev/null @@ -1,39 +0,0 @@ -# Copyright 2023 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM gcr.io/cloud-builders/gcloud-slim - -# Use ARG so that values can be overriden by user/cloudbuild -ARG TERRAFORM_VERSION=1.3.0 - -ENV ENV_TERRAFORM_VERSION=$TERRAFORM_VERSION - -RUN apt-get update && \ - /builder/google-cloud-sdk/bin/gcloud -q components install alpha beta terraform-tools && \ - apt-get -y install curl jq unzip git ca-certificates gnupg && \ - curl https://releases.hashicorp.com/terraform/${ENV_TERRAFORM_VERSION}/terraform_${ENV_TERRAFORM_VERSION}_linux_amd64.zip --output terraform_${ENV_TERRAFORM_VERSION}_linux_amd64.zip && \ - curl https://releases.hashicorp.com/terraform/${ENV_TERRAFORM_VERSION}/terraform_${ENV_TERRAFORM_VERSION}_SHA256SUMS.sig --output terraform_SHA256SUMS.sig && \ - curl https://releases.hashicorp.com/terraform/${ENV_TERRAFORM_VERSION}/terraform_${ENV_TERRAFORM_VERSION}_SHA256SUMS --output terraform_SHA256SUMS && \ - curl https://keybase.io/hashicorp/pgp_keys.asc --output pgp_keys.asc && \ - gpg --import pgp_keys.asc && \ - gpg --verify terraform_SHA256SUMS.sig terraform_SHA256SUMS && \ - grep terraform_${ENV_TERRAFORM_VERSION}_linux_amd64.zip terraform_SHA256SUMS | shasum --algorithm 256 --check && \ - unzip terraform_${ENV_TERRAFORM_VERSION}_linux_amd64.zip -d /builder/terraform && \ - rm -f terraform_${ENV_TERRAFORM_VERSION}_linux_amd64.zip terraform_SHA256SUMS && \ - apt-get --purge -y autoremove && \ - apt-get clean && \ - rm -rf /var/lib/apt/lists/* - -ENV PATH=/builder/terraform/:$PATH -ENTRYPOINT ["terraform"] diff --git a/build/gitlab/gitlab-build-image-ci.yml b/build/gitlab/gitlab-build-image-ci.yml deleted file mode 100644 index 50123620e..000000000 --- a/build/gitlab/gitlab-build-image-ci.yml +++ /dev/null @@ -1,39 +0,0 @@ -# Copyright 2023 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -stages: - - build - -build-image: - stage: build - image: - name: docker:stable - services: - - name: docker:dind - alias: dockerhost - - variables: - DOCKER_HOST: tcp://dockerhost:2375/ - DOCKER_DRIVER: overlay2 - DOCKER_TLS_CERTDIR: "" - - script: - - docker info # Optional: Display Docker info for debugging - - docker build -t $CI_REGISTRY/terraform-gcloud:latest . - - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - - docker push $CI_REGISTRY/terraform-gcloud:latest - - docker logout - - only: - - main diff --git a/build/gitlab/gitlab-tf-apply.yaml b/build/gitlab/gitlab-tf-apply.yaml deleted file mode 100644 index 3564d110e..000000000 --- a/build/gitlab/gitlab-tf-apply.yaml +++ /dev/null @@ -1,91 +0,0 @@ -# Copyright 2023 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -stages: - - build - - deploy - -build-image: - stage: build - tags: - - runner2 - image: - name: docker:stable - services: - - name: docker:dind - alias: dockerhost - - variables: - DOCKER_HOST: tcp://dockerhost:2375/ - DOCKER_DRIVER: overlay2 - DOCKER_TLS_CERTDIR: "" - - script: - - env # Shows env variables - - docker info # Optional: Display Docker info for debugging - - docker build -t $CI_REGISTRY_IMAGE . - - echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER $CI_REGISTRY --password-stdin - - docker push $CI_REGISTRY_IMAGE - - docker logout - - only: - - main - -# All branches need to be protected, otherwise the value of environment variables won't be read. - -image: - name: $CI_REGISTRY_IMAGE - entrypoint: [""] - -terraform-plan: - stage: deploy - tags: - - runner2 - script: - - bash run_gcp_sts.sh - - gcloud auth list - - bash tf-wrapper.sh plan_validate_all $CI_COMMIT_REF_NAME "$CI_PROJECT_DIR/policy-library" $CI_PROJECT_NAME - - git status - - git clean -ffdx - - git status - - only: - - plan - -terraform-apply: - stage: deploy - tags: - - runner2 - script: - - bash run_gcp_sts.sh - - gcloud auth list - - pwd - - ls -la - - echo "ci-commit-ref-name ${CI_COMMIT_REF_NAME}" - - echo "ci-builds-dir ${CI_BUILDS_DIR}" - - echo "ci project dir ${CI_PROJECT_DIR}" - - echo "ci-project-name ${CI_PROJECT_NAME}" - - ls -la - - bash tf-wrapper.sh init $CI_COMMIT_REF_NAME - - bash tf-wrapper.sh plan $CI_COMMIT_REF_NAME "$CI_PROJECT_DIR/policy-library" $CI_PROJECT_NAME - - bash tf-wrapper.sh validate $CI_COMMIT_REF_NAME "$CI_PROJECT_DIR/policy-library" $CI_PROJECT_NAME - - bash tf-wrapper.sh apply $CI_COMMIT_REF_NAME "$CI_PROJECT_DIR/policy-library" $CI_PROJECT_NAME - # - git status - # - git clean -ffdx - # - git status - - only: - - development - - production - - non-production diff --git a/build/gitlab/run_gcp_sts.sh b/build/gitlab/run_gcp_sts.sh deleted file mode 100755 index 2b15c2f2d..000000000 --- a/build/gitlab/run_gcp_sts.sh +++ /dev/null @@ -1,51 +0,0 @@ -#!/bin/sh -x - -# Copyright 2023 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -PAYLOAD=$(cat < .ci_job_jwt_file -gcloud iam workload-identity-pools create-cred-config "${WIF_PROVIDER_NAME}" \ ---service-account="${SERVICE_ACCOUNT_EMAIL}" \ ---output-file=.gcp_temp_cred.json \ ---credential-source-file=.ci_job_jwt_fgcloud conf listile -gcloud auth login --cred-file=.gcp_temp_cred.json diff --git a/build/gitlab/run_gcp_auth.sh b/build/run_gcp_auth.sh similarity index 100% rename from build/gitlab/run_gcp_auth.sh rename to build/run_gcp_auth.sh diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index 193c16556..dc854e3da 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -24,6 +24,7 @@ See [GLOSSARY.md](./GLOSSARY.md). - [Cannot assign requested address error in Cloud Shell](#cannot-assign-requested-address-error-in-cloud-shell) - [Error: Unsupported attribute](#error-unsupported-attribute) - [Error: Error adding network peering](#error-error-adding-network-peering) +- [Error: Repository not found](#error-repository-not-found) - - - @@ -388,3 +389,42 @@ You can get this information from step `0-bootstrap` by running the following co **Terraform State lock possible causes:** - If you realize that the Terraform State lock was due to a build timeout increase the build timeout on [build configuration](https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/build/cloudbuild-tf-apply.yaml#L15). + +### Terraform deploy fails due GitLab repositories not found + +**Error message:** + +```text +Error: POST https://gitlab.com/api/v4/projects///variables: 404 {message: 404 Project Not Found} + +``` + +**Cause:** + +This message means that you are using a wrong Access Token or you have Access Token created in both Gitlab Account/Group and GitLab Repository. + +Only Personal Access Token under GitLab Account/Group should exist. + +**Solution:** + +Remove any Access Token from the GitLab repositories used by Google Secure Foundation Blueprint. + + +######### +### Gitlab pipelines fails in 0-bootstrap + +**Error message:** + +From the logs of your Pipeline job: +```text +Error response from daemon: pull access denied for registry.gitlab.com///terraform-gcloud, repository does not exist or may require 'docker login': denied: requested access to the resource is denied + +``` + +**Cause:** + +This message means that you GitLab CICD repository may have `Limit access to this project` option enabled. + +**Solution:** + +Disable this option on your `CICD Repo -> Settings -> CI/CD -> Token Access`. diff --git a/scripts/validate-requirements.sh b/scripts/validate-requirements.sh index 25fd8fb5e..3e06f2f98 100755 --- a/scripts/validate-requirements.sh +++ b/scripts/validate-requirements.sh @@ -216,7 +216,7 @@ function check_billing_account_roles(){ # Checks if initial config was done for 0-bootstrap step function validate_bootstrap_step(){ SCRIPTS_DIR="$( dirname -- "$0"; )" - FILE="$SCRIPTS_DIR/../0-bootstrap/terraform.tfvars" + FILE=`pwd`/terraform.tfvars if [ ! -f "$FILE" ]; then echo " Rename the file 0-bootstrap/terraform.example.tfvars to 0-bootstrap/terraform.tfvars" ERRORS+=$' terraform.tfvars file must exist for 0-bootstrap step.\n'