Skip to content

Fix: Update Gitlab and GitHub steps #1398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

Fix: Update Gitlab and GitHub steps #1398

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7e51eda
refactor: remove base project (#6)
mariammartins Mar 24, 2025
9d3d83c
fix hashicorp/google version
mariammartins Mar 26, 2025
6d7558a
update steps for Gitlab and Github deploy
renato-rudnicki Mar 28, 2025
ba0af20
update readme gitlab
renato-rudnicki Apr 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 43 additions & 19 deletions 0-bootstrap/README-GitHub.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ export the GitHub fine grained access token as an environment variable:
terraform apply bootstrap.tfplan
```

1. Run `terraform output` to get the email address of the terraform service accounts that will be used to run manual steps for `shared` environments in steps `3-networks-dual-svpc`, `3-networks-hub-and-spoke`, and `4-projects`.
1. Run `terraform output` to get the email address of the terraform service accounts that will be used to run manual steps for `shared` environments in steps `3-networks-svpc`, `3-networks-hub-and-spoke`, and `4-projects`.

```bash
export network_step_sa=$(terraform output -raw networks_step_terraform_service_account_email)
Expand Down Expand Up @@ -508,7 +508,7 @@ See any of the envs folder [README.md](../2-environments/envs/production/README.
1. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/actions under `tf-apply`.

1. You can now move to the instructions in the network stage.
To use the [Dual Shared VPC](https://cloud.google.com/architecture/security-foundations/networking#vpcsharedvpc-id7-1-shared-vpc-) network mode go to [Deploying step 3-networks-dual-svpc](#deploying-step-3-networks-dual-svpc),
To use the [Shared VPC](https://cloud.google.com/architecture/security-foundations/networking#vpcsharedvpc-id7-1-shared-vpc-) network mode go to [Deploying step 3-networks-svpc](#deploying-step-3-networks-svpc),
or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hub-and-spoke) to use the [Hub and Spoke](https://cloud.google.com/architecture/security-foundations/networking#hub-and-spoke) network mode.

1. Before moving to the next step, go back to the parent directory.
Expand All @@ -517,9 +517,9 @@ or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hu
cd ..
```

## Deploying step 3-networks-dual-svpc
## Deploying step 3-networks-svpc

1. Clone the repository you created to host the `3-networks-dual-svpc` terraform configuration at the same level of the `terraform-example-foundation` folder.
1. Clone the repository you created to host the `3-networks-svpc` terraform configuration at the same level of the `terraform-example-foundation` folder.

```bash
git clone [email protected]:<GITHUB-OWNER>/<GITHUB-NETWORKS-REPO>.git gcp-networks
Expand Down Expand Up @@ -557,7 +557,7 @@ or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hu
1. Copy contents of foundation to new repo.

```bash
cp -RT ../terraform-example-foundation/3-networks-dual-svpc/ .
cp -RT ../terraform-example-foundation/3-networks-svpc/ .
cp -RT ../terraform-example-foundation/policy-library/ ./policy-library
mkdir -p .github/workflows
cp ../terraform-example-foundation/build/github-tf-* ./.github/workflows/
Expand Down Expand Up @@ -587,8 +587,8 @@ or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hu
```

1. Update `common.auto.tfvars` file with values from your GCP environment.
See any of the envs folder [README.md](../3-networks-dual-svpc/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file.
1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the restricted project.
See any of the envs folder [README.md](../3-networks-svpc/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file.
1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the project.
1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap in the `common.auto.tfvars` file.

```bash
Expand Down Expand Up @@ -643,24 +643,48 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th
./tf-wrapper.sh apply shared
```

1. Push your plan branch.
1. You must manually plan and apply the `production` environment since the `development`, `nonproduction` and `plan` environments depend on it.

```bash
git push --set-upstream origin plan
git checkout production
git merge plan
```

1. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/plan from the `plan` branch to the `production` branch and review the output.
1. Run `init` and `plan` and review output for environment production.

> NOTE: Development and Non-production branches depends on the production branch to be deployed first, for the `3-networks-dual-svpc`.
```bash
./tf-wrapper.sh init production
./tf-wrapper.sh plan production
```

1. The Pull request will trigger a GitHub Action that will run Terraform `init`/`plan`/`validate` in the `production` environment.
1. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under `tf-pull-request`.
1. If the GitHub action is successful, merge the pull request in to the `production` branch.
1. The merge will trigger a GitHub Action that will apply the terraform configuration for the `production` environment.
1. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under `tf-apply`.
1. If the GitHub action is successful, apply the next environment.
1. Run `apply` production.

1. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/plan from the `plan` branch to the `development` branch and review the output.
```bash
./tf-wrapper.sh apply production
```

1. Push your production branch since development and nonproduction depends it.

*Note:** The Production envrionment must be the first branch to be pushed as it includes the DNS Hub communication that will be used by other environments.

```bash
git add .
git commit -m 'Initialize networks repo - production'
git push --set-upstream origin production
```

1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `production` branch to the `plan` branch and review the output.

> NOTE: Development and Non-production branches depends on the production branch to be deployed first, for the `3-networks-dual-svpc`.

1. Push your plan branch.

```bash
git checkout plan --set-upstream origin plan
git push
```

1. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/plan from the `production` branch to the `development` branch and review the output.
1. The Pull request will trigger a GitHub Action that will run Terraform `init`/`plan`/`validate` in the `development` environment.
1. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under `tf-pull-request`.
1. If the GitHub action is successful, merge the pull request in to the `development` branch.
Expand Down Expand Up @@ -748,7 +772,7 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th

1. Update `common.auto.tfvars` file with values from your GCP environment.
See any of the envs folder [README.md](../3-networks-hub-and-spoke/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file.
1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the restricted project.
1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the project.
1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap in the `common.auto.tfvars` file.

```bash
Expand Down
62 changes: 43 additions & 19 deletions 0-bootstrap/README-GitLab.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ Run the `0-bootstrap/scripts/git_create_branches_helper.sh` script to create the

1. Go to https://gitlab.com/GITLAB-OWNER/GITLAB-RUNNER-REPO/-/settings/ci_cd#js-token-access
1. Add all the repositories: Bootstrap, Organization, Environments, Networks, and Projects to the allow list tha allow access to the CI/CD runner image.
1. In "Allow CI job tokens from the following projects to access this project" add the other projects/repositories. Format is <GITLAB-OWNER>/<GITLAB-REPO>
1. In "Allow CI job tokens from the following projects to access this project" add the other projects/repositories. Format is `<GITLAB-OWNER>/<GITLAB-REPO>`

### Deploying step 0-bootstrap

Expand Down Expand Up @@ -283,7 +283,7 @@ export the GitLab personal or group access token as an environment variable:
terraform apply bootstrap.tfplan
```

1. Run `terraform output` to get the email address of the terraform service accounts that will be used to run manual steps for `shared` environments in steps `3-networks-dual-svpc`, `3-networks-hub-and-spoke`, and `4-projects`.
1. Run `terraform output` to get the email address of the terraform service accounts that will be used to run manual steps for `shared` environments in steps `3-networks-svpc`, `3-networks-hub-and-spoke`, and `4-projects`.

```bash
export network_step_sa=$(terraform output -raw networks_step_terraform_service_account_email)
Expand Down Expand Up @@ -539,10 +539,10 @@ See any of the envs folder [README.md](../2-environments/envs/production/README.
```

1. You can now move to the instructions in the network stage.
To use the [Dual Shared VPC](https://cloud.google.com/architecture/security-foundations/networking#vpcsharedvpc-id7-1-shared-vpc-) network mode go to [Deploying step 3-networks-dual-svpc](#deploying-step-3-networks-dual-svpc),
To use the [Dual Shared VPC](https://cloud.google.com/architecture/security-foundations/networking#vpcsharedvpc-id7-1-shared-vpc-) network mode go to [Deploying step 3-networks-svpc](#deploying-step-3-networks-svpc),
or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hub-and-spoke) to use the [Hub and Spoke](https://cloud.google.com/architecture/security-foundations/networking#hub-and-spoke) network mode.

## Deploying step 3-networks-dual-svpc
## Deploying step 3-networks-svpc

1. Navigate into the repo. All subsequent steps assume you are running them from the `gcp-networks` directory.
If you run them from another directory, adjust your copy paths accordingly.
Expand All @@ -560,7 +560,7 @@ or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hu
1. Copy contents of foundation to new repo.

```bash
cp -RT ../terraform-example-foundation/3-networks-dual-svpc/ .
cp -RT ../terraform-example-foundation/3-networks-svpc/ .
cp -RT ../terraform-example-foundation/policy-library/ ./policy-library
cp ../terraform-example-foundation/build/gitlab-ci.yml ./.gitlab-ci.yml
cp ../terraform-example-foundation/build/run_gcp_auth.sh .
Expand Down Expand Up @@ -590,8 +590,8 @@ or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hu
```

1. Update `common.auto.tfvars` file with values from your GCP environment.
See any of the envs folder [README.md](../3-networks-dual-svpc/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file.
1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the restricted project.
See any of the envs folder [README.md](../3-networks-svpc/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file.
1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the project.
1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap in the `common.auto.tfvars` file.

```bash
Expand Down Expand Up @@ -646,37 +646,61 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th
./tf-wrapper.sh apply shared
```

1. You must manually plan and apply the `production` environment since the `development`, `nonproduction` and `plan` environments depend on it.

```bash
git checkout production
git merge plan
```

1. Run `init` and `plan` and review output for environment production.

```bash
./tf-wrapper.sh init production
./tf-wrapper.sh plan production
```

1. Run `apply` production.

```bash
./tf-wrapper.sh apply production
```

1. Push your production branch since development and nonproduction depends it.

*Note:** The Production envrionment must be the first branch to be pushed as it includes the DNS Hub communication that will be used by other environments.

```bash
git add .
git commit -m 'Initialize networks repo - production'
git push
```

1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `production` branch to the `plan` branch and review the output.

1. Push your plan branch.

```bash
git checkout plan
git push
```

1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `plan` branch to the `production` branch and review the output.
1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `production` branch to the `development` branch and review the output.

> NOTE: Development and Non-production branches depends on the production branch to be deployed first, for the `3-networks-dual-svpc`.

1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `production` environment.
1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines.
1. If the GitLab pipelines is successful, merge the merge request in to the `production` branch.
1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `production` environment.
1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines under `tf-apply`.

1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `plan` branch to the `development` branch and review the output.
1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `development` environment.
1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines.
1. If the GitLab pipelines is successful, merge the merge request in to the `development` branch.
1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `development` environment.
1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines under `tf-apply`.
1. If the GitLab pipelines is successful, apply the next environment.

1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `development` branch to the `nonproduction` branch and review the output.
1. Open a merge request in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/merge_requests?scope=all&state=opened from the `production` branch to the `nonproduction` branch and review the output.
1. The merge request will trigger a GitLab pipelines that will run Terraform `init`/`plan`/`validate` in the `nonproduction` environment.
1. Review the GitLab pipelines output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines.
1. If the GitLab pipelines is successful, merge the merge request in to the `nonproduction` branch.
1. The merge will trigger a GitLab pipelines that will apply the terraform configuration for the `nonproduction` environment.
1. Review merge output in GitLab https://gitlab.com/GITLAB-OWNER/GITLAB-NETWORKS-REPO/-/pipelines under `tf-apply`.
1. If the GitLab pipelines is successful, apply the next environment.

1. Before executing the next steps, unset the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` environment variable.

Expand Down Expand Up @@ -728,7 +752,7 @@ An environment variable `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` will be set with th

1. Update `common.auto.tfvars` file with values from your GCP environment.
See any of the envs folder [README.md](../3-networks-hub-and-spoke/envs/production/README.md#inputs) files for additional information on the values in the `common.auto.tfvars` file.
1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the restricted project.
1. You must add your user email in the variable `perimeter_additional_members` to be able to see the resources created in the project.
1. Update the `remote_state_bucket` variable with the backend bucket from step Bootstrap in the `common.auto.tfvars` file.

```bash
Expand Down
8 changes: 4 additions & 4 deletions 0-bootstrap/README-Jenkins.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -546,9 +546,9 @@ Here you will configure a VPN Network tunnel to enable connectivity between the
```

1. Review the apply output in your Controller's web UI (you might want to use the option to "Scan Multibranch Pipeline Now" in your Jenkins Controller UI).
1. You can now move to the instructions in the next step, go to [Deploying step 3-networks-dual-svpc](#deploying-step-3-networks-dual-svpc) to use the Dual Shared VPC mode, or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hub-and-spoke) to use the Hub and Spoke network mode.
1. You can now move to the instructions in the next step, go to [Deploying step 3-networks-svpc](#deploying-step-3-networks-svpc) to use the Dual Shared VPC mode, or go to [Deploying step 3-networks-hub-and-spoke](#deploying-step-3-networks-hub-and-spoke) to use the Hub and Spoke network mode.

## Deploying step 3-networks-dual-svpc
## Deploying step 3-networks-svpc

1. Clone the repo you created manually in 0-bootstrap.

Expand All @@ -568,7 +568,7 @@ Here you will configure a VPN Network tunnel to enable connectivity between the
1. Copy contents of foundation to new repo.

```bash
cp -RT ../terraform-example-foundation/3-networks-dual-svpc/ .
cp -RT ../terraform-example-foundation/3-networks-svpc/ .
cp -RT ../terraform-example-foundation/policy-library/ ./policy-library
cp ../terraform-example-foundation/build/Jenkinsfile .
cp ../terraform-example-foundation/build/tf-wrapper.sh .
Expand Down Expand Up @@ -607,7 +607,7 @@ Here you will configure a VPN Network tunnel to enable connectivity between the
mv access_context.auto.example.tfvars access_context.auto.tfvars
```

1. Update `common.auto.tfvars` file with values from your environment and bootstrap. See any of the envs folder [README.md](../3-networks-dual-svpc/envs/production/README.md) files for additional information on the values in the `common.auto.tfvars` file.
1. Update `common.auto.tfvars` file with values from your environment and bootstrap. See any of the envs folder [README.md](../3-networks-svpc/envs/production/README.md) files for additional information on the values in the `common.auto.tfvars` file.
1. Update `production.auto.tfvars` file with the `target_name_server_addresses`.
1. Update `access_context.auto.tfvars` file with the `access_context_manager_policy_id`.
1. Use `terraform output` to get the backend bucket and networks step Terraform Service Account values from gcp-bootstrap output.
Expand Down
Loading