Skip to content

Commit df0e241

Browse files
mrlockstarmrlockstar
committed
Ppha-417: Create infra as code for hub resources
This will include the terraform root module that will deploy an Azure Virtual Desktop (AVD) An ADO pipeline to deploy the hub terraform Frontdoor log analytics workspace DNS Firewall Subnets
1 parent 00c84f2 commit df0e241

31 files changed

+1760
-6
lines changed

‎.azuredevops/pipelines/deploy.yml‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
trigger: none
2+
pr: none
3+
4+
parameters:
5+
- name: commitSHA
6+
displayName: Commit SHA
7+
type: string
8+
- name: environment
9+
displayName: Environment
10+
type: string
11+
- name: prNumber
12+
displayName: Pull request number
13+
type: string
14+
default: ''
15+
- name: pool
16+
displayName: ADO management pool
17+
type: string
18+
19+
stages:
20+
- stage: ${{ parameters.environment }}
21+
displayName: Deploy to ${{ parameters.environment }} environment
22+
pool:
23+
name: ${{ parameters.pool }}
24+
lockBehavior: sequential
25+
isSkippable: false
26+
27+
jobs:
28+
- deployment: DeployApp
29+
displayName: Deploy application
30+
environment: ${{ parameters.environment }}
31+
strategy:
32+
runOnce:
33+
deploy:
34+
steps:
35+
- checkout: self
36+
37+
- task: UsePythonVersion@0
38+
inputs:
39+
versionSpec: '3.x'
40+
architecture: 'x64'
41+
42+
- task: TerraformInstaller@1
43+
displayName: Install terraform
44+
inputs:
45+
terraformVersion: 1.7.0
46+
47+
- task: AzureCLI@2
48+
displayName: Run terraform
49+
inputs:
50+
azureSubscription: lung-${{ parameters.environment }}
51+
scriptType: bash
52+
scriptLocation: inlineScript
53+
addSpnToEnvironment: true
54+
inlineScript: |
55+
export ARM_TENANT_ID="$tenantId"
56+
export ARM_CLIENT_ID="$servicePrincipalId"
57+
export ARM_OIDC_TOKEN="$idToken"
58+
export ARM_USE_OIDC=true
59+
make ci ${{ parameters.environment }} terraform-apply DOCKER_IMAGE_TAG=git-sha-${{ parameters.commitSHA }} PR_NUMBER=${{ parameters.prNumber }}
60+
61+
# - task: AzureCLI@2
62+
# displayName: Run database setup
63+
# inputs:
64+
# azureSubscription: lungcs-${{ parameters.environment }}
65+
# scriptType: bash
66+
# scriptLocation: inlineScript
67+
# addSpnToEnvironment: true
68+
# inlineScript: make ci ${{ parameters.environment }} db-setup PR_NUMBER=${{ parameters.prNumber }}
69+
70+
# - task: AzureCLI@2
71+
# displayName: Run notifications smoke test
72+
# inputs:
73+
# azureSubscription: lungcs-${{ parameters.environment }}
74+
# scriptType: bash
75+
# scriptLocation: inlineScript
76+
# addSpnToEnvironment: true
77+
# inlineScript: make ci ${{ parameters.environment }} notifications-smoke-test PR_NUMBER=${{ parameters.prNumber }}

‎.azuredevops/pipelines/hub-infrastructure-dev.yaml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
---
2+
name: $(Build.SourceBranchName)-$(Date:yyyyMMdd)_$(Rev:r)
3+
trigger: none
4+
pr: none
5+
6+
pool:
7+
name: private-pool-hub-nonlive-uks
8+
# vmImage: ubuntu-latest
9+
10+
resources:
11+
repositories:
12+
- repository: dtos-devops-templates
13+
type: github
14+
name: NHSDigital/dtos-devops-templates
15+
ref: PPHA-417-Create-infra-as-code-for-Hub-resources
16+
endpoint: NHSDigital
17+
18+
variables:
19+
- group: NON_LIVE_hub_backend
20+
- name: TF_DIRECTORY
21+
value: $(System.DefaultWorkingDirectory)/lung_cancer_screening/infrastructure/terraform/hub
22+
- name: TF_VERSION
23+
value: 1.14.3
24+
- name: TF_PLAN_ARTIFACT
25+
value: tf_plan_hub_art_NONLIVE_dev
26+
- name: ENVIRONMENT
27+
value: nonlive-hub
28+
29+
stages:
30+
- stage: terraform_plan
31+
displayName: Terraform Plan
32+
condition: eq(variables['Build.Reason'], 'Manual')
33+
variables:
34+
tfVarsFile: ../../environments/$(ENVIRONMENT)/variables.tfvars
35+
jobs:
36+
- job: init_and_plan
37+
displayName: Init, plan, store artifact
38+
steps:
39+
- checkout: self
40+
- checkout: dtos-devops-templates
41+
- template: .azuredevops/templates/steps/tf_plan.yaml@dtos-devops-templates
42+
43+
- stage: terraform_apply
44+
displayName: Terraform Apply
45+
dependsOn: [terraform_plan]
46+
condition: and(eq(dependencies.terraform_plan.outputs['init_and_plan.TerraformPlan.changesPresent'], 'true'), eq(variables['Build.Reason'], 'Manual'))
47+
jobs:
48+
- deployment: terraform_apply
49+
displayName: Init, get plan artifact, apply
50+
environment: $(ENVIRONMENT)
51+
strategy:
52+
runOnce:
53+
deploy:
54+
steps:
55+
- checkout: self
56+
- checkout: dtos-devops-templates
57+
- template: .azuredevops/templates/steps/tf_apply.yaml@dtos-devops-templates

‎.github/workflows/stage-1-commit.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ jobs:
7777
fetch-depth: 0 # Full history is needed to compare branches
7878
- name: "Check English usage"
7979
uses: ./.github/actions/check-english-usage
80-
# Github actiuons dont have terrafomr installed at the moment
80+
# GitHub actions dont have terraform installed at the moment
8181
# lint-terraform:
8282
# name: "Lint Terraform"
8383
# runs-on: ubuntu-latest

‎.gitleaksignore‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@ infrastructure/bootstrap/main.bicep:generic-api-key:32
2929
infrastructure/bootstrap/modules/storage.bicep:generic-api-key:59
3030
infrastructure/bootstrap/modules/keyVault.bicep:generic-api-key:10
3131
infrastructure/bootstrap/modules/storage.bicep:generic-api-key:59
32-
infrastructure/terraform/hub/data.tf:generic-api-key:18
33-
infrastructure/terraform/hub/data.tf:generic-api-key:22
32+
infrastructure/terraform/hub/data.tf:generic-api-key:19
33+
infrastructure/terraform/hub/data.tf:generic-api-key:23
3434
infrastructure/terraform/resource_group_init/core.bicep:generic-api-key:11
3535
infrastructure/terraform/resource_group_init/keyVault.bicep:generic-api-key:10
3636
infrastructure/terraform/resource_group_init/main.bicep:generic-api-key:30

‎docs/infrastructure/bootstrap.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Bootstrap environment deployment
2+
3+
## Deploy the bootstrap environment
4+
5+
Run the bootstrap deployment for the required hub environment:
6+
7+
```bash
8+
make <hub-environment> bootstrap
9+
```
10+
11+
Example
12+
13+
```bash
14+
make hub-nonlive bootstrap
15+
```
16+
17+
## Find the AVD SP object id (run as someone with AAD read access)
18+
19+
```bash
20+
az ad sp show --id <principle id> --query id
21+
```
22+
23+
## Then assign the role (run as Owner)
24+
25+
```bash
26+
az role assignment create \
27+
--assignee-object-id <AVD_SP_OBJECT_ID> \
28+
--role "Desktop Virtualization Power On Contributor" \
29+
--scope /subscriptions/<SUBSCRIPTION_ID>
30+
```

‎docs/infrastructure/create-environment.md‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# Create an environment
2+
3+
This is the initial manual process to create a new environment like review, dev, production...
4+
5+
## Hub
6+
7+
The environment requires a shared Azure front door profile created in the hub. The service name must be declared in [the hub configuration](https://github.com/NHSDigital/dtos-hub/tree/main/infrastructure/environments). And run the Azure devops pipeline for the corresponding hub (non-live or live).
8+
9+
## Image Gallery
10+
11+
- create a new gallery in the Azure compute galleries with name nonlive_lungcs_compute_gallery
12+
- create a resource group with name rg_hub_nonlive_lungcs_compute_gallery
13+
14+
## Code
15+
16+
- Create the configuration files in `infrastructure/environments/[environment]`
17+
- Add the `[environment]:` target in `scripts/terraform/terraform.mk`
18+
- Add [environment] to the list of environments in `deploy-stage` step of `cicd-2-main-branch.yaml`. For the review environment, there is a single item in `cicd-1-pull-request.yaml`.
19+
- Set the `fetch_secrets_from_app_key_vault` terraform variable to `false`. This is to let terraform create the key vault and prevent reading before it is ready.
20+
21+
## Entra ID
22+
23+
- Create Entra ID groups in `Digital screening` Administrative Unit:
24+
- `postgres_lungcs_[environment]_uks_admin`
25+
- `screening_lungcs_[environment]`
26+
- Ask CCOE to assign role:
27+
- [Form for PIM](https://nhsdigitallive.service-now.com/nhs_digital?id=sc_cat_item&sys_id=28f3ab4f1bf3ca1078ac4337b04bcb78&sysparm_category=114fced51bdae1502eee65b9bd4bcbdc)
28+
- Approver: Add someone from the infrastructure team
29+
- Role Name: `Group.Read.All`
30+
- Application Name: `mi-lungcs-[environment]-adotoaz-uks`
31+
- Application ID: [client.id] (would be of `mi-lungcs-[environment]-ghtoado-uks`)
32+
- Managed identity: `mi-lungcs-[environment]-adotoaz-uks`
33+
- Description: - Managed identity: `mi-lungcs-[environment]-adotoaz-uks` - Role: permanent on Directory
34+
35+
## Bicep
36+
37+
> [!IMPORTANT]
38+
> **Required permissions**: Owner role on both the hub and resource subscriptions
39+
40+
- From AVD:
41+
- Login with Microsoft Graph scope: `az login --scope https://graph.microsoft.com//.default -t HSCIC365.onmicrosoft.com`
42+
- Run bicep: `make [environment] resource-group-init`
43+
44+
## Infra secrets
45+
46+
Add the infrastructure secrets to the _inf_ key vault `kv-lungcs-[environment]-inf`:
47+
48+
- For entra ID authentication (when `enable_entra_id_authentication` is true): aad-client-audiences, aad-client-id, aad-client-secret
49+
- `monitoring-email-address`: email distribution list to receive alerts
50+
51+
## Azure devops
52+
53+
- Create ADO group
54+
- Name: `Run pipeline - [environment]`
55+
- Members: `mi-lungcs-[environment]-ghtoado-uks`. There may be more than 1 in the list. Check client id printed below the name.
56+
- Permissions:
57+
- View project-level information
58+
- Create new pipeline:
59+
- Name: `Deploy to Azure - [environment]`
60+
- Pipeline yaml: `.azuredevops/pipelines/deploy.yml`
61+
- Manage pipeline security:
62+
- Add group: `Run pipeline - [environment]`
63+
- Permissions:
64+
- Edit queue build configuration
65+
- Queue builds
66+
- View build pipeline
67+
- View builds
68+
- Create service connection (ADO)
69+
- Connection type: `Azure Resource Manager`
70+
- Identity type: `Managed identity`
71+
- Subscription for managed identity: `Lung Cancer Risk Check - Non-live hub` or `Lung Cancer Risk Check - Live hub` for prod.
72+
- Resource group for managed identity: `rg-mi-[environment]-uks`
73+
- Managed identity: `mi-lungcs-[environment]-adotoaz-uks`
74+
- Scope level: `Subscription`
75+
- Subscription: `Digital Screening DToS - Core Services Dev`
76+
- Resource group for Service connection: leave blank
77+
- Service Connection Name: `lungcs-[environment]`
78+
- Do NOT tick: Grant access permission to all pipelines
79+
- Security: allow `Deploy to Azure - [environment]` pipeline
80+
- Create ADO environment: [environment]
81+
- Set: exclusive lock (except for review)
82+
- Add pipeline permission for `Deploy to Azure - [environment]` pipeline
83+
84+
## GitHub
85+
86+
- Create GitHub environment [environment]
87+
- Add the protection rule (except in review):
88+
- Deselect `Allow administrators to bypass configured protection rules`
89+
- In `Deployment branches and tags` choose `Selected branches and tags` from the drop-down menu
90+
- Click `Add deployment branch or tag rule` and enter "main"
91+
- Add environment secrets, from `mi-lungcs-[environment]-ghtoado-uks` in GitHub
92+
- _AZURE_CLIENT_ID_
93+
- _AZURE_SUBSCRIPTION_ID_
94+
95+
## First run
96+
97+
- Test running terraform manually from the AVD (Optional)
98+
- Raise a pull request, review and merge to trigger the pipeline
99+
- Check ADO pipeline. You may be prompted to authorise:
100+
- Pipeline: service connection
101+
- Environment: service connection and agent pool
102+
103+
## App secrets
104+
105+
- Add the application secrets to the _app_ key vault `kv-lungcs-[environment]-app`
106+
- Set `fetch_secrets_from_app_key_vault` terraform variable to `true`
107+
- Test running terraform manually from the AVD (Optional)
108+
- Raise a pull request, review and merge to trigger the pipeline

0 commit comments

Comments
 (0)