You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add link to DfE Analytics terraform docs from setup readme. (#190)
* Add link to DfE Analytics terraform docs from setup readme
* Add Terraform Automation regarding setup for datasets, events table and service accounts
Copy file name to clipboardExpand all lines: docs/google_cloud_bigquery_setup.md
+16-4Lines changed: 16 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -453,6 +453,12 @@ be leaked.
453
453
> **NB:** It may be easier to perform these instructions with two browser tabs
454
454
> open, one for BigQuery and the other for IAM
455
455
456
+
### Terraform Automation
457
+
458
+
The per project setup for datasets, events table and service accounts should be automated through terraform and this should be the preferred option. See the [dfe analytics terraform documentation](https://github.com/DFE-Digital/terraform-modules/blob/main/aks/dfe_analytics/README.md#create-new-dataset-and-events-table) for instructions on how to do this.
459
+
460
+
If datasets, events table and service accounts have been setup using terraform, then steps 1, 2, 3 and 4 below can be skipped, otherwise manual setup will be required as described in these steps.
461
+
456
462
### 1. Create dataset(s)
457
463
458
464
Start by creating a dataset.
@@ -523,7 +529,7 @@ With dfe-analytics our strong preference is to use WIF where possible. Where WIF
523
529
524
530
The diagram below demonstrates our use of WIF within dfe-analytics connecting from an Azure client to BigQuery.
The steps below outline how to setup WIF for service accounts using either gcloud shell scripts or gcloud console.
529
535
@@ -555,21 +561,27 @@ For each project a workload identity pool provider with the name `azure-cip-oidc
555
561
556
562
If this does not exist then one can be created with either the [create gcp workload identity pool provider](https://github.com/DFE-Digital/teacher-services-analytics-cloud/blob/main/scripts/gcloud/create-gcp-workload-identity-pool-provider.sh) gcloud script or from the [IAM](https://console.cloud.google.com/iam-admin/workload-identity-pools/pool/azure-cip-identity-pool) gcloud console using the attributes specified in the gcloud script.
557
563
558
-
### 4. Service account
564
+
### 4. Terraform automation
565
+
566
+
The per project setup for WIF service accounts, permissions and client credentials should be automated through terraform and this should be the preferred option. See the [dfe analytics terraform documentation](https://github.com/DFE-Digital/terraform-modules/blob/main/aks/dfe_analytics/README.md) for instructions on how to do this.
567
+
568
+
If WIF has been setup using terraform, then steps 5, 6 and 7 below can be skipped, otherwise manual setup will be required as described in these steps.
569
+
570
+
### 5. Service account
559
571
560
572
A service account with the correct permissions on the events table should exist.
561
573
562
574
If this does not exist then follow the steps above:
563
575
-[Create an appender service account](#3-create-an-appender-service-account)
564
576
-[Give the service account access to your dataset](#3-create-an-appender-service-account)
565
577
566
-
### 5. Service account permissions for workload identity federation
578
+
### 6. Service account permissions for workload identity federation
567
579
568
580
The service account defined in step 4 above should be granted access using service account impersonation.
569
581
570
582
If this does not exist then access can be granted with either the [update wif service account permissions ](https://github.com/DFE-Digital/teacher-services-analytics-cloud/blob/main/scripts/gcloud/update-wif-service-account-permissions.sh) gcloud script or from the [IAM](https://console.cloud.google.com/iam-admin/workload-identity-pools/pool/azure-cip-identity-pool) gcloud console, by navigating to the "GRANT ACCESS" window. Use the attributes specified in the gcloud script. Note that the subject must be set to the Managed Identity Object ID from Azure for each environment (see Step 1 above).
571
583
572
-
### 6. Download the WIF client credentials
584
+
### 7. Download the WIF client credentials
573
585
574
586
Download the JSON WIF Credentials file and set to following environment variables to the content of this file:
0 commit comments