Add IAM leveled secondary admin/read-only group permissions hierarchy to workloads projects for tiered CI/CD

[obriensystems]

20230808: todo: add project level iam access to https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/environments/common/iam-groups.auto.tfvars#L18

In addition to #263 add alternate admin permissions to workload projects

Opinionated groups gexample

ProdOpsAdmin group

• administrator

ProdRead group

• view only

ProdTelcoAdmin group

• network admin
• or support user
• or view only

ProdSecAdmin group

• view only

ProdBilling group

• view only

Similar to the TEF or TOC default groups

or kcc groups https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/main/solutions/guardrails/configs/iam/group-iam.yaml

[obriensystems]

Work Items

• 5 groups - created in admin, console or ideally via gcloud or terraform
• see https://cloud.google.com/sdk/gcloud/reference/identity/groups/create
• see https://registry.terraform.io/modules/terraform-google-modules/group/google/latest
• associate 2 or more users across the groups
• see https://github.com/CloudLandingZone/accelerators_accelerateurs-gcp/blob/main/deployment-templates/Terraform/guardrails/1-guardrails/iam.tf#L37
• assign a new user to only a read only group to test project read only access
• https://registry.terraform.io/modules/terraform-google-modules/iam/google/latest

[obriensystems]

testing gcloud

michael@cloudshell:~ (vpc-host-nonprod-hh015-gz357)$ gcloud identity groups create security@ob...are --organization="obri...re" --group-type="security" --display-name="Security Group" --description="Description of Security Group"
API [cloudresourcemanager.googleapis.com] not enabled on project [476118442898]. Would you like to enable and retry (this will take a few minutes)? (y/N)?  y

Enabling service [cloudresourcemanager.googleapis.com] on project [476118442898]...
Operation "operations/acat.p2-476118442898-57a388fd-596a-4582-aa18-6a22d8c2270f" finished successfully.
API [cloudidentity.googleapis.com] not enabled on project [476118442898]. Would you like to enable and retry (this will take a few minutes)? (y/N)?  y

Enabling service [cloudidentity.googleapis.com] on project [476118442898]...
Operation "operations/acat.p2-476118442898-2c8ba9c0-aedc-4e5c-8a63-40182b71d29c" finished successfully.
Created.

[obriensystems]

testing terraform
I expected to have to add the two services to the bootstrap project
cloudresourcemanager.googleapis.com
cloudidentity.googleapis.com

Step #4 - "tf apply": │ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project 407485305437 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=407485305437 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.


bootstrap = {
  projectServices = [
 ...   "cloudresourcemanager.googleapis.com", # for iam group creation
    "cloudidentity.googleapis.com" # for iam group creation

rerun bootstrap cloud build job

Step #4 - "tf apply": module.landing_zone_bootstrap.module.project.google_project_service.project_services["cloudidentity.googleapis.com"]: Creation complete after 21s [id=tzpe-tlz-tlz-de/cloudidentity.googleapis.com]

root_@cloudshell:~/lz-tls/_lz2/pbmm-on-gcp-onboarding (tzpe-tlz-tlz-de)$ gcloud services list | grep NAME
NAME: accesscontextmanager.googleapis.com
NAME: appengine.googleapis.com
NAME: artifactregistry.googleapis.com
NAME: cloudbilling.googleapis.com
NAME: cloudbuild.googleapis.com
NAME: cloudidentity.googleapis.com
NAME: cloudkms.googleapis.com
NAME: cloudresourcemanager.googleapis.com
NAME: compute.googleapis.com
NAME: containerregistry.googleapis.com
NAME: iam.googleapis.com
NAME: iamcredentials.googleapis.com
NAME: logging.googleapis.com
NAME: oslogin.googleapis.com
NAME: pubsub.googleapis.com
NAME: secretmanager.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sourcerepo.googleapis.com
NAME: storage-api.googleapis.com


prod apply
3 - "tf plan": Terraform will perform the following actions:
Step #3 - "tf plan": 
Step #3 - "tf plan":   # module.group.google_cloud_identity_group.group will be created
Step #3 - "tf plan":   + resource "google_cloud_identity_group" "group" {
Step #3 - "tf plan":       + create_time          = (known after apply)
Step #3 - "tf plan":       + description          = "security group"
Step #3 - "tf plan":       + display_name         = "security"
Step #3 - "tf plan":       + id                   = (known after apply)
Step #3 - "tf plan":       + initial_group_config = "EMPTY"
Step #3 - "tf plan":       + labels               = {
Step #3 - "tf plan":           + "cloudidentity.googleapis.com/groups.discussion_forum" = ""
Step #3 - "tf plan":         }
Step #3 - "tf plan":       + name                 = (known after apply)
Step #3 - "tf plan":       + parent               = "customers/C02e6x9c7"
Step #3 - "tf plan":       + update_time          = (known after apply)
Step #3 - "tf plan": 
Step #3 - "tf plan":       + group_key {
Step #3 - "tf plan":           + id = "security@terraform.landing.systems"
Step #3 - "tf plan":         }
Step #3 - "tf plan":     }
Step #3 - "tf plan": 
Step #3 - "tf plan":   # module.group.google_cloud_identity_group_membership.managers["root@terraform.landing.systems"] will be created
Step #3 - "tf plan":   + resource "google_cloud_identity_group_membership" "managers" {
Step #3 - "tf plan":       + create_time = (known after apply)
Step #3 - "tf plan":       + group       = (known after apply)
Step #3 - "tf plan":       + id          = (known after apply)
Step #3 - "tf plan":       + name        = (known after apply)
Step #3 - "tf plan":       + type        = (known after apply)
Step #3 - "tf plan":       + update_time = (known after apply)
Step #3 - "tf plan": 
Step #3 - "tf plan":       + member_key {
Step #3 - "tf plan":           + id        = (known after apply)
Step #3 - "tf plan":           + namespace = (known after apply)
Step #3 - "tf plan":         }
Step #3 - "tf plan": 
Step #3 - "tf plan":       + preferred_member_key {
Step #3 - "tf plan":           + id = "root@terraform.landing.systems"
Step #3 - "tf plan":         }
Step #3 - "tf plan": 
Step #3 - "tf plan":       + roles {
Step #3 - "tf plan":           + name = "MANAGER"
Step #3 - "tf plan":         }
Step #3 - "tf plan":       + roles {
Step #3 - "tf plan":           + name = "MEMBER"
Step #3 - "tf plan":         }
Step #3 - "tf plan":     }
Step #3 - "tf plan": 
Step #3 - "tf plan":   # module.group.google_cloud_identity_group_membership.members["root@terraform.landing.systems"] will be created
Step #3 - "tf plan":   + resource "google_cloud_identity_group_membership" "members" {
Step #3 - "tf plan":       + create_time = (known after apply)
Step #3 - "tf plan":       + group       = (known after apply)
Step #3 - "tf plan":       + id          = (known after apply)
Step #3 - "tf plan":       + name        = (known after apply)
Step #3 - "tf plan":       + type        = (known after apply)
Step #3 - "tf plan":       + update_time = (known after apply)
Step #3 - "tf plan": 
Step #3 - "tf plan":       + member_key {
Step #3 - "tf plan":           + id        = (known after apply)
Step #3 - "tf plan":           + namespace = (known after apply)
Step #3 - "tf plan":         }
Step #3 - "tf plan": 
Step #3 - "tf plan":       + preferred_member_key {
Step #3 - "tf plan":           + id = "root@terraform.landing.systems"
Step #3 - "tf plan":         }
Step #3 - "tf plan": 
Step #3 - "tf plan":       + roles {
Step #3 - "tf plan":           + name = "MEMBER"
Step #3 - "tf plan":         }
Step #3 - "tf plan":     }
Step #3 - "tf plan": 
Step #3 - "tf plan":   # module.group.google_cloud_identity_group_membership.owners["root@terraform.landing.systems"] will be created
Step #3 - "tf plan":   + resource "google_cloud_identity_group_membership" "owners" {
Step #3 - "tf plan":       + create_time = (known after apply)
Step #3 - "tf plan":       + group       = (known after apply)
Step #3 - "tf plan":       + id          = (known after apply)
Step #3 - "tf plan":       + name        = (known after apply)
Step #3 - "tf plan":       + type        = (known after apply)
Step #3 - "tf plan":       + update_time = (known after apply)
Step #3 - "tf plan": 
Step #3 - "tf plan":       + member_key {
Step #3 - "tf plan":           + id        = (known after apply)
Step #3 - "tf plan":           + namespace = (known after apply)
Step #3 - "tf plan":         }
Step #3 - "tf plan": 
Step #3 - "tf plan":       + preferred_member_key {
Step #3 - "tf plan":           + id = "root@terraform.landing.systems"
Step #3 - "tf plan":         }
Step #3 - "tf plan": 
Step #3 - "tf plan":       + roles {
Step #3 - "tf plan":           + name = "MEMBER"
Step #3 - "tf plan":         }
Step #3 - "tf plan":       + roles {
Step #3 - "tf plan":           + name = "OWNER"
Step #3 - "tf plan":         }
Step #3 - "tf plan":     }

missing terraform service account permissions
4 - "tf apply": │ Error: Error creating Group: googleapi: Error 403: Error(2015): Permission denied for group resource 'security@terraform.landing.systems'.
Step #

add
https://cloud.google.com/iam/docs/understanding-roles

Identity platform admin
organization administrator

"roles/resourcemanager.organizationAdmin" "roles/identityplatform.admin"

and add the Google Workspace Group Administrator Role
https://cloud.google.com/identity/docs/how-to/setup#assigning_an_admin_role_to_the_service_account
or make the tf sa an owner

owners = ["root...ems", "tfsa0131@tzpe-tlz-tlz-de.iam.gserviceaccount.com"]

does not work - need full https://workspaceupdates.googleblog.com/2020/08/use-service-accounts-google-groups-without-domain-wide-delegation.html

manually add tf sa to admin role

groups admin role

assign service accounts

Step #4 - "tf apply": │ Error: Error creating GroupMembership: googleapi: Error 409: The operation was aborted.
Step #4 - "tf apply": │ 
Step #4 - "tf apply": │   with module.group.google_cloud_identity_group_membership.owners["roo..stems"],
Step #4 - "tf apply": │   on .terraform/modules/group/main.tf line 51, in resource "google_cloud_identity_group_membership" "owners":
Step #4 - "tf apply": │   51: resource "google_cloud_identity_group_membership" "owners" {

for
module "group" {
  source  = "terraform-google-modules/group/google"
  version = "~> 0.6"

  id           = "security@terraform.landing.systems"
  display_name = "security"
  description  = "security group"
  domain       = "terraform.landing.systems"
  owners       = ["ro..systems", "tfsa0131@tzpe-tlz-tlz-de.iam.gserviceaccount.com"]
  managers     = ["ro..stems"]
  members      = ["ro..tems"]
}

retry - already existing - will full delete/recreate
Step #4 - "tf apply": │ Error: Error creating GroupMembership: googleapi: Error 409: Error(4003): Cannot create membership 'root@terraform.landing.systems' in 'groups/00tyjcwt1xepbql' because it already exists.

destroy

Step #4 - "tf apply": module.net-host-prj.module.network["tlzprod-svpc"].module.subnets["prsubnet02"].google_compute_subnetwork.subnetwork: Modifying... [id=projects/tzpe-tlz-tlzprod-host4/regions/northamerica-northeast1/subnetworks/tzpecnr-prsubnet02-host4-snet]
Step #4 - "tf apply": module.group.google_cloud_identity_group_membership.members["root@terraform.landing.systems"]: Destroying... [id=groups/00tyjcwt1xepbql/memberships/101582649481786918348]
Step #4 - "tf apply": module.net-host-prj.module.network["tlzprod-svpc"].module.subnets["prsubnet02"].google_compute_subnetwork.subnetwork: Modifications complete after 1s [id=projects/tzpe-tlz-tlzprod-host4/regions/northamerica-northeast1/subnetworks/tzpecnr-prsubnet02-host4-snet]
Step #4 - "tf apply": module.group.google_cloud_identity_group_membership.members["root@terraform.landing.systems"]: Destruction complete after 1s
Step #4 - "tf apply": module.group.google_cloud_identity_group.group: Destroying... [id=groups/00tyjcwt1xepbql]
Step #4 - "tf apply": module.group.google_cloud_identity_group.group: Destruction complete after 6s

recreate

Step #4 - "tf apply": module.group.google_cloud_identity_group.group: Creating...
Step #4 - "tf apply": module.net-host-prj.module.network["tlzprod-svpc"].module.subnets["prsubnet02"].google_compute_subnetwork.subnetwork: Modifying... [id=projects/tzpe-tlz-tlzprod-host4/regions/northamerica-northeast1/subnetworks/tzpecnr-prsubnet02-host4-snet]
Step #4 - "tf apply": module.net-host-prj.module.network["tlzprod-svpc"].module.subnets["prsubnet02"].google_compute_subnetwork.subnetwork: Modifications complete after 1s [id=projects/tzpe-tlz-tlzprod-host4/regions/northamerica-northeast1/subnetworks/tzpecnr-prsubnet02-host4-snet]
Step #4 - "tf apply": module.group.google_cloud_identity_group.group: Creation complete after 7s [id=groups/02zbgiuw25iqgfh]
Step #4 - "tf apply": module.group.google_cloud_identity_group_membership.members["root@terraform.landing.systems"]: Creating...
Step #4 - "tf apply": module.group.google_cloud_identity_group_membership.owners["root@terraform.landing.systems"]: Creating...
Step #4 - "tf apply": module.group.google_cloud_identity_group_membership.managers["root@terraform.landing.systems"]: Creating...
Step #4 - "tf apply": module.group.google_cloud_identity_group_membership.members["root@terraform.landing.systems"]: Creation complete after 1s [id=groups/02zbgiuw25iqgfh/memberships/101582649481786918348]



Step #4 - "tf apply": │ Error: Error creating GroupMembership: googleapi: Error 409: The operation was aborted.
Step #4 - "tf apply": │ 
Step #4 - "tf apply": │   with module.group.google_cloud_identity_group_membership.owners["root@terraform.landing.systems"],
Step #4 - "tf apply": │   on .terraform/modules/group/main.tf line 51, in resource "google_cloud_identity_group_membership" "owners":
Step #4 - "tf apply": │   51: resource "google_cloud_identity_group_membership" "owners" {
Step #4 - "tf apply": │ 
Step #4 - "tf apply": ╵
Step #4 - "tf apply": ╷
Step #4 - "tf apply": │ Error: Error creating GroupMembership: googleapi: Error 409: The operation was aborted.
Step #4 - "tf apply": │ 
Step #4 - "tf apply": │   with module.group.google_cloud_identity_group_membership.managers["root@terraform.landing.systems"],
Step #4 - "tf apply": │   on .terraform/modules/group/main.tf line 64, in resource "google_cloud_identity_group_membership" "managers":
Step #4 - "tf apply": │   64: resource "google_cloud_identity_group_membership" "managers" {
Step #4 - "tf apply": │ 
Step #4 - "tf apply": ╵

however the recreated group is good as of "members" - commenting owners and managers

using

module "group" {
  source  = "terraform-google-modules/group/google"
  version = "~> 0.6"

  id           = "security@terraform.landing.systems"
  display_name = "security"
  description  = "security group"
  domain       = "terraform.landing.systems"
  #owners       = ["ro..tems"]#, "tfsa0131@tzpe-tlz-tlz-de.iam.gserviceaccount.com"]
  #managers     = ["ro..stems"]
  members      = ["ro..ms"]
}

we are good

Step #4 - "tf apply": module.net-host-prj.module.network["tlzprod-svpc"].module.subnets["prsubnet02"].google_compute_subnetwork.subnetwork: Modifying... [id=projects/tzpe-tlz-tlzprod-host4/regions/northamerica-northeast1/subnetworks/tzpecnr-prsubnet02-host4-snet]
Step #4 - "tf apply": module.group.google_cloud_identity_group.group: Creating...
Step #4 - "tf apply": module.net-host-prj.module.network["tlzprod-svpc"].module.subnets["prsubnet02"].google_compute_subnetwork.subnetwork: Modifications complete after 1s [id=projects/tzpe-tlz-tlzprod-host4/regions/northamerica-northeast1/subnetworks/tzpecnr-prsubnet02-host4-snet]
Step #4 - "tf apply": module.group.google_cloud_identity_group.group: Creation complete after 7s [id=groups/03as4poj0lu6dzm]
Step #4 - "tf apply": module.group.google_cloud_identity_group_membership.members["root@terraform.landing.systems"]: Creating...
Step #4 - "tf apply": module.group.google_cloud_identity_group_membership.members["root@terraform.landing.systems"]: Creation complete after 1s [id=groups/03as4poj0lu6dzm/memberships/101582649481786918348]

Step #4 - "tf apply": Apply complete! Resources: 2 added, 1 changed, 0 destroyed.