Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
6d26c13
add suport to billing account logs
renato-rudnicki Feb 14, 2024
690f6e7
small improvements
renato-rudnicki Feb 15, 2024
57ec9f6
create custom log bucket and sink for project destination
renato-rudnicki Feb 15, 2024
85d6844
fix lint
renato-rudnicki Feb 15, 2024
bebcbb2
Fix duplicated sink name
renato-rudnicki Feb 16, 2024
f4f2eb8
fix integration tests
renato-rudnicki Feb 16, 2024
14a3a99
adds unique_writer_identity in internal_project_log_export module
renato-rudnicki Feb 19, 2024
ee3554c
update code
renato-rudnicki Feb 22, 2024
5530ed9
Revert "update code"
renato-rudnicki Feb 22, 2024
4775ef3
Revert "adds unique_writer_identity in internal_project_log_export mo…
renato-rudnicki Feb 22, 2024
df8dc0c
Revert "fix integration tests"
renato-rudnicki Feb 22, 2024
47418a3
Revert "Fix duplicated sink name"
renato-rudnicki Feb 22, 2024
06ec1c1
Revert "fix lint"
renato-rudnicki Feb 22, 2024
5c34cd5
Revert "create custom log bucket and sink for project destination"
renato-rudnicki Feb 22, 2024
fa23017
add randon for billing sink
renato-rudnicki Feb 22, 2024
46e44cb
fix for billing account sink
renato-rudnicki Feb 28, 2024
648ed1a
update README
renato-rudnicki Feb 28, 2024
f3c7fbc
fix lint
renato-rudnicki Feb 28, 2024
db9f4e1
fixes for module log_export_billing
renato-rudnicki Feb 29, 2024
ea4e2c0
add PR suggestions
renato-rudnicki Mar 4, 2024
b6bf537
update
renato-rudnicki Mar 5, 2024
07df788
fixes for log_export_billing
renato-rudnicki Mar 6, 2024
e759cf9
add integration tests for sink under billing account level
renato-rudnicki Mar 12, 2024
b41b322
code review improvements
renato-rudnicki Mar 12, 2024
459fc90
fix lint
renato-rudnicki Mar 12, 2024
32fb194
chore(deps): bump google.golang.org/protobuf from 1.31.0 to 1.33.0 in…
dependabot[bot] Mar 14, 2024
8b17343
Merge branch 'master' into feat_sink_billing_account
renato-rudnicki Mar 14, 2024
1fe50db
removing comments
renato-rudnicki Mar 14, 2024
a8b11b8
change default value for enable_billing_account_sink variable
renato-rudnicki Mar 14, 2024
bc5c46d
adds get sink permission on CICD SA
renato-rudnicki Mar 15, 2024
e1274a6
update resource name for billign account sink
renato-rudnicki Mar 15, 2024
f57a47c
fix!: Groups creation and permissions (#1110)
Samir-Cit Mar 18, 2024
27807fd
Merge branch 'master' into feat_sink_billing_account
apeabody Mar 21, 2024
be6b808
chore: update CODEOWNERS
cloud-foundation-bot Mar 22, 2024
a950798
Merge branch 'master' into feat_sink_billing_account
daniel-cit Mar 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 6 additions & 13 deletions 0-bootstrap/README-GitHub.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,19 +37,12 @@ Also make sure that you have the following:
- A Google Cloud [organization](https://cloud.google.com/resource-manager/docs/creating-managing-organization).
- A Google Cloud [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account).
- Cloud Identity or Google Workspace groups for organization and billing admins.
- Add the Identity (user or Service Account) who will run Terraform to the `group_org_admins` group.
They must be in this group, or they won't have `roles/resourcemanager.projectCreator` access.
- For the Identity who will run the procedures in this document, grant the following roles:
- The `roles/resourcemanager.organizationAdmin` role on the Google Cloud organization.
- The `roles/orgpolicy.policyAdmin` role on the Google Cloud organization.
- The `roles/billing.admin` role on the billing account.
- The `roles/resourcemanager.folderCreator` role.

If other users need to be able to run these procedures, add them to the group
represented by the `org_project_creators` variable.
For more information about the permissions that are required, and the resources
that are created, see the organization bootstrap module
[documentation.](https://github.com/terraform-google-modules/terraform-google-bootstrap)
- For the user who will run the procedures in this document, grant the following roles:
- The `roles/resourcemanager.organizationAdmin` role on the Google Cloud organization.
- The `roles/orgpolicy.policyAdmin` role on the Google Cloud organization.
- The `roles/resourcemanager.projectCreator` role on the Google Cloud organization.
- The `roles/billing.admin` role on the billing account.
- The `roles/resourcemanager.folderCreator` role.

## Instructions

Expand Down
19 changes: 6 additions & 13 deletions 0-bootstrap/README-GitLab.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,19 +37,12 @@ Also make sure that you have the following:
- A Google Cloud [organization](https://cloud.google.com/resource-manager/docs/creating-managing-organization).
- A Google Cloud [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account).
- Cloud Identity or Google Workspace groups for organization and billing admins.
- Add the Identity (user or Service Account) who will run Terraform to the `group_org_admins` group.
They must be in this group, or they won't have `roles/resourcemanager.projectCreator` access.
- For the Identity who will run the procedures in this document, grant the following roles:
- The `roles/resourcemanager.organizationAdmin` role on the Google Cloud organization.
- The `roles/orgpolicy.policyAdmin` role on the Google Cloud organization.
- The `roles/billing.admin` role on the billing account.
- The `roles/resourcemanager.folderCreator` role.

If other users need to be able to run these procedures, add them to the group
represented by the `org_project_creators` variable.
For more information about the permissions that are required, and the resources
that are created, see the organization bootstrap module
[documentation.](https://github.com/terraform-google-modules/terraform-google-bootstrap)
- For the user who will run the procedures in this document, grant the following roles:
- The `roles/resourcemanager.organizationAdmin` role on the Google Cloud organization.
- The `roles/orgpolicy.policyAdmin` role on the Google Cloud organization.
- The `roles/resourcemanager.projectCreator` role on the Google Cloud organization.
- The `roles/billing.admin` role on the billing account.
- The `roles/resourcemanager.folderCreator` role.

## Instructions

Expand Down
19 changes: 6 additions & 13 deletions 0-bootstrap/README-Terraform-Cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,19 +43,12 @@ Also make sure that you have the following:
- A Google Cloud [organization](https://cloud.google.com/resource-manager/docs/creating-managing-organization).
- A Google Cloud [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account).
- Cloud Identity or Google Workspace groups for organization and billing admins.
- Add the Identity (user or Service Account) who will run Terraform to the `group_org_admins` group.
They must be in this group, or they won't have `roles/resourcemanager.projectCreator` access.
- For the Identity who will run the procedures in this document, grant the following roles:
- The `roles/resourcemanager.organizationAdmin` role on the Google Cloud organization.
- The `roles/orgpolicy.policyAdmin` role on the Google Cloud organization.
- The `roles/billing.admin` role on the billing account.
- The `roles/resourcemanager.folderCreator` role.

If other users need to be able to run these procedures, add them to the group
represented by the `org_project_creators` variable.
For more information about the permissions that are required, and the resources
that are created, see the organization bootstrap module
[documentation.](https://github.com/terraform-google-modules/terraform-google-bootstrap)
- For the user who will run the procedures in this document, grant the following roles:
- The `roles/resourcemanager.organizationAdmin` role on the Google Cloud organization.
- The `roles/orgpolicy.policyAdmin` role on the Google Cloud organization.
- The `roles/resourcemanager.projectCreator` role on the Google Cloud organization.
- The `roles/billing.admin` role on the billing account.
- The `roles/resourcemanager.folderCreator` role.

### Instructions

Expand Down
30 changes: 7 additions & 23 deletions 0-bootstrap/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,37 +75,26 @@ Also make sure that you've done the following:
[organization](https://cloud.google.com/resource-manager/docs/creating-managing-organization).
1. Set up a Google Cloud
[billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account).
1. Create Cloud Identity or Google Workspace groups for
organization and billing admins.
1. Add the user who will use Terraform to the `group_org_admins` group.
They must be in this group, or they won't have
`roles/resourcemanager.projectCreator` access.
1. Create Cloud Identity or Google Workspace groups as defined in [groups for access control](https://cloud.google.com/architecture/security-foundations/authentication-authorization#groups_for_access_control).
Set the variables in **terraform.tfvars** (`groups` block) to use the specific group names you create.
1. For the user who will run the procedures in this document, grant the following roles:
- The `roles/resourcemanager.organizationAdmin` role on the Google Cloud organization.
- The `roles/orgpolicy.policyAdmin` role on the Google Cloud organization.
- The `roles/resourcemanager.projectCreator` role on the Google Cloud organization.
- The `roles/billing.admin` role on the billing account.
- The `roles/resourcemanager.folderCreator` role.

If other users need to be able to run these procedures, add them to the group
represented by the `org_project_creators` variable.
For more information about the permissions that are required, and the resources
that are created, see the organization bootstrap module
[documentation.](https://github.com/terraform-google-modules/terraform-google-bootstrap)

### Optional - Automatic creation of Google Cloud Identity groups

In the foundation, Google Cloud Identity groups are used for [authentication and access management](https://cloud.google.com/architecture/security-foundations/authentication-authorization) .

To enable automatic creation of the [required groups](https://cloud.google.com/architecture/security-foundations/authentication-authorization#users_and_groups), complete the following actions:
To enable automatic creation of the [groups](https://cloud.google.com/architecture/security-foundations/authentication-authorization#groups_for_access_control), complete the following actions:

- Have an existing project for Cloud Identity API billing.
- Enable the Cloud Identity API (`cloudidentity.googleapis.com`) on the billing project.
- Grant role `roles/serviceusage.serviceUsageConsumer` to the user running Terraform on the billing project.
- Provide values for the groups and billing project in the variable `groups`.

All groups in the `groups.required_groups` are required.

All groups in the `groups.optional_groups` are optional.
- Change the field `groups.create_required_groups` to **true** to create the required groups.
- Change the field `groups.create_optional_groups` to **true** and fill the `groups.optional_groups` with the emails to be created.

### Optional - Cloud Build access to on-prem

Expand Down Expand Up @@ -305,13 +294,10 @@ Each step has instructions for this change.
| bucket\_tfstate\_kms\_force\_destroy | When deleting a bucket, this boolean option will delete the KMS keys used for the Terraform state bucket. | `bool` | `false` | no |
| default\_region | Default region to create resources where applicable. | `string` | `"us-central1"` | no |
| folder\_prefix | Name prefix to use for folders created. Should be the same in all steps. | `string` | `"fldr"` | no |
| group\_billing\_admins | Google Group for GCP Billing Administrators | `string` | n/a | yes |
| group\_org\_admins | Google Group for GCP Organization Administrators | `string` | n/a | yes |
| groups | Contain the details of the Groups to be created. | <pre>object({<br> create_groups = bool<br> billing_project = string<br> required_groups = object({<br> group_org_admins = string<br> group_billing_admins = string<br> billing_data_users = string<br> audit_data_users = string<br> monitoring_workspace_users = string<br> })<br> optional_groups = object({<br> gcp_platform_viewer = string<br> gcp_security_reviewer = string<br> gcp_network_viewer = string<br> gcp_scc_admin = string<br> gcp_global_secrets_admin = string<br> gcp_audit_viewer = string<br> })<br> })</pre> | <pre>{<br> "billing_project": null,<br> "create_groups": false,<br> "optional_groups": {<br> "gcp_audit_viewer": "",<br> "gcp_global_secrets_admin": "",<br> "gcp_network_viewer": "",<br> "gcp_platform_viewer": "",<br> "gcp_scc_admin": "",<br> "gcp_security_reviewer": ""<br> },<br> "required_groups": {<br> "audit_data_users": "",<br> "billing_data_users": "",<br> "group_billing_admins": "",<br> "group_org_admins": "",<br> "monitoring_workspace_users": ""<br> }<br>}</pre> | no |
| groups | Contain the details of the Groups to be created. | <pre>object({<br> create_required_groups = optional(bool, false)<br> create_optional_groups = optional(bool, false)<br> billing_project = optional(string, null)<br> required_groups = object({<br> group_org_admins = string<br> group_billing_admins = string<br> billing_data_users = string<br> audit_data_users = string<br> monitoring_workspace_users = string<br> })<br> optional_groups = optional(object({<br> gcp_security_reviewer = optional(string, "")<br> gcp_network_viewer = optional(string, "")<br> gcp_scc_admin = optional(string, "")<br> gcp_global_secrets_admin = optional(string, "")<br> gcp_kms_admin = optional(string, "")<br> }), {})<br> })</pre> | n/a | yes |
| initial\_group\_config | Define the group configuration when it is initialized. Valid values are: WITH\_INITIAL\_OWNER, EMPTY and INITIAL\_GROUP\_CONFIG\_UNSPECIFIED. | `string` | `"WITH_INITIAL_OWNER"` | no |
| org\_id | GCP Organization ID | `string` | n/a | yes |
| org\_policy\_admin\_role | Additional Org Policy Admin role for admin group. You can use this for testing purposes. | `bool` | `false` | no |
| org\_project\_creators | Additional list of members to have project creator role across the organization. Prefix of group: user: or serviceAccount: is required. | `list(string)` | `[]` | no |
| parent\_folder | Optional - for an organization with existing projects or for development/validation. It will place all the example foundation resources under the provided folder instead of the root organization. The value is the numeric folder ID. The folder must already exist. | `string` | `""` | no |
| project\_prefix | Name prefix to use for projects created. Should be the same in all steps. Max size is 3 characters. | `string` | `"prj"` | no |

Expand All @@ -332,8 +318,6 @@ Each step has instructions for this change.
| gcs\_bucket\_cloudbuild\_artifacts | Bucket used to store Cloud Build artifacts in cicd project. |
| gcs\_bucket\_cloudbuild\_logs | Bucket used to store Cloud Build logs in cicd project. |
| gcs\_bucket\_tfstate | Bucket used for storing terraform state for Foundations Pipelines in Seed Project. |
| group\_billing\_admins | Google Group for GCP Billing Administrators. |
| group\_org\_admins | Google Group for GCP Organization Administrators. |
| networks\_step\_terraform\_service\_account\_email | Networks Step Terraform Account |
| optional\_groups | List of Google Groups created that are optional to the Example Foundation steps. |
| organization\_step\_terraform\_service\_account\_email | Organization Step Terraform Account |
Expand Down
2 changes: 1 addition & 1 deletion 0-bootstrap/cb.tf
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ module "tf_source" {
project_id = "${var.project_prefix}-b-cicd-${random_string.suffix.result}"
location = var.default_region
billing_account = var.billing_account
group_org_admins = local.group_org_admins
group_org_admins = var.groups.required_groups.group_org_admins
buckets_force_destroy = var.bucket_force_destroy

activate_apis = [
Expand Down
12 changes: 6 additions & 6 deletions 0-bootstrap/groups.tf
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,18 @@
#Groups creation resources

locals {
optional_groups_to_create = {
for key, value in var.groups.optional_groups : key => value
if value != "" && var.groups.create_groups == true
}
required_groups_to_create = {
for key, value in var.groups.required_groups : key => value
if var.groups.create_groups == true
if var.groups.create_required_groups == true
}
optional_groups_to_create = {
for key, value in var.groups.optional_groups : key => value
if value != "" && var.groups.create_optional_groups == true
}
}

data "google_organization" "org" {
count = var.groups.create_groups ? 1 : 0
count = var.groups.create_required_groups || var.groups.create_optional_groups ? 1 : 0
organization = var.org_id
}

Expand Down
11 changes: 4 additions & 7 deletions 0-bootstrap/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,10 @@ locals {
"serviceAccount:${google_service_account.terraform-env-sa["net"].email}",
"serviceAccount:${google_service_account.terraform-env-sa["proj"].email}",
]
org_project_creators = distinct(concat(var.org_project_creators, local.step_terraform_sa))
parent = var.parent_folder != "" ? "folders/${var.parent_folder}" : "organizations/${var.org_id}"
parent = var.parent_folder != "" ? "folders/${var.parent_folder}" : "organizations/${var.org_id}"
org_admins_org_iam_permissions = var.org_policy_admin_role == true ? [
"roles/orgpolicy.policyAdmin", "roles/resourcemanager.organizationAdmin", "roles/billing.user"
] : ["roles/resourcemanager.organizationAdmin", "roles/billing.user"]
group_org_admins = var.groups.create_groups ? module.required_group["group_org_admins"].id : var.group_org_admins
group_billing_admins = var.groups.create_groups ? module.required_group["group_billing_admins"].id : var.group_billing_admins
}

resource "google_folder" "bootstrap" {
Expand All @@ -52,10 +49,10 @@ module "seed_bootstrap" {
state_bucket_name = "${var.bucket_prefix}-${var.project_prefix}-b-seed-tfstate"
force_destroy = var.bucket_force_destroy
billing_account = var.billing_account
group_org_admins = local.group_org_admins
group_billing_admins = local.group_billing_admins
group_org_admins = var.groups.required_groups.group_org_admins
group_billing_admins = var.groups.required_groups.group_billing_admins
default_region = var.default_region
org_project_creators = local.org_project_creators
org_project_creators = local.step_terraform_sa
sa_enable_impersonation = true
create_terraform_sa = false
parent_folder = var.parent_folder == "" ? "" : local.parent
Expand Down
14 changes: 2 additions & 12 deletions 0-bootstrap/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -63,24 +63,14 @@ output "common_config" {
}
}

output "group_org_admins" {
description = "Google Group for GCP Organization Administrators."
value = var.groups.create_groups == true ? module.required_group["group_org_admins"].id : var.group_org_admins
}

output "group_billing_admins" {
description = "Google Group for GCP Billing Administrators."
value = var.groups.create_groups == true ? module.required_group["group_billing_admins"].id : var.group_billing_admins
}

output "required_groups" {
description = "List of Google Groups created that are required by the Example Foundation steps."
value = var.groups.create_groups == true ? module.required_group : {}
value = var.groups.create_required_groups == true ? module.required_group : tomap(var.groups.required_groups)
}

output "optional_groups" {
description = "List of Google Groups created that are optional to the Example Foundation steps."
value = var.groups.create_groups == true ? module.optional_group : {}
value = var.groups.create_optional_groups == true ? module.optional_group : tomap(var.groups.optional_groups)
}

/* ----------------------------------------
Expand Down
7 changes: 7 additions & 0 deletions 0-bootstrap/sa.tf
Original file line number Diff line number Diff line change
Expand Up @@ -227,3 +227,10 @@ resource "google_billing_account_iam_member" "billing_admin_user" {
google_billing_account_iam_member.tf_billing_user
]
}

resource "google_billing_account_iam_member" "billing_account_sink" {
billing_account_id = var.billing_account
role = "roles/logging.configWriter"
member = "serviceAccount:${google_service_account.terraform-env-sa["org"].email}"
}

Loading