Skip to content

Commit bb63475

Browse files
ksobrenat32ksobrenat32
authored
Add infra policy compliance checkers (#35848)
* Add IAM policy compliance checker and configuration files `infra/iam/generate` has been removed as it has been assimilated into `iam.py` * fix typo * standardize the keys yaml * Fix error on IAM not found * Add service account keys compliance checker * Improve readme by adding details, formats and Account Keys info * Modified account keys compliance check class to allow reading an empty keys.yaml * Change the project id to apache beam * Allow service accounts on terraform * Proposed changes to make the policy compliant It is adding some user permission changes and the service accounts roles. * Proposed account keys changes to make it compliant **Warning**: This commit modifies service account keys, clearing them and starting them as custom, modified the generated file to just include the keys you want to manage * Solution for issues found by gemini-code-assitant * Implement SendingClient for GitHub issue notifications and email alerts * Restore original configuration files * Add license to SendingClient * Implement a print announcement to avoid issues and email all together * Added announcement functionality in AccountKeys and IAM checkers - Added SendingClient integration for creating and printing compliance announcements. - Updated main execution flow to support new announcement actions. - Improved logging and error handling for announcement processes.
1 parent 8f888fb commit bb63475

File tree

10 files changed

+1342
-231
lines changed

10 files changed

+1342
-231
lines changed

‎infra/enforcement/README.md‎

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
<!--
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
20+
# Infrastructure rules enforcement
21+
22+
This module is used to check that the infrastructure rules are being used.
23+
24+
## IAM Policies
25+
26+
The enforcement is done by validating the IAM policies against the defined policies.
27+
The tool monitors and enforces compliance for user permissions, service account roles, and group memberships across your GCP project.
28+
29+
### Usage
30+
31+
You can specify the action either through the configuration file (`config.yml`) or via command-line arguments:
32+
33+
```bash
34+
# Check compliance and report issues (default)
35+
python iam.py --action check
36+
37+
# Create GitHub issue if compliance violations are found
38+
python iam.py --action issue
39+
40+
# Generate new compliance file based on current IAM policy
41+
python iam.py --action generate
42+
```
43+
44+
### Actions
45+
46+
- **check**: Validates IAM policies against defined policies and reports any differences (default behavior)
47+
- **issue**: Creates a GitHub issue when IAM policies differ from the defined ones, including detailed permission discrepancies
48+
- **generate**: Updates the compliance file to match the current GCP IAM policy, creating a new baseline from existing permissions
49+
50+
### Features
51+
52+
The IAM Policy enforcement tool provides the following capabilities:
53+
54+
- **Comprehensive Policy Export**: Automatically exports all IAM bindings and roles from the GCP project
55+
- **Member Type Recognition**: Handles users, service accounts, and groups with proper parsing and identification
56+
- **Permission Comparison**: Detailed comparison between expected and actual permissions for each user
57+
- **Conditional Role Filtering**: Automatically excludes conditional roles (roles with conditions) from compliance checks
58+
- **Sorted Output**: Provides consistent, sorted output for easy comparison and review
59+
- **Detailed Reporting**: Comprehensive reporting of permission differences with clear before/after comparisons
60+
- **GitHub Integration**: Automatic issue creation with detailed compliance violation reports
61+
62+
### Configuration
63+
64+
The `config.yml` file supports the following parameters for IAM policies:
65+
66+
- `project_id`: GCP project ID to check (default: `apache-beam-testing`)
67+
- `users_file`: Path to the YAML file containing expected IAM policies (default: `../iam/users.yml`)
68+
- `action`: Default action to perform (`check`, `issue`, or `generate`)
69+
- `logging`: Logging configuration (level and format)
70+
71+
### IAM Policy File Format
72+
73+
The IAM policy file should follow this YAML structure:
74+
75+
```yaml
76+
- username: john.doe
77+
email: [email protected]
78+
permissions:
79+
- role: roles/viewer
80+
- role: roles/storage.objectViewer
81+
- username: service-account-name
82+
email: [email protected]
83+
permissions:
84+
- role: roles/compute.instanceAdmin
85+
- role: roles/iam.serviceAccountUser
86+
```
87+
88+
Each user entry includes:
89+
- `username`: The derived username (typically the part before @ in email addresses)
90+
- `email`: The full email address of the user or service account
91+
- `permissions`: List of IAM roles assigned to this member
92+
- `role`: The full GCP IAM role name (e.g., `roles/viewer`, `roles/editor`)
93+
94+
### Compliance Checking Process
95+
96+
1. **Policy Extraction**: Retrieves current IAM policy from the GCP project
97+
2. **Member Parsing**: Parses all IAM members and extracts usernames, emails, and types
98+
3. **Role Processing**: Processes all roles while filtering out conditional bindings
99+
4. **Comparison**: Compares current permissions with expected permissions from the policy file
100+
5. **Reporting**: Generates detailed reports of any discrepancies found
101+
102+
Command-line arguments take precedence over configuration file settings.
103+
104+
## Account Keys
105+
106+
The enforcement is also done by validating service account keys and their access permissions against the defined policies.
107+
The tool supports three different actions when discrepancies are found:
108+
109+
### Usage
110+
111+
You can specify the action either through the configuration file (`config.yml`) or via command-line arguments:
112+
113+
```bash
114+
# Check compliance and report issues (default)
115+
python account_keys.py --action check
116+
117+
# Create GitHub issue if compliance violations are found
118+
python account_keys.py --action issue
119+
120+
# Generate new compliance file based on current service account keys policy
121+
python account_keys.py --action generate
122+
```
123+
124+
### Actions
125+
126+
- **check**: Validates service account keys and their permissions against defined policies and reports any differences (default behavior)
127+
- **issue**: Creates a GitHub issue when service account keys policies differ from the defined ones
128+
- **generate**: Updates the compliance file to match the current GCP service account keys and Secret Manager permissions
129+
130+
### Features
131+
132+
The Account Keys enforcement tool provides the following capabilities:
133+
134+
- **Service Account Discovery**: Automatically discovers all active (non-disabled) service accounts in the project
135+
- **Secret Manager Integration**: Monitors secrets created by the beam-infra-secret-manager service
136+
- **Permission Validation**: Ensures that Secret Manager permissions match the declared authorized users
137+
- **Compliance Reporting**: Identifies missing service accounts, undeclared managed secrets, and permission mismatches
138+
- **Automatic Remediation**: Can automatically update the compliance file to match current infrastructure state
139+
140+
### Configuration
141+
142+
The `config.yml` file supports the following parameters for account keys:
143+
144+
- `project_id`: GCP project ID to check
145+
- `service_account_keys_file`: Path to the YAML file containing expected service account keys policies (default: `../keys/keys.yaml`)
146+
- `action`: Default action to perform (`check`, `issue`, or `generate`)
147+
- `logging`: Logging configuration (level and format)
148+
149+
### Service Account Keys File Format
150+
151+
The service account keys file should follow this YAML structure:
152+
153+
```yaml
154+
service_accounts:
155+
- account_id: example-service-account
156+
display_name: [email protected]
157+
authorized_users:
158+
- email: [email protected]
159+
- email: [email protected]
160+
```
161+
162+
Each service account entry includes:
163+
- `account_id`: The unique identifier for the service account (without the full email domain)
164+
- `display_name`: The full service account email address or any custom display name
165+
- `authorized_users`: List of users who should have access to the service account's secrets

0 commit comments

Comments
 (0)