Skip to content

Commit 04dba9c

Browse files
Android Build Filesystem (ABFS) Teamsce-taid
authored andcommitted
updating documentation
PiperOrigin-RevId: 809979554
1 parent 606440a commit 04dba9c

File tree

4 files changed

+214
-68
lines changed

4 files changed

+214
-68
lines changed

examples/pre-start-hooks/README.md

Lines changed: 44 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,60 +1,86 @@
11
# Use pre-start hooks to modify the ABFS container before launching ABFS
22

3-
This example shows how to use pre-start hooks to run custom scripts inside the ABFS (Android Build File System) uploader container before the main ABFS service starts.
3+
This example shows how to use pre-start hooks to run custom scripts inside the
4+
ABFS (Android Build File System) uploader container before the main ABFS service
5+
starts.
46

5-
Pre-start hooks are useful for preparing the container environment. For example, you can use them to:
7+
Pre-start hooks are useful for preparing the container environment. For example,
8+
you can use them to:
69

710
* Fetch credentials or secrets required to use your Android source.
811
* Install additional packages or dependencies.
912

1013
## How it works
1114

12-
The `abfs_uploaders` Terraform module accepts a `pre_start_hooks` variable, which is a path to a local directory containing one or more scripts.
15+
The `abfs_uploaders` Terraform module accepts a `pre_start_hooks` variable,
16+
which is a path to a local directory containing one or more scripts.
1317

1418
When you run the `terraform apply` command:
1519

16-
1. Terraform uses cloud-init to copy the scripts from the specified directory to the ABFS uploader host VMs.
17-
2. This directory is then mounted to the ABFS Docker container as a read-only volume at `/etc/abfs/pre-start.d`.
18-
3. An entrypoint script inside the container checks for this directory. Before starting the main ABFS service, it executes any executable scripts (<code>+x</code>) it finds there.
19-
4. The entrypoint script executes the scripts in alphabetical order. The entrypoint script waits for each hook to complete before proceeding to the next one. If any hook script fails (returns a non-zero exit code), the container startup stops.
20+
1. Terraform uses cloud-init to copy the scripts from the specified directory
21+
to the ABFS uploader host VMs.
22+
2. This directory is then mounted to the ABFS Docker container as a read-only
23+
volume at `/etc/abfs/pre-start.d`.
24+
3. An entrypoint script inside the container checks for this directory.
25+
Before starting the main ABFS service, it executes any executable scripts
26+
(<code>+x</code>) it finds there.
27+
4. The entrypoint script executes the scripts in alphabetical order. The
28+
entrypoint script waits for each hook to complete before proceeding to the
29+
next one. If any hook script fails (returns a non-zero exit code), the
30+
container startup stops.
2031

2132
## This example: Fetching a Git Token
2233

23-
This example provides a practical pre-start hook script, `pre-start.d/fetch-git-token.sh`, that securely fetches a Git token from Google Cloud Secret Manager and configures Git for `github.com` authentication. This is useful if your Android build needs to clone private repositories from GitHub.
34+
This example provides a practical pre-start hook script,
35+
`pre-start.d/fetch-git-token.sh`, that securely fetches a Git token from
36+
Google Cloud Secret Manager and configures Git for `github.com` authentication.
37+
This is useful if your Android build needs to clone private repositories from
38+
GitHub.
2439

2540
### The hook script: `pre-start.d/fetch-git-token.sh`
2641

2742
This script performs the following steps:
2843

29-
1. Determine the Google Cloud project number, either from a `PROJECT_NUMBER` environment variable or by querying the Google Cloud metadata server.
30-
2. Retrieve an access token for the VM's service account from the metadata server.
31-
3. Use this access token to call the `Secret Manager API` and retrieve a secret. By default, it looks for a secret named `GIT_TOKEN`.
32-
4. Configure Git to use the retrieved secret as an authentication token for all HTTPS requests to `github.com`.
44+
1. Determine the Google Cloud project number, either from a `PROJECT_NUMBER`
45+
environment variable or by querying the Google Cloud metadata server.
46+
2. Retrieve an access token for the VM's service account from the metadata
47+
server.
48+
3. Use this access token to call the `Secret Manager API` and retrieve a
49+
secret. By default, it looks for a secret named `GIT_TOKEN`.
50+
4. Configure Git to use the retrieved secret as an authentication token for all
51+
HTTPS requests to `github.com`.
3352

3453
### Terraform Configuration: `main.tf`
3554

36-
The `main.tf` file in this example shows how to use the `pre_start_hooks` variable:
55+
The `main.tf` file in this example shows how to use the `pre_start_hooks`
56+
variable:
3757

3858
```
3959
module "abfs_uploaders" {
40-
source = "github.com/terraform-google-modules/terraform-google-abfs//modules/uploaders?ref=v0.8.0"
60+
source = "../modules/uploaders"
4161
4262
# Other configuration here
4363
pre_start_hooks = "${path.module}/pre-start.d"
4464
}
4565
```
4666

47-
This configuration tells Terraform to package the contents of the local `pre-start.d` directory and make them available to the ABFS uploader containers.
67+
This configuration tells Terraform to package the contents of the local
68+
`pre-start.d` directory and make them available to the ABFS uploader containers.
4869

4970
## How to use this example
5071

5172
1. Store your Git token in Secret Manager:
52-
* Create a new secret in Google Cloud Secret Manager (for example, named `GIT_TOKEN`).
73+
* Create a new secret in Google Cloud Secret Manager (for example, named `GIT_TOKEN`).
5374
* Add your GitHub personal access token as the secret value.
5475
2. Grant permissions:
55-
* Ensure the service account used by the ABFS uploader VMs has the **Secret Manager Secret Accessor** (`roles/secretmanager.secretAccessor`) IAM role for the secret you just created.
76+
* Ensure the service account used by the ABFS uploader VMs has the
77+
**Secret Manager Secret Accessor** (`roles/secretmanager.secretAccessor`)
78+
IAM role for the secret you just created.
5679
3. Deploy with Terraform:
5780
* Customize the `main.tf` file with your project details, such as project ID and zone, similar to the simple example.
5881
* Run the `terraform init` and `terraform apply` commands.
5982

60-
After the ABFS uploader instances start, the hook script automatically runs, and your ABFS uploaders can authenticate against your private GitHub repositories. You can check the container logs to see the output from the hook script and verify that it ran successfully.
83+
After the ABFS uploader instances start, the hook script automatically runs, and
84+
your ABFS uploaders can authenticate against your private GitHub repositories.
85+
You can check the container logs to see the output from the hook script and
86+
verify that it ran successfully.

examples/simple/README.md

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# ABFS Environment Blueprint
2+
3+
This blueprint deploys a complete Android Build Filesystem (ABFS) environment on
4+
Google Cloud. It provides a scalable and secure solution for building Android
5+
and is ideal for teams looking to modernize their Android development
6+
infrastructure. The blueprint also includes a sophisticated CI/CD pipeline for
7+
creating and maintaining custom developer environments using Cloud Workstations.
8+
9+
## Deployed Resources
10+
11+
This blueprint will deploy the following key resources to stand up a fully
12+
functional ABFS environment:
13+
14+
- **Spanner Instance and Database**: A fully managed, mission-critical,
15+
relational database service that provides a scalable and highly available
16+
backend for ABFS.
17+
- **ABFS Server**: The core component of the Android Build Filesystem,
18+
deployed on Google Compute Engine (GCE).
19+
- **ABFS Uploaders**: Services responsible for uploading build artifacts into
20+
the ABFS environment.
21+
- **Optional ABFS Client**: A Compute Engine instance can be optionally
22+
created to act as a client for interacting with and testing the ABFS setup.
23+
24+
## ABFS Licensing and Deployment Flow
25+
26+
Deploying the ABFS environment involves a two-step process due to licensing
27+
requirements:
28+
29+
1. **Initial `terraform apply`**: Run `terraform apply` for the first time.
30+
This will provision the necessary project infrastructure and output the
31+
license information.
32+
33+
2. **Submit EAP Form**: Use the license information from the Terraform output
34+
to fill out the EAP (Early Access Program) form provided by the Google team.
35+
This will initiate the licensing process for your service account.
36+
37+
3. **Update `terraform.tfvars`**: Once you receive the license key, add it to
38+
your `terraform.tfvars` file. For example:
39+
```hcl
40+
abfs_license = "your-license-key-here"
41+
```
42+
43+
4. **Final `terraform apply`**: Run `terraform apply` a second time. With the
44+
license key in place, Terraform will now deploy and start all the ABFS
45+
components, including the server and uploaders.
46+
47+
5. **Seeding and Building**: After the deployment is complete, the ABFS
48+
uploaders will begin seeding the environment with data from the Gerrit
49+
server. Once this process is finished, you can use the ABFS client to mount
50+
the filesystem and start your first Android build.
51+
52+
## Automated Custom Images for Cloud Workstations
53+
54+
A key feature of this blueprint is its ability to create and manage custom
55+
container images for
56+
[Cloud Workstations](https://cloud.google.com/workstations). This allows you to
57+
provide developers with tailored, up-to-date, and secure development
58+
environments like **Android Studio (AS)**,
59+
**Android Studio for Platform (ASfP)**, and more.
60+
61+
The blueprint sets up a complete, GitOps-driven CI/CD pipeline to automate the
62+
build, deployment, and maintenance of these custom images.
63+
64+
### The CI/CD Workflow for Custom Images
65+
66+
The process is designed to be fully automated after the initial setup:
67+
68+
1. **Initial Deployment (`terraform apply`)**:
69+
When you first apply the Terraform configuration for this blueprint, it
70+
provisions the entire CI/CD foundation, including:
71+
* A **Secure Source Manager (SSM)** repository to host the source code for
72+
your custom images.
73+
* A **Cloud Build trigger** configured with a webhook to automatically
74+
start a build on a `git push` to the SSM repository.
75+
* An **Artifact Registry** repository to securely store the built
76+
container images.
77+
* **Cloud Workstations configurations** that are pre-configured to use the
78+
`:latest` tag of your custom images.
79+
* A **Cloud Scheduler** job to periodically rebuild the images.
80+
81+
2. **Adding Image Source Code**:
82+
After the infrastructure is deployed, you push the source code for your
83+
custom workstation images to the repository. An example
84+
repository with Dockerfiles and build configurations can be found here:
85+
[Android Open Source Project Images](https://github.com/GoogleCloudPlatform/cloud-workstations-custom-image-examples/tree/main/examples/images/android-open-source-project).
86+
```bash
87+
# Configure the gcloud helper for Secure Source Manager
88+
git config --global credential.'https://*.*.sourcemanager.dev'.helper gcloud.sh
89+
90+
# Clone the example repository
91+
git clone https://github.com/GoogleCloudPlatform/cloud-workstations-custom-image-examples.git
92+
cd cloud-workstations-custom-image-examples/examples/images/android-open-source-project
93+
94+
# Add your repository as a remote
95+
git remote add private $(terraform output -raw secure_source_manager_repository_git_https)
96+
97+
# Push the code to the main branch
98+
git push private main
99+
```
100+
101+
3. **Automated Build and Deployment**:
102+
The `git push` automatically triggers the Cloud Build pipeline via a
103+
webhook. Cloud Build:
104+
* Builds the container image from your Dockerfile.
105+
* Pushes the newly built image to Artifact Registry.
106+
* Tags the image as `:latest`.
107+
108+
4. **Periodic Rebuilds for Security**:
109+
The Cloud Scheduler job runs on a defined schedule (e.g., nightly) and
110+
triggers the same Cloud Build pipeline. This is critical for security and
111+
maintenance, as it ensures your custom images are regularly rebuilt on top
112+
of their base layers. This process automatically incorporates the latest OS
113+
updates and security patches into your development environments without any
114+
manual intervention.
115+
116+
5. **Seamless User Experience**:
117+
When a developer starts a new Cloud Workstation, it automatically pulls the
118+
`:latest` image tag from Artifact Registry. Because of the automated build
119+
and periodic rebuild process, they are always launching a workstation that
120+
is up-to-date, secure, and contains the preinstalled tools they need.

0 commit comments

Comments
 (0)