You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/user/Bazel-caching.md
+51-60Lines changed: 51 additions & 60 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,103 +6,94 @@ This document provides instructions for developers on how to use Bazel's caching
6
6
7
7
Our Bazel setup uses a multi-layered caching strategy to provide optimal performance for different types of users:
8
8
9
-
1.**Local Cache (Default):** All builds use a local on-disk cache by default. This provides a speed-up for incremental builds without requiring any special configuration or authentication.
10
-
2.**Remote Cache (OpenROAD Developers & CI):** We maintain a remote cache on Google Cloud Storage. This allows team members and CI jobs to share build artifacts, dramatically reducing build times.
9
+
1.**Local Cache (Default):** All builds use a local on-disk cache by default. This provides a speed-up for incremental builds without requiring any special configuration.
10
+
2.**Remote Cache:** We maintain a remote cache server at `https://bazel.precisioninno.com`. The contents of the cache are generated by the CI for all branches. This allows anyone that wishes to build OpenROAD locally and CI jobs to share build artifacts, dramatically reducing build times. All non-CI builds have **read-only** access to the remote cache.
11
11
12
12
---
13
13
14
14
### 1. Local On-Disk Cache
15
15
16
-
By default, all builds use a local on-disk cache to speed up incremental builds. The default location is `~/.cache/bazel-disk-cache`.
16
+
All builds use a local on-disk cache to speed up incremental builds. The default location is `~/.cache/bazel-disk-cache` and is set inside the repo's `.bazelrc` file.
17
17
18
-
You can override this location by setting the `OPENROAD_BAZEL_CACHE` environment variable:
18
+
You can override this config by setting a systemwide or home `.bazelrc` file. See more at [Write bazelrc configuration files](https://bazel.build/run/bazelrc) or by passing the `--disk_cache` argument to your bazel commands.
19
19
20
-
```bash
21
-
export OPENROAD_BAZEL_CACHE=/path/to/your/cache
22
-
bazel build //...
23
-
```
24
-
25
-
No remote cache is accessed, and no authentication is required for the local cache. This provides a seamless experience for external contributors.
26
-
Secure VMs already have this set up, so no additional configuration is needed.
20
+
#### Using a `user.bazelrc` file
27
21
28
-
---
22
+
For more advanced customization, you can create a `user.bazelrc` file in the root of the repository. This file is ignored by Git and can be used to override any default settings. For example, you could use it to specify a different remote cache or to enable features for your local builds.
Team members can enable **read-only** access to the remote cache by using the `openroad-dev` configuration. This will dramatically speed up local builds by downloading artifacts already built by CI.
For a full list of command-line options, refer to the [Bazel Command-Line Reference](https://bazel.build/reference/command-line-reference).
35
33
36
-
If you are working on a secure development VM provided by OpenROAD, you do **not** need to authenticate. The VM's service account already has the necessary read access to the cache.
34
+
---
37
35
38
-
#### On a Local Environment
36
+
###2. Remote Cache Access
39
37
40
-
If you are working on your local machine, you must first authenticate with Google Cloud using your @openroad.tools account. All OpenROAD team members have read access to the remote cache.
38
+
All builds are configured to use our remote cache for shared build artifacts. This is configured in the main `.bazelrc` file with the following settings:
After authenticating (if necessary), run your build with the `--config=openroad-dev` flag:
46
+
This means that all developer builds will **read from**the public remote cache but **cannot write to** it. This provides significant build speed improvements without requiring any authentication.
47
47
48
-
```bash
49
-
bazel build --config=openroad-dev //...
50
-
```
48
+
#### Disabling the Remote Cache
51
49
52
-
This configuration is **read-only** to prevent local, unverified builds from populating the shared cache.
50
+
If you encounter issues with the remote cache, you can disable it by overriding the `--remote_cache` flag. You can do this in your `user.bazelrc` file or by passing the flag directly to your Bazel command:
53
51
54
-
### Using a `user.bazelrc` file instead of `--config=openroad-dev`
52
+
**Option 1: Using `user.bazelrc`**
55
53
56
-
You may prefer `user.bazelrc` file instead of using the `--config=openroad-dev` option.
54
+
Add the following line to your `user.bazelrc` file:
This will cause Bazel to fall back to using only the local on-disk cache for the build
76
69
---
77
70
78
71
### 3. CI Access (Jenkins Pipeline)
79
72
80
-
The Jenkins pipeline uses a unified caching strategy for all builds. This is primarily for informational purposes, as developers will not typically use the `ci` configuration locally.
73
+
The Jenkins pipeline uses a specific configuration for its builds. This is primarily for informational purposes, as developers will not typically use the `ci` configuration locally.
81
74
82
-
*All CI builds use the `--config=ci` profile.
83
-
* Builds on the `master` branch have**read/write** access, populating the remote cachewith the latest artifacts.
84
-
* Builds on PR branches are set to **read-only**.
75
+
* All CI builds use the `--config=ci` profile.
76
+
*The `ci` config enables**read/write** access to the remote cache, populating it with the latest artifacts for **all branches**.
77
+
*It also disables the local disk cache for CI builds to ensure clean builds.
85
78
86
-
---
87
-
88
-
## `.bazelrc` Configurations Summary
79
+
Here are the relevant settings from `.bazelrc`:
89
80
90
-
The caching behavior is controlled by two new configurations in the `.bazelrc` file:
81
+
```
82
+
build:ci --remote_download_minimal
83
+
build:ci --remote_upload_local_results=true
84
+
build:ci --disk_cache=
85
+
```
91
86
92
-
***`build:ci` (Write Access):**
93
-
*`--remote_upload_local_results=true`: **Enables writing** to the remote cache.
94
-
* Uses service account credentials provided by Jenkins.
87
+
---
95
88
96
-
***`build:openroad-dev` (Read-Only Access):**
97
-
-`--remote_upload_local_results=false`: **Disables writing**, making the cache read-only.
98
-
-`--google_default_credentials=true`: Uses the developer's local `gcloud` credentials.
89
+
## "Build without the Bytes" (BwoB)
99
90
100
-
Both configurations include performance optimizations to reduce network traffic and improve build speeds. The primary mechanism for this is the "Build without the Bytes" (BwoB) feature, which minimizes the download of build artifacts from the remote cache.
91
+
Our configurations include performance optimizations to reduce network traffic and improve build speeds using the "Build without the Bytes" (BwoB) feature, which minimizes the download of build artifacts from the remote cache.
101
92
102
93
There are two main BwoB settings:
103
94
104
-
*`--remote_download_toplevel`: This is the default setting in Bazel. It downloads only the outputs of the top-level targets you specify in your build command. This is ideal for interactive development, where you need to use the final build artifacts (e.g., run a binary or inspect a generated file).
95
+
*`--remote_download_toplevel`: This is the default setting in Bazel. It downloads only the outputs of the top-level targets you specify in your build command. This is ideal for interactive development, where you need to use the final build artifacts (e.g., run a binary or inspect a generated file).
105
96
106
-
*`--remote_download_minimal`: This is a more aggressive setting that downloads only the artifacts essential for the build to continue. It is primarily intended for CI environments, where you are only concerned with the success or failure of a build, not the artifacts themselves.
97
+
*`--remote_download_minimal`: This is a more aggressive setting that downloads only the artifacts essential for the build to continue. It is primarily intended for CI environments, where you are only concerned with the success or failure of a build, not the artifacts themselves.
107
98
108
-
Our configurations use `--remote_download_toplevel` for developer builds (`openroad-dev`) and `--remote_download_minimal` for CI builds (`ci`) to provide the best balance of performance and usability for each environment.
99
+
Our configurations use the default (`--remote_download_toplevel`) for developer builds and `--remote_download_minimal` for CI builds (`ci`) to provide the best balance of performance and usability for each environment.
0 commit comments