You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _site/docs/quick_start.md
+32-16Lines changed: 32 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ Here we describe how to use bazel remote caching or remote execution with buildf
10
10
11
11
## Setup
12
12
13
-
You can run this quick start on a single computer running nearly any flavor of linux. This computer is the localhost for the rest of the description.
13
+
You can run this quick start on a single computer running any flavor of linux that bazel supports. A C++ compiler is used here to demonstrate action execution. This computer is the localhost for the rest of the description.
14
14
15
15
### Backplane
16
16
@@ -44,33 +44,43 @@ cc_binary(
44
44
45
45
And an empty WORKSPACE file.
46
46
47
-
As a test, verify that `bazel run :main` builds your main program and runs it, and prints `Hello, World!`. This will ensure that you have properly installed bazel and a C++ compiler, and have a working target before moving on to remote execution.
47
+
As a test, verify that `bazel run :main` builds your main program and runs it, and prints `Hello, World!`. This will ensure that you have properly installed `bazel` and a C++ compiler, and have a working target before moving on to remote caching or remote execution.
48
48
49
49
Download and extract the buildfarm repository. Each command sequence below will have the intended working directory indicated, between the client (workspace running bazel), and buildfarm.
50
50
51
51
This tutorial assumes that you have a bazel binary in your path and you are in the root of your buildfarm clone/release, and has been tested to work with bash on linux.
52
52
53
53
## Remote Caching
54
54
55
-
A Buildfarm server with an instance can be used strictly as an ActionCache and ContentAddressableStorage to improve build performance. This is an example of running a bazel client that will retrieve results if available, and store them if the cache is missed and the execution needs to run locally.
55
+
A Buildfarm cluster can be used strictly as an ActionCache (AC) and ContentAddressableStorage (CAS) to improve build performance. This is an example of running a bazel client that will retrieve results if available, otherwise store them on a cache miss after executing locally.
56
56
57
57
Download the buildfarm repository and change into its directory, then:
58
58
59
-
run `bazelisk run src/main/java/build/buildfarm:buildfarm-server $PWD/examples/config.minimal.yml`
59
+
*run `bazel run src/main/java/build/buildfarm:buildfarm-server $PWD/examples/config.minimal.yml`
60
60
61
61
This will wait while the server runs, indicating that it is ready for requests.
62
62
63
-
From another prompt (i.e. a separate terminal) in your newly created workspace directory from above:
63
+
A server alone does not itself store the content of action results. It acts as an endpoint for any number of workers that present storage, so we must also start a single worker.
64
64
65
-
run `bazel clean`
66
-
run `bazel run --remote_cache=grpc://localhost:8980 :main`
65
+
From another prompt (i.e. a separate terminal) in the buildfarm repository directory:
66
+
67
+
* run `bazel run src/main/java/build/buildfarm:buildfarm-shard-worker -- --prometheus_port=9091 $PWD/examples/config.minimal.yml`
68
+
69
+
The `--` option is bazel convention to treat all subsequent arguments as parameters to the running app, like our `--prometheus_port`, instead of interpreting them with `run`
70
+
The `--prometheus_port=9091` option allows this worker to run alongside our server, who will have started and logged that it has started a service on port `9090`. You can also turn this option off (with `--` separator), with `--prometheus_option=0` for either server or worker.
71
+
This will also wait while the worker runs, indicating it will be available to store cache content.
72
+
73
+
From another prompt in your newly created workspace directory from above:
74
+
75
+
* run `bazel clean`
76
+
* run `bazel run --remote_cache=grpc://localhost:8980 :main`
67
77
68
78
Why do we clean here? Since we're verifying re-execution and caching, this ensures that we will execute any actions in the `run` step and interact with the remote cache. We should be attempting to retrieve cached results, and then when we miss - since we just started this memory resident server - bazel will upload the results of the execution for later use. There will be no change in the output of this bazel run if everything worked, since bazel does not provide output each time it uploads results.
69
79
70
80
To prove that we have placed something in the action cache, we need to do the following:
71
81
72
-
run `bazel clean`
73
-
run `bazel run --remote_cache=localhost:8980 :main`
82
+
*run `bazel clean`
83
+
*run `bazel run --remote_cache=localhost:8980 :main`
74
84
75
85
This should now print statistics on the `processes` line that indicate that you've retrieved results from the cache for your actions:
Now we will use buildfarm for remote execution with a minimal configuration - a single memory instance, with a worker on the localhost that can execute a single process at a time - via a bazel invocation on our workspace.
93
+
Now we will use buildfarm for remote execution with a minimal configuration with a worker on the localhost that can execute a single process at a time, via a bazel invocation on our workspace.
84
94
85
-
First, we should restart the buildfarm server to ensure that we get remote execution (this can also be forced from the client by using `--noremote_accept_cached`). From the buildfarm server prompt and directory:
95
+
First, to clean out the results from the previous cached actions, flush your local redis database:
86
96
87
-
interrupt a running `buildfarm-server`
88
-
run `bazelisk run src/main/java/build/buildfarm:buildfarm-server $PWD/examples/config.minimal.yml`
97
+
* run `redis-cli flushdb`
89
98
90
-
From another prompt in the buildfarm repository directory:
99
+
Next, we should restart the buildfarm server, and delete the worker's cas storage to ensure that we get remote execution (this can also be forced from the client by using `--noremote_accept_cached`). From the buildfarm server prompt and directory:
91
100
92
-
run `bazelisk run src/main/java/build/buildfarm:buildfarm-shard-worker $PWD/examples/config.minimal.yml`
101
+
* interrupt the running `buildfarm-server` (i.e. Ctrl-C)
102
+
* run `bazel run src/main/java/build/buildfarm:buildfarm-server $PWD/examples/config.minimal.yml`
103
+
104
+
You can leave the worker running from the Remote Caching step, it will not require a restart
93
105
94
106
From another prompt, in your client workspace:
95
107
96
-
run `bazel run --remote_executor=grpc://localhost:8980 :main`
108
+
*run `bazel run --remote_executor=grpc://localhost:8980 :main`
97
109
98
110
Your build should now print out the following on its `processes` summary line:
99
111
@@ -117,6 +129,10 @@ To stop the containers, run:
117
129
./examples/bf-run stop
118
130
```
119
131
132
+
## Next Steps
133
+
134
+
We've started our worker on the same host as our server, and also the same host on which we built with bazel, but these services can be spread across many machines, per 'remote'. A large number of workers, with a relatively small number of servers (10:1 and 100:1 ratios have been used in practice), consolidating large disks and beefy multicore cpus/gpus on workers, with specialization of what work they perform for bazel builds (or other client work), and specializing servers to have hefty network connections to funnel content traffic. A buildfarm deployment can service hundreds or thousands of developers or CI processes, enabling them to benefit from each others' shared context in the AC/CAS, and the pooled execution of a fleet of worker hosts eager to consume operations and deliver results.
135
+
120
136
## Buildfarm Manager
121
137
122
138
You can now easily launch a new Buildfarm cluster locally or in AWS using an open sourced [Buildfarm Manager](https://github.com/80degreeswest/bfmgr).
0 commit comments