Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -119,9 +119,9 @@ build:release --stamp
build:release --compilation_mode=opt

# As regular `release` but all the build work happens on the RBE
build:remote_release --config=release
build:remote_release --config=remote
build:remote_release --remote_download_toplevel
build:rbe_release --config=release
build:rbe_release --config=rbe
build:rbe_release --remote_download_toplevel

# RBE
import %workspace%/.bazelrc.remote
82 changes: 41 additions & 41 deletions .bazelrc.remote
Original file line number Diff line number Diff line change
@@ -1,68 +1,68 @@
# The RBE to use
build:remote --bes_results_url=https://gypsum.cluster.engflow.com/invocation
build:remote --bes_backend=grpcs://gypsum.cluster.engflow.com
build:remote --remote_executor=grpcs://gypsum.cluster.engflow.com
build:remote --remote_cache=grpcs://gypsum.cluster.engflow.com
build:rbe --bes_results_url=https://gypsum.cluster.engflow.com/invocation
build:rbe --bes_backend=grpcs://gypsum.cluster.engflow.com
build:rbe --remote_executor=grpcs://gypsum.cluster.engflow.com
build:rbe --remote_cache=grpcs://gypsum.cluster.engflow.com

# The number of cores available
build:remote -j 50
build:rbe -j 50

build:remote --define=EXECUTOR=remote
build:remote --experimental_inmemory_dotd_files
build:remote --experimental_inmemory_jdeps_files
build:remote --remote_timeout=3600
build:remote --spawn_strategy=remote,local
#build:remote --nolegacy_important_outputs
build:remote --incompatible_strict_action_env=true
build:rbe --define=EXECUTOR=remote
build:rbe --experimental_inmemory_dotd_files
build:rbe --experimental_inmemory_jdeps_files
build:rbe --remote_timeout=3600
build:rbe --spawn_strategy=remote,local
#build:rbe --nolegacy_important_outputs
build:rbe --incompatible_strict_action_env=true

build:remote --crosstool_top=//common/remote-build/cc:toolchain
build:remote --extra_execution_platforms=//common/remote-build:platform
build:remote --extra_toolchains=//common/remote-build:cc-toolchain
build:remote --host_platform=//common/remote-build:platform
build:remote --platforms=//common/remote-build:platform
build:remote --cxxopt=-std=c++14
build:rbe --crosstool_top=//common/remote-build/cc:toolchain
build:rbe --extra_execution_platforms=//common/remote-build:platform
build:rbe --extra_toolchains=//common/remote-build:cc-toolchain
build:rbe --host_platform=//common/remote-build:platform
build:rbe --platforms=//common/remote-build:platform
build:rbe --cxxopt=-std=c++14

# The Docker images are running Linux
build:remote --cpu=k8
build:remote --host_cpu=k8
build:rbe --cpu=k8
build:rbe --host_cpu=k8

build:remote --disk_cache=
build:rbe --disk_cache=

build:remote --incompatible_enable_cc_toolchain_resolution
build:remote --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
test:remote --test_env=DISPLAY=:99.0
test:remote --test_tag_filters=-exclusive-if-local,-skip-rbe,-remote
build:rbe --incompatible_enable_cc_toolchain_resolution
build:rbe --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
test:rbe --test_env=DISPLAY=:99.0
test:rbe --test_tag_filters=-exclusive-if-local,-skip-rbe,-remote

# Env vars we can hard code
build:remote --action_env=HOME=/home/dev
build:remote --action_env=PATH=/bin:/usr/bin:/usr/local/bin
test:remote --test_env=PATH=/bin:/usr/bin:/usr/local/bin
test:remote --test_env=HOME=/home/dev
build:rbe --action_env=HOME=/home/dev
build:rbe --action_env=PATH=/bin:/usr/bin:/usr/local/bin
test:rbe --test_env=PATH=/bin:/usr/bin:/usr/local/bin
test:rbe --test_env=HOME=/home/dev

# Make sure we sniff credentials properly
build:remote --credential_helper=gypsum.cluster.engflow.com=%workspace%/scripts/credential-helper.sh
build:rbe --credential_helper=gypsum.cluster.engflow.com=%workspace%/scripts/credential-helper.sh

# Use pinned browsers when running remotely
build:remote --//common:pin_browsers
build:rbe --//common:pin_browsers

# The remote build machines are pretty small, and 50 threads may leave them
# thrashing, but our dev machines are a lot larger. Scale the workload so we
# make reasonable usage of everything, everywhere, all at once.
build:remote --local_resources=cpu='HOST_CPUS*10'
build:remote --local_resources=memory='HOST_RAM*4.0'
build:rbe --local_resources=cpu='HOST_CPUS*10'
build:rbe --local_resources=memory='HOST_RAM*4.0'

# A small hint that we're running our tests remotely
test:remote --test_env=REMOTE_BUILD=1
test:rbe --test_env=REMOTE_BUILD=1

# Wait for up to 5 minutes for a test to pass
test:remote --test_timeout=600
test:rbe --test_timeout=600

# Extend the remote config for CI
build:remote-ci --config=remote
build:remote-ci --curses=no --color=yes --show_timestamps --show_progress_rate_limit=5
build:remote-ci --bes_upload_mode=wait_for_upload_complete
build:remote-ci --remote_download_minimal
build:rbe-ci --config=rbe
build:rbe-ci --curses=no --color=yes --show_timestamps --show_progress_rate_limit=5
build:rbe-ci --bes_upload_mode=wait_for_upload_complete
build:rbe-ci --remote_download_minimal

# Configuration changes suggested by EngFlow
build:remote --grpc_keepalive_time=30s
build:remote --nolegacy_important_outputs
build:rbe --grpc_keepalive_time=30s
build:rbe --nolegacy_important_outputs
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -358,8 +358,8 @@ Check https://selenium.dev/support/ for a complete list of options to communicat

To access the EngFlow RBE, a developer needs to be granted access to our project
container repository. Once that has been done, then any bazel command can be run
remotely by using `--config=remote`. For example: `bazel build --config=remote
grid` or `bazel test --config=remote java/test/...`
remotely by using `--config=rbe`. For example: `bazel build --config=rbe
grid` or `bazel test --config=rbe java/test/...`

When you run a remote build, one of the first lines of output from Bazel will
include a link to the EngFlow UI so you can track the progress of the build and
Expand Down
2 changes: 1 addition & 1 deletion Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -378,7 +378,7 @@ task ios_driver: [
# ./go java:package['--config=release']
desc 'Create stamped zipped assets for Java for uploading to GitHub'
task :'java-release-zip' do
Rake::Task['java:package'].invoke('--config=remote_release')
Rake::Task['java:package'].invoke('--config=rbe_release')
end

task 'release-java': %i[java-release-zip publish-maven]
Expand Down
4 changes: 2 additions & 2 deletions scripts/github-actions/ci-build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ set -x
# Now run the tests. The engflow build uses pinned browsers
# so this should be fine
# shellcheck disable=SC2046
bazel test --config=remote-ci --build_tests_only \
bazel test --config=rbe-ci --build_tests_only \
--keep_going --flaky_test_attempts=2 \
//... -- $(cat .skipped-tests | tr '\n' ' ')

# Build the packages we want to ship to users
bazel build --config=remote-ci --build_tag_filters=release-artifact //...
bazel build --config=rbe-ci --build_tag_filters=release-artifact //...
Loading