Skip to content

Commit 8a1b6b4

Browse files
mhuckapavoljuhas
andauthored
Rework CI workflow (#944)
Splitting up the previous workflows into separate files worked, but on balance, I find the disadvantages outweigh the advantages. The modularity of using separate files was somewhat good for reducing cognitive load and making maintenance a little bit easier, but it also decreased overall execution efficiency because it required more jobs overall and required more overhead. This new single file `ci.yaml` puts all the CI checks into one place, and adds some additional lint jobs compared to the previous set of workflow files. It also introduces a separate action for setting up Bazel that lets us control more aspects of the configuration. Note: this `ci.yaml` file does _not_ have the wheel-building steps that the previous setup had. The wheel builds and PyPI uploads will be reintroduced in an upcoming separate workflow. --------- Co-authored-by: Pavol Juhas <[email protected]>
1 parent 8319acd commit 8a1b6b4

24 files changed

+689
-1094
lines changed

.github/actionlint.yaml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Summary: config actionlint (https://github.com/rhysd/actionlint) for certain
16+
# things we use.
17+
18+
self-hosted-runner:
19+
# We don't have self-hosted runners, but we do use some of the "partner"
20+
# runner images from https://github.com/actions/partner-runner-images
21+
# and some runners that actionlint doesn't currently recognize.
22+
labels:
23+
- ubuntu-24.04-arm
24+
- ubuntu-slim
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
name: Set up Bazel with caching
16+
description: Installs Bazel and sets up multiple caches
17+
18+
# Summary: reusable workflow to install Bazelisk (for Bazel) and set up Bazel
19+
# caches. It grew out of trouble getting the existing bazel-contrib/setup-bazel
20+
# action (from the GitHub Marketplace) to work with Netkos' "act".
21+
22+
inputs:
23+
debug:
24+
description: 'Run with debugging options'
25+
type: boolean
26+
required: false
27+
default: true
28+
bazel-version:
29+
description: 'Version of Bazel to use:'
30+
type: string
31+
required: false
32+
default: ''
33+
34+
permissions:
35+
actions: write
36+
contents: read
37+
38+
runs:
39+
using: 'composite'
40+
steps:
41+
- name: Determine the version of Bazel to use
42+
shell: bash
43+
env:
44+
SHELLOPTS: ${{(inputs.debug || runner.debug) && 'xtrace' || ''}}
45+
run: |
46+
if [[ -n "${{inputs.bazel-version}}" ]]; then
47+
version="${{inputs.bazel-version}}"
48+
elif [[ -f ".bazelversion" ]]; then
49+
version="$(tr -d ' \n\r' <.bazelversion)"
50+
else
51+
echo "::error::Bazel version has not been specified."
52+
exit 1
53+
fi
54+
# This variable is read by Bazelisk.
55+
echo "USE_BAZEL_VERSION=${version}" >> "$GITHUB_ENV"
56+
57+
- name: Install Bazelisk
58+
env:
59+
BAZELISK_SHOW_PROGRESS: ${{inputs.debug || runner.debug}}
60+
shell: bash
61+
run: npm install @bazel/bazelisk
62+
63+
# This cache stores copies of Bazel downloaded by Bazelisk. The copies are
64+
# stored in subdirs coded by hash numbers, and thus this cache can be
65+
# shared between workflows.
66+
- name: Set up cache for Bazelisk
67+
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
68+
with:
69+
path: ~/.cache/bazelisk
70+
key: bazelisk-cache
71+
72+
# This cache stores downloaded files (like .zip, .tar.gz) specified by
73+
# repository rules like http_archive. This can be shared between workflows.
74+
- name: Set up Bazel repository cache
75+
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
76+
with:
77+
path: ~/.cache/bazel/repository_cache
78+
key: bazel-repo-${{env.USE_BAZEL_VERSION}}
79+
restore-keys: bazel-repo-
80+
81+
# This cache stores the compiled outputs of build actions, such as object
82+
# files (.o), linked libraries (.so), binaries, & test results.
83+
- name: Set up Bazel disk cache
84+
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
85+
with:
86+
path: ~/.cache/bazel/disk_cache
87+
# yamllint disable-line rule:line-length
88+
key: bazel-disk-${{runner.os}}-${{runner.arch}}-${{github.ref}}-${{hashFiles('**/WORKSPACE', '**/BUILD', '**/BUILD.tpl', '**/*.bzl')}}
89+
restore-keys: bazel-disk-${{runner.os}}-${{runner.arch}}-${{github.ref}}-
90+
91+
- name: Add configuration settings to .bazelrc
92+
shell: bash
93+
env:
94+
SHELLOPTS: ${{(inputs.debug || runner.debug) && 'xtrace' || ''}}
95+
run: |
96+
if [[ "${{startsWith(runner.os, 'win')}}" == "true" ]]; then
97+
num_cpus=${NUMBER_OF_PROCESSORS}
98+
else
99+
num_cpus=$(getconf _NPROCESSORS_ONLN)
100+
fi
101+
{
102+
echo "startup --output_base=~/.cache/bazel/output_base"
103+
echo "build --disk_cache=~/.cache/bazel/disk_cache"
104+
echo "build --repository_cache=~/.cache/bazel/repository_cache"
105+
echo "build --jobs=${num_cpus}"
106+
echo "test --cache_test_results=no"
107+
} >> .bazelrc

.github/problem-matchers/black.json

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,12 @@
22
"problemMatcher": [
33
{
44
"owner": "black",
5-
"severity": "error",
5+
"severity": "warning",
66
"pattern": [
77
{
8-
"regexp": "^would reformat (.+)$",
9-
"file": 1,
10-
"message": "File needs reformatting."
8+
"regexp": "^(would reformat\\s(.*))$",
9+
"message": 1,
10+
"file": 2
1111
}
1212
]
1313
}

.github/problem-matchers/hadolint.json

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,12 @@
44
"owner": "hadolint",
55
"pattern": [
66
{
7-
"regexp": "^(.+?):(\\d+)\\s+(.*)$",
7+
"regexp": "^(.*?):(\\d+) (DL\\d+|SC\\d+) (warning|error|info|style): (.*)$",
88
"file": 1,
99
"line": 2,
10-
"message": 3
10+
"code": 3,
11+
"severity": 4,
12+
"message": 5
1113
}
1214
]
1315
}

.github/workflows/_build_wheels.yaml

Lines changed: 0 additions & 121 deletions
This file was deleted.

.github/workflows/_find_changes.yaml

Lines changed: 0 additions & 95 deletions
This file was deleted.

0 commit comments

Comments
 (0)