Skip to content

Commit 3d26ad7

Browse files
olix0rolix0r
authored
ci: Split out worflows (#1485)
This change splits out our GitHub workflows to run conditionally, depending on which files change in a pull request. * `build`: builds the proxy in release mode wh * `check-each`: checks that each Cargo.toml checks cleanly on its own. This used to be a serial task, but has been converted to a matrix job. * `deps`: Only runs on dependency changes to check cargo-deny * `integration`: Runs integration tests on code & dependency changes. * `lint`: Runs clippy, fmt, and doc on all source changes. Not run on dependency changes. * `test`: Runs unit tests on code and dependency changes This prevents doing needless work on dependency changes (as these are extremely common) and makes it easier to rerun narrower workflows, if necessary. The best part of this, though, is that we've made `check-each` a matrix job so that its tests can be parallelized. We'll update the github settings to make none of these checks strictly required. Signed-off-by: Oliver Gould <[email protected]>
1 parent 170762f commit 3d26ad7

File tree

8 files changed

+217
-118
lines changed

8 files changed

+217
-118
lines changed

‎.github/workflows/build.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Builds the proxy as if it were a release.
2+
name: build
3+
4+
permissions:
5+
contents: read
6+
7+
on:
8+
pull_request:
9+
paths:
10+
# We don't run this workflow on dependency changes. It's mainly intended to determine how long
11+
# a release build takes given Linkerd changes. We don't really need to run this on every
12+
# dependabot change, though.
13+
- "**/*.rs"
14+
- .github/workflows/build.yml
15+
16+
env:
17+
CARGO_INCREMENTAL: 0
18+
CARGO_NET_RETRY: 10
19+
RUST_BACKTRACE: short
20+
RUSTUP_MAX_RETRIES: 10
21+
22+
jobs:
23+
release:
24+
timeout-minutes: 20
25+
runs-on: ubuntu-latest
26+
steps:
27+
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
28+
- env:
29+
CARGO_RELEASE: "1"
30+
run: make build

‎.github/workflows/check-each.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Check each crate independently. Helps to catch dependency issues.
2+
name: check-each
3+
4+
permissions:
5+
contents: read
6+
7+
on:
8+
pull_request:
9+
paths:
10+
- Cargo.lock
11+
- "**/*.rs"
12+
- "**/Cargo.toml"
13+
- .github/workflows/check-each.yml
14+
15+
env:
16+
CARGO_INCREMENTAL: 0
17+
CARGO_NET_RETRY: 10
18+
RUST_BACKTRACE: short
19+
RUSTUP_MAX_RETRIES: 10
20+
21+
jobs:
22+
enumerate:
23+
timeout-minutes: 3
24+
runs-on: ubuntu-latest
25+
container:
26+
image: docker://rust:1.56.1-buster
27+
env:
28+
DEBIAN_FRONTEND: noninteractive
29+
steps:
30+
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
31+
- run: apt update && apt install -y jq
32+
- run: cargo fetch
33+
- name: list crates
34+
id: list-crates
35+
run: |
36+
echo "::set-output name=crates::$(cargo metadata --frozen --format-version=1 \
37+
| jq -cr "[.packages[] | select(.manifest_path | startswith(\"$PWD\")) | .name | select(. != \"linkerd-meshtls-boring\")]")"
38+
outputs:
39+
crates: ${{ steps.list-crates.outputs.crates }}`
40+
41+
check:
42+
needs: enumerate
43+
timeout-minutes: 20
44+
runs-on: ubuntu-latest
45+
container:
46+
image: docker://rust:1.56.1-buster
47+
strategy:
48+
matrix:
49+
crate: ${{ fromJson(needs.enumerate.outputs.crates) }}
50+
steps:
51+
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
52+
- run: cargo fetch
53+
- run: cargo check -p ${{ matrix.crate }} --frozen --all-targets
54+

‎.github/workflows/coverage.yml‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,24 @@
1-
name: Coverage
1+
name: coverage
2+
3+
permissions:
4+
contents: read
25

3-
# Run daily at 11AM UTC (3AM PST).
46
on:
7+
pull_request:
8+
paths:
9+
- .github/workflows/coverage.yml
510
schedule:
6-
# cron: minute hour day month day-of-week
11+
# Run daily at 11AM UTC (3AM PST).
712
- cron: '0 11 * * *'
813

914
env:
1015
CARGO_INCREMENTAL: 0
1116
CARGO_NET_RETRY: 10
1217
CARGO_TARPAULIN_VERSION: 0.18.5
18+
DEBIAN_FRONTEND: noninteractive
1319
RUST_BACKTRACE: short
1420
RUSTUP_MAX_RETRIES: 10
1521

16-
permissions:
17-
contents: read
18-
1922
jobs:
2023
test:
2124
name: codecov

‎.github/workflows/deps.yml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Audits dependencies with cargo-deny
2+
name: deps
3+
4+
permissions:
5+
contents: read
6+
7+
on:
8+
pull_request:
9+
paths:
10+
- Cargo.lock
11+
- .github/workflows/deps.yml
12+
13+
env:
14+
CARGO_INCREMENTAL: 0
15+
CARGO_NET_RETRY: 10
16+
RUST_BACKTRACE: short
17+
RUSTUP_MAX_RETRIES: 10
18+
19+
jobs:
20+
# Check for security advisories.
21+
#
22+
# Failures are not fatal, since issues are opened in the linkerd2 repo via rustsecbot.
23+
advisories:
24+
timeout-minutes: 10
25+
runs-on: ubuntu-latest
26+
continue-on-error: true
27+
steps:
28+
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
29+
- uses: EmbarkStudios/cargo-deny-action@4340bbf5bc9e7034fae7c4857e9ab87cab35c905
30+
with:
31+
command: check advisories
32+
33+
# Audit licenses, unreleased crates, and unexpected duplicate versions.
34+
bans:
35+
timeout-minutes: 10
36+
runs-on: ubuntu-latest
37+
steps:
38+
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
39+
- uses: EmbarkStudios/cargo-deny-action@4340bbf5bc9e7034fae7c4857e9ab87cab35c905
40+
with:
41+
command: check bans licenses sources

‎.github/workflows/slow.yml‎ renamed to ‎.github/workflows/integration.yml‎

Lines changed: 15 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -1,53 +1,28 @@
1-
# Each job typically runs for more than 5 minutes.
2-
name: slow
1+
# Run integration tests
2+
name: integration
3+
4+
permissions:
5+
contents: read
36

47
on:
5-
pull_request: {}
8+
pull_request:
9+
paths:
10+
- Cargo.lock
11+
- "**/*.rs"
12+
- "**/*.toml"
13+
- .github/workflows/integration.yml
614

715
env:
816
CARGO_INCREMENTAL: 0
917
CARGO_NET_RETRY: 10
1018
RUST_BACKTRACE: short
1119
RUSTUP_MAX_RETRIES: 10
1220

13-
permissions:
14-
contents: read
15-
21+
# Run only the app-level tests. These may take longer to compile (usually due to very large stack
22+
# types) and have the potential to be flakey as they depend on opening sockets and may have timing
23+
# sensitivity.
1624
jobs:
17-
18-
# Iterate through all (non-fuzzer) sub-crates to ensure each compiles independently.
19-
check-each-crate:
20-
timeout-minutes: 20
21-
runs-on: ubuntu-latest
22-
container:
23-
image: docker://rust:1.56.1-buster
24-
steps:
25-
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
26-
- run: |
27-
for toml in $(find . -mindepth 2 \
28-
-not -path '*/fuzz/*' \
29-
-not -path './linkerd/meshtls/boring/*' \
30-
-name Cargo.toml \
31-
| sort -r)
32-
do
33-
d="${toml%/*}"
34-
echo "# $d"
35-
(cd $d ; cargo check --all-targets)
36-
done
37-
38-
check-release:
39-
timeout-minutes: 20
40-
runs-on: ubuntu-latest
41-
steps:
42-
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
43-
- env:
44-
CARGO_RELEASE: "1"
45-
run: make build
46-
47-
# Run only the app-level tests. These may take longer to compile (usually due to very large stack
48-
# types) and have the potential to be flakey as they depend on opening sockets and may have timing
49-
# sensitivity.
50-
test-integration:
25+
test:
5126
timeout-minutes: 20
5227
runs-on: ubuntu-latest
5328
container:

‎.github/workflows/lint.yml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Each job should typically run in under 5 minutes.
2+
name: lint
3+
4+
permissions:
5+
contents: read
6+
7+
on:
8+
pull_request:
9+
paths:
10+
- "**/*.rs"
11+
- .github/workflows/lint.yml
12+
13+
env:
14+
CARGO_INCREMENTAL: 0
15+
CARGO_NET_RETRY: 10
16+
RUST_BACKTRACE: short
17+
RUSTUP_MAX_RETRIES: 10
18+
19+
jobs:
20+
clippy:
21+
timeout-minutes: 10
22+
runs-on: ubuntu-latest
23+
container:
24+
image: docker://rust:1.56.1-buster
25+
steps:
26+
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
27+
- run: rustup component add clippy
28+
- run: cargo clippy --all --exclude=linkerd-meshtls-boring
29+
30+
fmt:
31+
timeout-minutes: 10
32+
runs-on: ubuntu-latest
33+
container:
34+
image: docker://rust:1.56.1-buster
35+
steps:
36+
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
37+
- run: rustup component add rustfmt
38+
- run: make check-fmt
39+
40+
docs:
41+
timeout-minutes: 10
42+
runs-on: ubuntu-latest
43+
container:
44+
image: docker://rust:1.56.1-buster
45+
steps:
46+
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
47+
- run: |
48+
cargo doc --all --no-deps \
49+
--exclude=linkerd-meshtls \
50+
--exclude=linkerd-meshtls-boring \
51+
--exclude=linkerd-meshtls-rustls
52+

‎.github/workflows/release.yml‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Release
1+
name: release
22

33
on:
44
push:
@@ -68,7 +68,6 @@ jobs:
6868

6969
release:
7070
needs: [package]
71-
name: GitHub Release
7271
runs-on: ubuntu-latest
7372
timeout-minutes: 5
7473
permissions:
@@ -91,7 +90,7 @@ jobs:
9190
- name: display structure of downloaded files
9291
run: ls -R artifacts
9392

94-
- name: release
93+
- name: publish
9594
uses: softprops/action-gh-release@1e07f4398721186383de40550babbdf2b84acfc5
9695
env:
9796
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

0 commit comments

Comments
 (0)