Skip to content

Commit ed12950

Browse files
arman-bdclaude
andcommitted
perf: cache BoringSSL builds across Python versions on Linux
Optimize Linux wheel builds to reduce time from ~40 minutes to ~10 minutes by reusing BoringSSL builds across Python versions. Changes: 1. Modified scripts/linux/setup_vendors.sh to check for existing built libraries BEFORE removing the directory, allowing cibuildwheel to reuse builds across Python versions 2. The script now: - Checks if boringssl/build contains valid libssl.a and libcrypto.a - Only removes and rebuilds if libraries are missing or invalid - Skips expensive 5-10 minute BoringSSL build for Python versions 2-7 3. Updated cache key to v11 to ensure clean state with new logic How it works: - First Python version (cp38) builds BoringSSL (~5-10 min) - Subsequent Python versions (cp39-cp314) reuse the build (~10 sec each) - Total build time: ~10-15 min instead of ~40 min The vendor directory is mounted into each cibuildwheel manylinux container, so builds persist across Python versions within the same CI run. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 76a8c16 commit ed12950

File tree

2 files changed

+19
-25
lines changed

2 files changed

+19
-25
lines changed

.github/workflows/_build_linux.yml

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,32 +27,33 @@ jobs:
2727
cache: true
2828

2929
# Cache vendor dependencies to speed up builds
30-
# Note: For Linux, vendors are built inside manylinux container, so we cache the vendor directory
3130
- name: Restore vendor cache
3231
id: cache-vendor
3332
uses: actions/cache/restore@v4
3433
with:
3534
path: vendor
36-
key: vendor-linux-manylinux-${{ hashFiles('scripts/linux/setup_vendors.sh') }}-v10
35+
key: vendor-linux-manylinux-${{ hashFiles('scripts/linux/setup_vendors.sh') }}-v11
3736
restore-keys: |
38-
vendor-linux-manylinux-${{ hashFiles('scripts/linux/setup_vendors.sh') }}-v10
37+
vendor-linux-manylinux-${{ hashFiles('scripts/linux/setup_vendors.sh') }}-v11
3938
4039
# Build wheels with cibuildwheel (handles manylinux containers)
41-
# The vendor directory is mounted into the container automatically
40+
# The vendor directory is mounted into the container and shared across Python versions
41+
# The setup script will skip rebuilding if libraries already exist
4242
- name: Build wheels
4343
uses: pypa/cibuildwheel@v2.22.0
4444
env:
4545
# Build for specified Python versions
4646
CIBW_BUILD: cp38-manylinux_x86_64 cp39-manylinux_x86_64 cp310-manylinux_x86_64 cp311-manylinux_x86_64 cp312-manylinux_x86_64 cp313-manylinux_x86_64 cp314-manylinux_x86_64
4747
# Vendor build happens inside manylinux container via before-build (from pyproject.toml)
48+
# The script will detect cached libraries and skip rebuilding
4849

4950
# Save vendor cache after build
5051
- name: Save vendor cache
5152
if: steps.cache-vendor.outputs.cache-hit != 'true'
5253
uses: actions/cache/save@v4
5354
with:
5455
path: vendor
55-
key: vendor-linux-manylinux-${{ hashFiles('scripts/linux/setup_vendors.sh') }}-v10
56+
key: vendor-linux-manylinux-${{ hashFiles('scripts/linux/setup_vendors.sh') }}-v11
5657

5758
# Setup Python versions for testing
5859
- name: Setup Python versions

scripts/linux/setup_vendors.sh

Lines changed: 13 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -28,20 +28,20 @@ cd "$VENDOR_DIR"
2828
#
2929
echo "==> Setting up BoringSSL..."
3030

31-
# Always remove and re-clone BoringSSL to ensure clean state
32-
# This prevents platform cross-contamination (e.g., macOS files on Linux runners)
33-
if [ -d "boringssl" ]; then
34-
echo "Removing existing BoringSSL directory to ensure clean build..."
35-
rm -rf boringssl
36-
fi
37-
38-
echo "Cloning BoringSSL..."
39-
git clone --depth 1 https://boringssl.googlesource.com/boringssl
31+
# Check if BoringSSL is already built with valid libraries
32+
if [ -d "boringssl/build" ] && [ -f "boringssl/build/libssl.a" ] && [ -f "boringssl/build/libcrypto.a" ]; then
33+
echo "✓ BoringSSL already built (using cached build)"
34+
else
35+
# Remove any incomplete or contaminated BoringSSL directory
36+
if [ -d "boringssl" ]; then
37+
echo "Removing existing BoringSSL directory to ensure clean build..."
38+
rm -rf boringssl
39+
fi
4040

41-
cd boringssl
41+
echo "Cloning BoringSSL..."
42+
git clone --depth 1 https://boringssl.googlesource.com/boringssl
4243

43-
# Check if already built
44-
if [ ! -f "build/libssl.a" ]; then
44+
cd boringssl
4545
echo "Building BoringSSL..."
4646

4747
# Check for required tools
@@ -54,12 +54,6 @@ if [ ! -f "build/libssl.a" ]; then
5454
echo "WARNING: go not found. BoringSSL will build without some tests."
5555
fi
5656

57-
# Clean build directory if it exists (in case of previous failed build)
58-
if [ -d "build" ]; then
59-
echo "Cleaning previous build..."
60-
rm -rf build
61-
fi
62-
6357
mkdir -p build
6458
cd build
6559

@@ -78,6 +72,7 @@ if [ ! -f "build/libssl.a" ]; then
7872
# BoringSSL's CMake is incorrectly trying to use Apple assembly files on Linux
7973
# Using -DOPENSSL_NO_ASM=1 forces pure C implementations instead
8074
echo "Building BoringSSL with assembly disabled to avoid platform detection issues..."
75+
echo "(This may take 5-10 minutes on first build, but will be cached for subsequent builds)"
8176

8277
cmake -DCMAKE_BUILD_TYPE=Release \
8378
-DCMAKE_POSITION_INDEPENDENT_CODE=ON \
@@ -95,8 +90,6 @@ if [ ! -f "build/libssl.a" ]; then
9590
echo "Cleaning up .git directory to save cache space..."
9691
rm -rf .git
9792
fi
98-
else
99-
echo "✓ BoringSSL already built"
10093
fi
10194

10295
cd "$VENDOR_DIR"

0 commit comments

Comments
 (0)