Skip to content

Release#4

Merged
arman-bd merged 102 commits intomainfrom
develop
Oct 12, 2025
Merged

Release#4
arman-bd merged 102 commits intomainfrom
develop

Conversation

@arman-bd
Copy link
Owner

No description provided.

arman-bd and others added 30 commits October 8, 2025 23:01
- Add _POSIX_C_SOURCE and _XOPEN_SOURCE macros for strdup declaration
- Include strings.h for strcasecmp function
- Fix http2_request signature to accept const request parameter
- Fix C23 label-declaration warning by wrapping in block scope

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Build fixes:
- Compile nghttp2 with -fPIC flag for shared library compatibility
- Add POSIX feature macros to browser_profiles.c
- Include strings.h for strcasecmp function
- Fix ja3_hash to use MD5 hash instead of full JA3 string (33 chars max)

Docker testing:
- Add Dockerfile.test that mimics GitHub Actions ubuntu-latest
- Add docker-compose.test.yml for easier Docker management
- Add Makefile targets: docker-build, docker-test, docker-shell
- Add .dockerignore to optimize build context

This allows local testing in the exact CI environment before pushing,
significantly reducing CI iteration time.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
arman-bd and others added 26 commits October 11, 2025 15:00
Key changes:
1. Skip 32-bit Windows builds (*-win32) - they're rarely needed
2. Remove vcpkg dependency - build all libs from source
3. Build nghttp2 from source on Windows using CMake
4. Build zlib from source on Windows using CMake
5. Update setup.py to find vendor-built nghttp2 libraries

This ensures consistent builds across all platforms and eliminates
architecture mismatch issues (x64 libs being used for 32-bit builds).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add caching for vendor dependencies in release.yml to significantly
speed up wheel builds:
- Cache vendor directory across builds
- Only rebuild when setup_vendors.sh changes
- Saves 10-12 minutes per build

This brings release workflow up to parity with test workflow caching.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Major performance improvement for wheel builds:

Before:
- cibuildwheel rebuilt vendors for EACH Python version (3.9-3.12)
- 4 separate builds × 15min = 60 minutes total
- No caching between Python versions

After:
- Build vendors ONCE before cibuildwheel runs
- Cache vendors across all Python versions
- cibuildwheel skips before-build step (CIBW_BEFORE_BUILD="")
- Estimated time: 15min (first) or 3min (cached) + 2min × 4 = ~11min

Key changes:
1. Separate vendor build step outside cibuildwheel
2. Explicit cache restore/save with conditional logic
3. Skip redundant before-build in cibuildwheel
4. Add verification step to ensure vendor libs exist

Expected speedup: 60min → 11min (first build) or 11min → 3min (cached)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Restructure release workflow for better maintainability:

New structure:
- _build_windows.yml - Windows-specific build configuration
- _build_linux.yml - Linux-specific build configuration
- _build_macos.yml - macOS-specific build configuration
- release.yml - Orchestrates all builds and publishing

Benefits:
✅ OS-specific settings isolated in separate files
✅ Easier to maintain and customize per platform
✅ Better code organization following DRY principle
✅ Parallel builds across all platforms
✅ Platform-specific caching strategies
✅ Clearer debugging when one platform fails

Each OS workflow includes:
- Platform-specific tool installation
- Vendor dependency caching (with OS-specific cache keys)
- Build verification steps
- Optimized cibuildwheel configuration

The main release.yml now simply:
1. Calls each OS-specific build workflow in parallel
2. Collects artifacts from all platforms
3. Publishes to GitHub and PyPI

This matches the pattern used in _test.yml and _benchmark.yml
for consistency across the project.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses platform cross-contamination issues where cached
BoringSSL directories from one OS (e.g., macOS) would cause build
failures on another OS (e.g., Linux trying to compile Apple assembly files).

Changes:
- Always remove and re-clone BoringSSL to prevent contamination
- Clean build directories before building
- Apply consistent build approach across Linux, macOS, and Windows
- Temporarily reduce Python test matrix to 3.8 only for faster iteration

This fixes the Linux build error where it was trying to assemble
`aes-gcm-avx2-x86_64-apple.S` files with the GNU assembler.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit changes the CI cache strategy to always rebuild vendor
dependencies instead of relying on cached artifacts. This ensures:

1. No cross-platform contamination from cached directories
2. Fresh builds with correct platform-specific assembly files
3. Consistent build environment across all CI runs

Changes:
- Bump cache version from v9 to v10 to invalidate old caches
- Remove conditional build steps (always build vendors)
- Update restore-keys to match exact hash (no fallback to old caches)
- Apply to Windows, Linux, and macOS workflows

This fixes issues where:
- Linux was getting macOS BoringSSL assembly files
- Windows couldn't find ssl.lib due to corrupted cache
- Builds were failing with platform detection errors

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
The test workflow was still using the old v7 cache key and had
fallback restore-keys that could restore contaminated caches from
different platforms. This was causing the Linux test to get macOS
BoringSSL assembly files.

Changes:
- Bump cache version from v7 to v10 in test workflow
- Remove conditional build (always rebuild vendors)
- Update restore-keys to match exact hash (no fallback)
- Align with build workflow cache strategy

This ensures the test workflow gets fresh, platform-specific vendor
builds just like the wheel build workflows.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
BoringSSL's CMake configuration was incorrectly trying to compile
Apple-specific assembly files (e.g., aes-gcm-avx2-x86_64-apple.S)
on Linux runners, causing build failures.

Root cause: BoringSSL ships with pre-generated assembly files for
all platforms (apple, linux, win) and CMake should filter them based
on the target platform. However, something in the GitHub Actions
environment was confusing the platform detection.

Workaround: Explicitly remove Apple-specific assembly files before
running CMake on Linux. This ensures only Linux-compatible assembly
files are present during the build.

Changes:
- Add find command to delete *-apple.S files on Linux
- Unset APPLE environment variable that might confuse CMake
- This is a temporary workaround until root cause is identified

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…on issues

Changed approach from deleting files to disabling assembly entirely
using -DOPENSSL_NO_ASM=1. This is the same solution that worked for
macOS ARM64 builds.

The previous approach of deleting Apple assembly files failed because
CMake's sources.cmake still referenced them, causing configuration errors.

Using -DOPENSSL_NO_ASM=1:
- Bypasses platform detection issues entirely
- Uses pure C implementations instead of assembly
- Slightly slower but guaranteed to work
- Same approach successfully used on macOS ARM64

This trades a small performance penalty for build reliability.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
PyPI was rejecting uploads with error: "Invalid distribution metadata:
unrecognized or malformed field 'license-file'"

Root cause: Setuptools was automatically adding a deprecated
'license-file' metadata field when it detected a LICENSE file in the
repository root. PyPI no longer accepts this field format.

Solution: Explicitly set license-files = [] in [tool.setuptools] to
prevent automatic LICENSE file inclusion in wheel metadata.

The license is still properly declared via license = {text = "MIT"}
in the [project] section.

Also updated release workflow to build for all Python versions 3.8-3.14.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Optimize Linux wheel builds to reduce time from ~40 minutes to ~10 minutes
by reusing BoringSSL builds across Python versions.

Changes:
1. Modified scripts/linux/setup_vendors.sh to check for existing built
   libraries BEFORE removing the directory, allowing cibuildwheel to
   reuse builds across Python versions

2. The script now:
   - Checks if boringssl/build contains valid libssl.a and libcrypto.a
   - Only removes and rebuilds if libraries are missing or invalid
   - Skips expensive 5-10 minute BoringSSL build for Python versions 2-7

3. Updated cache key to v11 to ensure clean state with new logic

How it works:
- First Python version (cp38) builds BoringSSL (~5-10 min)
- Subsequent Python versions (cp39-cp314) reuse the build (~10 sec each)
- Total build time: ~10-15 min instead of ~40 min

The vendor directory is mounted into each cibuildwheel manylinux container,
so builds persist across Python versions within the same CI run.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@arman-bd arman-bd self-assigned this Oct 12, 2025
@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@arman-bd arman-bd merged commit 00e3277 into main Oct 12, 2025
27 of 37 checks passed
arman-bd added a commit that referenced this pull request Oct 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants