Skip to content

Release build failure: tts-wheels-linux-x86_64.tar.gz exceeds GitHub's 2 GB asset size limit #769

@mhamann

Description

@mhamann

Summary

The "Upload Addon Wheels to Release (tts, linux-x86_64)" job fails because the tarball exceeds GitHub's 2 GB release asset limit.

Root Cause

The tts-wheels-linux-x86_64 artifact is 4,121 MB — nearly 13x larger than the same addon on other platforms:

Artifact Size
tts-wheels-macos-arm64 195 MB
tts-wheels-linux-arm64 325 MB
tts-wheels-linux-x86_64 4,121 MB

PyTorch CUDA wheels are the culprit. kokoro depends on torch, and on linux-x86_64 pip download --only-binary=:all: resolves to the full CUDA-enabled torch wheel (~2–4 GB). On macOS and linux-arm64, only CPU-only wheels are available on PyPI.

Why PR builds didn't catch this

The upload-release job only runs on release events. PR builds successfully create the 4.1 GB workflow artifact (10 GB limit) but never attempt the release upload (2 GB limit).

This same failure also occurred on the v0.0.1+feat.addons pre-release.

Error

Validation Failed: {"resource":"ReleaseAsset","code":"custom","field":"size","message":"size must be less than or equal to 2147483648"}

Failed run: https://github.com/llama-farm/llamafarm/actions/runs/22078014087/job/63820178925

Recommended Fix

Move dependency filtering from install-time to build-time

The CLI already handles this at install time — removeCommonPackages() in addons_downloader.go introspects the component's venv and removes packages that are already present. However, by then the oversized tarball has already been built and (unsuccessfully) uploaded. The filtering needs to happen during the build in build_addon_wheels.py.

Proposed approach for build_addon_wheels.py

After pip download completes, automatically exclude wheels for packages that are already provided by the base install:

  1. Resolve the base runtime's dependency tree — parse the uv.lock lockfile from runtimes/universal/ (or run uv pip compile) to get the canonical set of packages provided by the base install. This covers all packages managed by uv sync.

  2. Also account for CLI-installed packages — certain packages (e.g., PyTorch itself) are installed by the lf CLI rather than through uv sync, based on detected hardware. These need to be included in the exclusion set as well. The build script should have a way to incorporate these — either by:

    • Maintaining a small addons/base-packages.txt manifest that lists packages the CLI installs outside of uv (e.g., torch, torchaudio, torchvision)
    • Or parsing the pyproject.toml optional dependencies and [tool.uv] overrides that reference these packages
  3. Post-download filter — after pip download, delete any .whl files whose package name appears in the combined exclusion set (uv lockfile + CLI-managed packages) before creating the tarball.

This mirrors the client-side removeCommonPackages() logic but runs at build time, keeping release assets small and staying in sync automatically as dependencies evolve.

Benefits over current state

  • Release assets stay well under the 2 GB limit
  • Users download smaller tarballs (seconds vs minutes)
  • The existing client-side removeCommonPackages() cleanup becomes a safety net rather than the primary mechanism
  • No manual exclusion lists to maintain — derived from existing lockfile and config

Context

All other addon bundles (stt, onnxruntime) are well under 200 MB. This is isolated to TTS on linux-x86_64 due to the torch CUDA dependency chain.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions