Source tar too large / drastically larger vs before jupyter releaser

[andrii-i]

Description

Looking for help in understanding why this happened and how to prevent it from happening during the next release.

During the Step 2: Publish Release of Jupyter Scheduler new release 2.10.0 Source Distribution upload failed due to PyPI source distributions size limits (~150 Mb) due to tar build being drastically larger in size vs before jupyter-releaser introduction. Built distribution upload went through, npm upload did not as it's later in the script.

By now 2.10.0 is fixed by building and uploading missing files manually but running jupyter-releaser build-python produces the source of the same size >150 Mb (which causes PyPI size limit errors) vs up to 3 Mb before the jupyter-releaser was introduced. We need to avoid this happening in the the next release.

Reproduce

• See CI failure: https://github.com/jupyter-server/jupyter-scheduler/actions/runs/11808986203/job/32898512565#step:5:372
• Try building jupyter scheduler PyPI source distribution locally with jupyter-releaser build-python, see its size (>100 Mb)

Expected behavior

Workflow does not fail, source distribution of the reasonable size is built and uploaded to PyPI, upload to npm happens.

Context

• Jupyter Scheduler 2.10.0 Source Distribution tar built too large causing PyPI upload failure jupyter-scheduler#558

[andrii-i]

For reference, here is the relevant log excerpt source distribution upload logs:

100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 152.0/152.0 MB • 00:03 • 62.2 MB/s
WARNING  Error during upload. Retry with the --verbose option for more details.
ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
         File too large. Limit for project 'jupyter-scheduler' is 100 MB. See
         https://pypi.org/help/#file-size-limit for more information.

https://github.com/jupyter-server/jupyter-scheduler/actions/runs/11808986203/job/32898512565#step:5:373

[krassowski]

Comparing artifacts from

• "build" which uses hatch alone https://github.com/jupyter-server/jupyter-scheduler/actions/runs/11809051198
• "check-release" which uses jupyter-releaser https://github.com/jupyter-server/jupyter-scheduler/actions/runs/11809051201

plain hatch	jupyter-releaser

[krassowski]

Good news! New version of pypiserver bundles bottle version which respects file size limits, which means that check-release job will now fail when we attempt to upload too big a file. Bad news: it is not configurable and set to very small number by default so most of our CI now fails, see pypiserver/pypiserver#630