Skip to content

ci: Add package verification check#1893

Draft
vdusek wants to merge 1 commit into
masterfrom
ci/verify-built-package
Draft

ci: Add package verification check#1893
vdusek wants to merge 1 commit into
masterfrom
ci/verify-built-package

Conversation

@vdusek
Copy link
Copy Markdown
Collaborator

@vdusek vdusek commented May 12, 2026

Summary

Adds an automated end-to-end verification of the built sdist + wheel to guard against the silent failure mode behind #1890 — where artifacts build successfully but crash on install or import.

New: scripts/verify_built_package.py — a standalone script (runnable locally, no crawlee install required) that:

  • Validates sdist contents: required metadata files (LICENSE, README.md, CHANGELOG.md, CONTRIBUTING.md, pyproject.toml), no tests/ / docs/ / website/ / .github/ / examples/ leakage, no uv.lock, all 192 .py sources present, all 27 data files (every py.typed, every project_template/**, every _redis/lua_scripts/*.lua) present.
  • Validates wheel contents: .dist-info/METADATA present plus the same source + data assertions.
  • Installs the wheel into a fresh Python 3.14 venv with [all] extras and runs an import + crawler-construction smoke test.
  • Installs the sdist into a fresh venv — this is the load-bearing check, since pip rebuilds the wheel from sdist contents and any missing source surfaces here.
  • Runs crawlee create --no-apify --no-install ... and asserts the scaffolded project files exist (catches a silently-dropped project_template/).

Takes ~10s locally, ~1 min in CI. Exits non-zero on any failure.

CI wiring:

  • .github/workflows/_check_package.yaml — new reusable workflow that builds + verifies.
  • on_pull_request.yaml — runs package_check on every PR (parallel with tests and code_checks).
  • manual_release_stable.yaml + manual_release_beta.yaml — inserts a Verify built package step inside the pypi_publish job, between the prepare-distribution and PyPI upload steps. The exact byte-identical artifact about to be uploaded is verified.

Verified locally

  • Pass path: clean build → all 5 check groups green, exit 0
  • Negative path: tampered sdist with project_template/cookiecutter.json and a .lua script removed → flagged both, exited 1
  • actionlint clean on all four workflows

@vdusek vdusek added t-tooling Issues with this label are in the ownership of the tooling team. adhoc Ad-hoc unplanned task added during the sprint. labels May 12, 2026
@vdusek vdusek self-assigned this May 12, 2026
@github-actions github-actions Bot added this to the 140th sprint - Tooling team milestone May 12, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.84%. Comparing base (0e7402f) to head (5b35186).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1893      +/-   ##
==========================================
+ Coverage   92.82%   92.84%   +0.01%     
==========================================
  Files         167      167              
  Lines       11699    11699              
==========================================
+ Hits        10860    10862       +2     
+ Misses        839      837       -2     
Flag Coverage Δ
unit 92.84% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Guards against shipping a wheel or sdist that builds but crashes when
installed - the silent failure mode behind PR #1890 - by adding a
verification script and wiring it into PR CI and the release workflows
as a pre-publish gate. Checks artifact contents, fresh-venv install of
both wheel and sdist, core imports, and `crawlee create` scaffolding.
@vdusek vdusek force-pushed the ci/verify-built-package branch from d3b1195 to 5b35186 Compare May 12, 2026 09:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants