Migrate config to pyproject.toml and use uv for project management

[danceratopz]

This PR modernizes the project structure by migrating configuration to pyproject.toml and adds project management via uv and a uv.lock lockfile.

• Migrated Python spec generation from custom setuptools commands to a standalone script, ./pysetup/generate_specs.py.

  • This also replaced distutils with pathlib for modern path handling.

• Migrated to uv-based dependency management with a committed lockfile for reproducible builds.

• Moved "low-hanging" packaging config from setup.py and mypy.ini into pyproject.toml.

• Updated Makefile to use uv run while maintaining all existing developer-facing targets.

• Added uv availability check to Makefile:

  make lint
  Error: uv is required but not installed.
  Install with: curl -LsSf https://astral.sh/uv/install.sh | sh
  make: *** [Makefile:196: sync] Error 1

• Removed Python version script and check; uv handles this.

• Added lockfile verification to lint target to catch dependency drifts.

• Updated CI flows to use the astral-sh/setup-uv workflow (docs) with caching and additionally:

  • Replaced hardcoded python-version: '3.13' with python-version-file: "pyproject.toml" (this will use the latest supported version).
  • Removed ineffective cache: 'pip' from setup-python steps.

• Reverted: Updated mypy to check the entire eth2spec source tree directly (using --explicit-package-bases) instead of checking only fork-specific packages via -p.

  Expand to see commands

  We run:

  uv run mypy --explicit-package-bases tests/core/pyspec/eth2spec/

  instead of:

  uv run mypy  -p eth2spec.phase0  -p eth2spec.altair  -p eth2spec.bellatrix  -p eth2spec.capella  -p eth2spec.deneb  -p eth2spec.electra  -p eth2spec.fulu  -p eth2spec.gloas  -p eth2spec.eip6800  -p eth2spec.eip7441  -p eth2spec.eip7805  -p eth2spec.eip7928

Benefits

Besides clean-up:

• consensus-specs can now use more features of uv.
• Builds are now entirely reproducible due to the inclusion of a lockfile uv.lock.
• uv creates the venv instead of python -m venv (which saves ~7s on my machine).
• Reverted The entire eth2spec source tree is now checked via mypy.

Breaking Change

Users must install uv themselves, preferably via curl (which allows Python installation and uv self update):

curl -LsSf https://astral.sh/uv/install.sh | sh

Aftwards, make can be used as before.

Todo

• Fix mypy the new errors in this PR caused by checking eth2spec source directly instead of as an installed package -p eth2spec. Reverted to previous mypy behavior.

Suggested Next Steps

I didn't proceed with these changes; I'd suggest tackling them as separate PRs:

1. The eth2spec package location tests/core/pyspec/eth2spec is very unexpected; we could move this to ./src/eth2spec/.
2. Remove setup.py entirely. This will require a bit of clean-up due to the non-standard location of data directories (which should be nested under eth2spec, I believe). We could move them under ./src/eth2spec/resources, for example, and then move the remaining packaging config to pyproject.toml.
3. Add the markdown files as a dependency for _pyspec; only regenerate if they've changed (this is make's main feature, but we're not using at the mo ;) ).
4. Loosen the package pinning in pyproject.toml; it's not strictly needed if we use uv.lock for dev and is rather restrictive if consensus-specs is ever published/used as a package.
5. Move ./pyspec to ./src/pyspec (a sibling package; it doesn't need to be published), then add ./src/pysetup/generate_specs.py as an entrypoint under [project.scripts].
   • Check whether ruamel.yaml and marko need to be in project deps; if not they could be moved as dependencies to the script.
   • To use generate_specs.py as a script (i.e., without running via python3 -m) might additionally require renaming ./pyspec/typing to ./pyspec/spec_types.py (for example) to avoid shadowing Python's builtin typing module.

[jtraglia]

Nice 🔥 this looks like a great improvement.

Let's remove some of the excess print statements & look into the linting issue.

[jtraglia]

I believe we need to change this from pip to uv too, right?

consensus-specs/.github/dependabot.yml

Line 3 in 6c9cf03

- package-ecosystem: pip

[danceratopz]

I believe we need to change this from pip to uv too, right?

consensus-specs/.github/dependabot.yml

Line 3 in 6c9cf03

- package-ecosystem: pip

Good catch, @jtraglia! I'm not sure how good the support is but we can try - added! 07849e3
https://docs.astral.sh/uv/guides/integration/dependency-bots/#dependabot

[jtraglia]

Nice, sorry for all of the random feedback! I think this is almost ready to merge 😄

[danceratopz]

Nice, sorry for all of the random feedback! I think this is almost ready to merge 😄

No worries, thanks for the fast feedback, it flowed well tbh!

Are you happy to squash 😅 I can organize the commits if you like if that's how you do things around here ))

[jtraglia]

Are you happy to squash 😅

Yes, we squash all PRs here now! So no need to clean anything up.

[danceratopz]

Yes, we squash all PRs here now! So no need to clean anything up.

Great! The last commit added a small update to the readme that I forgot to add earlier.

[jtraglia]

Awesome, this looks great. Thanks again @danceratopz! I'm going to merge this after CI checks pass.

@leolara please follow up post-merge & open a PR if there's anything you'd like to change.

[etan-status]

Ever since this was merged, make reftests fails with

  File "/opt/homebrew/Cellar/[email protected]/3.13.7/Frameworks/Python.framework/Versions/3.13/lib/python3.13/pickle.py", line 1087, in save_global
    raise PicklingError(
        "Can't pickle %r: it's not found as %s.%s" %
        (obj, module_name, name)) from None
_pickle.PicklingError: Can't pickle <class '_lru.LRU'>: it's not found as _lru.LRU

Terminal Saved Output.txt

@jtraglia @danceratopz

[etan-status]

It doesn't fail when only selecting a single fork or runner, only the full one.

[danceratopz]

Thanks for flagging @etan-status! The problem is described here:

• Editable installs + multiprocessing break vector generation #4633
  Fix incoming!