feat(cli,fill): make the eip version checker a separate pytest cli tool#1537
feat(cli,fill): make the eip version checker a separate pytest cli tool#1537danceratopz merged 25 commits intomainfrom
Conversation
…cker This fixes up the previous commit that no longer includes the `spec_version_checker` plugin when executing `fill` or `execute` by removing it from their pytest-*ini files.
marioevz
left a comment
There was a problem hiding this comment.
Left just one comment regarding the frequency of the runs for the new CI workflow.
|
Tagging this comment, as another alternative to prevent rate limiting! I really think we should use it. Essentially a single endpoint call that gets all the EIP sha's at once! Check it out :) https://api.github.com/repos/ethereum/EIPs/git/trees/master?recursive=1 |
That's a nice solution, but I'm out of steam for this topic/PR. For now this (or using a env var) is working nicely: Tbh, the plugin should be re-written to not collect the evm python test cases at all (it currently collects ~50,000 test cases). I think it should be possible to simply discover test modules (containing "eip") and then create the eip version check test. Feel free to PR your suggestion, if you fancy. Otherwise, we collect these ideas into a follow-up issue. |
|
Created an issue :) |
🗒️ Description
pytest_plugins.spec_version_checker.spec_version_checkerplugin as part offillorexecute.check_eip_versionsthat only performs EIP spec version checking in a pytest session.check_eip_versionsdaily and create an issue upon failure (example issues chore(tests): eip spec references outdated #1553 (prague mismatches only) and chore(tests): eip spec references outdated #1552 (all forks).Note
This PR unfortunately adds a new pytest ini
pytest-check-eip-versions.iniwhich is essentially a duplicate offill'spytest.ini. This is an ugly code duplication, that will likely causecheck_eip_versionsto break iffill's options get changed w/o changing thecheck_eip_versionspytest ini. The issue is that thepytest_plugins.spec_version_checker.spec_version_checkerneeds to be loaded first, which doesn't seem to be possible if used in conjunction with-c pytest.ini.🔗 Related Issues
Fixes #533, #1552, #1553
Todo before merge
✅ Checklist