Skip to content

Commit 885afa6

Browse files
committed
chore(config): add a config class for check_eip_versions
1 parent cf183a9 commit 885afa6

File tree

7 files changed

+55
-9
lines changed

7 files changed

+55
-9
lines changed

docs/library/cli/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# EEST CLI Tools
22

3+
- [`check_eip_versions`](../../writing_tests/reference_specification.md) - A CLI tool to check the SHA values specified in EIP tests match the latest version from ethereum/EIPs.
34
- [`eest`](eest.md) - A CLI tool that helps with routine tasks in ethereum/execution-spec-tests.
45
- [`evm_bytes`](evm_bytes.md) - Convert the given EVM bytes from a binary file or a hex string to EEST's python opcodes.
20.8 KB
Loading
-12.6 KB
Loading

docs/writing_tests/reference_specification.md

Lines changed: 40 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,21 @@
11
# Referencing an EIP Spec Version
22

3-
An Ethereum Improvement Proposal ([ethereum/EIPs](https://github.com/ethereum/EIPs/tree/master/EIPS)) and its SHA digest can be directly referenced within a python test module in order to check whether the test implementation could be out-dated. The test framework automatically generates tests for every module that defines a spec version. If the spec is out-of-date because the SHA of the specified file in the remote repo changes, the corresponding `test_eip_spec_version()` test fails.
3+
Tests that implement features from an Ethereum Improvement Proposal ([ethereum/EIPs](https://github.com/ethereum/EIPs/tree/master/EIPS)) must define the EIP's markdown SHA digest within the test's Python module. This ensures our tests stay up-to-date with any changes to the EIP specifications.
4+
5+
The `check_eip_versions` command-line utility automatically verifies that all EIP references in the codebase are current. It works by comparing the SHA specified in the test against the latest version in the ethereum/EIPs repository. This utility uses pytest to generate test cases for every module that includes "eip" in its path.
46

57
<figure markdown> <!-- markdownlint-disable MD033 (MD033=no-inline-html) -->
68
![Test framework summary for a failing EIP spec version test](./img/eip_reference_spec_console_output.png){ width=auto align=center}
7-
`<-snip->`
8-
![EIP spec version test fail](./img/eip_reference_spec_console_output_fail.png){ width=auto align=center}
99
</figure>
1010

11-
!!! info ""
11+
!!! note "The SHA digest value is provided in the failure message of the corresponding test"
12+
13+
<figure markdown> <!-- markdownlint-disable MD033 (MD033=no-inline-html) -->
14+
![EIP spec version test fail](./img/eip_reference_spec_console_output_fail.png){ width=auto align=center}
15+
</figure>
16+
17+
!!! info "Understanding and Retrieving the EIP Markdown's SHA Digest"
18+
1219
The SHA value is the output from git's `hash-object` command, for example:
1320

1421
```console
@@ -18,7 +25,7 @@ An Ethereum Improvement Proposal ([ethereum/EIPs](https://github.com/ethereum/EI
1825
```
1926

2027
and can be retrieved from the remote repo via the Github API on the command-line as following:
21-
28+
2229
```console
2330
sudo apt install jq
2431
curl -s -H "Accept: application/vnd.github.v3+json" \
@@ -31,10 +38,34 @@ An Ethereum Improvement Proposal ([ethereum/EIPs](https://github.com/ethereum/EI
3138

3239
This check accomplished by adding the following two global variables anywhere in the Python source file:
3340

34-
| Variable Name | Explanation |
35-
|-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|
36-
| `REFERENCE_SPEC_GIT_PATH` | The relative path of the EIP markdown file in the [ethereum/EIPs](https://github.com/ethereum/EIPs/) repository, e.g. "`EIPS/eip-1234.md`" |
37-
| `REFERENCE_SPEC_VERSION` | The SHA hash of the latest version of the file retrieved from the Github API:<br>`https://api.github.com/repos/ethereum/EIPs/contents/EIPS/eip-<EIP Number>.md` |
41+
| Variable Name | Explanation |
42+
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
43+
| `REFERENCE_SPEC_GIT_PATH` | The relative path of the EIP markdown file in the [ethereum/EIPs](https://github.com/ethereum/EIPs/) repository, e.g. "`EIPS/eip-1234.md`" |
44+
| `REFERENCE_SPEC_VERSION` | The SHA hash of the latest version of the file retrieved from the Github API:<br>`https://api.github.com/repos/ethereum/EIPs/contents/EIPS/eip-<EIP Number>.md` |
45+
46+
## Running the `check_eip_versions` Command Locally
47+
48+
A Github Personal Access Token (PAT) is required to avoid rate-limiting issues when using the Github API. A PAT can be created at: https://github.com/settings/personal-access-tokens/new.
49+
50+
After setting the `GITHUB_TOKEN` environment variable to the value of your PAT, EIP versions can be checked locally using:
51+
52+
```shell
53+
uv run check_eip_versions
54+
```
55+
56+
By default, only tests up to and including the current fork under development will be checked. This is controlled by the `UNTIL_FORK` setting in the `src/config/check_eip_versions.py` configuration file. You can also pass a specific test path to limit the scope:
57+
58+
```shell
59+
uv run check_eip_versions tests/shanghai/eip3651_warm_coinbase/
60+
```
61+
62+
This would only check EIP versions for the EIP-3651 tests in the `shanghai/eip3651_warm_coinbase` sub-directory.
63+
64+
## Automated Checks via GitHub Actions
65+
66+
The repository includes a [GitHub Actions workflow](https://github.com/ethereum/execution-spec-tests/actions/workflows/check_eip_versions.yaml) that automatically runs `check_eip_versions` on a daily schedule. If any outdated EIP references are detected, the workflow creates an issue in the repository with details about which references need to be updated.
67+
68+
This workflow uses GitHub's built-in token for authentication, so there's no need to configure personal access tokens for the automated checks. The issue will include links to the relevant workflow run and details about which tests need updating.
3869

3970
## Example
4071

src/cli/pytest_commands/check_eip_versions.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@
66
import click
77
import pytest
88

9+
from config.check_eip_versions import CheckEipVersionsConfig
10+
911
from .common import common_click_options, handle_help_flags
1012

1113

@@ -14,6 +16,7 @@
1416
def check_eip_versions(pytest_args: List[str], **kwargs) -> None:
1517
"""Run pytest with the `spec_version_checker` plugin."""
1618
args = ["-c", "pytest-check-eip-versions.ini"]
19+
args += ["--until", CheckEipVersionsConfig().UNTIL_FORK]
1720
args += handle_help_flags(list(pytest_args), pytest_type="check-eip-versions")
1821
result = pytest.main(args)
1922
sys.exit(result)

src/config/check_eip_versions.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
"""A module for managing configuration for the `check_eip_version` utility."""
2+
3+
from pydantic import BaseModel
4+
5+
6+
class CheckEipVersionsConfig(BaseModel):
7+
"""A class for accessing configurations for `check_eip_version`."""
8+
9+
UNTIL_FORK: str = "Prague"
10+
"""The target fork to check eip versions until."""

whitelist.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,7 @@ P7692
137137
eip7251
138138
eips
139139
EIPs
140+
EIP's
140141
el
141142
endianness
142143
EngineAPI

0 commit comments

Comments
 (0)