Skip to content

Commit f62890f

Browse files
refactor(fill,deps): forbid yul code in python tests; remove solc-select dependency
Co-authored-by: danceratopz <[email protected]>
1 parent 2a7a6d7 commit f62890f

40 files changed

+458
-1192
lines changed

CLAUDE.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,6 @@ All commands use `uv run` prefix.
3232

3333
```bash
3434
uv sync --all-extras
35-
uv run solc-select use 0.8.24 --always-install
3635
uvx pre-commit install
3736
```
3837

CONTRIBUTING.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,6 @@ We welcome contributions via pull requests! This section will guide you through
8585
uv python install 3.12
8686
uv python pin 3.12
8787
uv sync --all-extras
88-
uv run solc-select use 0.8.24 --always-install
8988
```
9089

9190
See [installation troubleshooting](https://eest.ethereum.org/main/getting_started/installation_troubleshooting) if you encounter issues.

README.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,6 @@ cd execution-spec-tests
109109
uv python install 3.11
110110
uv python pin 3.11
111111
uv sync --all-extras
112-
uv run solc-select use 0.8.24 --always-install
113112
```
114113

115114
See [Installation Troubleshooting](https://eest.ethereum.org/main/getting_started/installation_troubleshooting/) in the online docs if you encounter issues.

docs/CHANGELOG.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,11 @@ Test fixtures for use by clients are available for each release on the [Github r
1010

1111
- Python 3.10 support was removed in this release ([#1808](https://github.com/ethereum/execution-spec-tests/pull/1808)).
1212

13+
#### 💥 Important Change for test contributors
14+
15+
- EEST no longer allows usage of Yul code in Python tests. From now on, please make use of our opcode wrapper. Yul code is now only allowed in the "static tests" located in `./tests/static/` (these are test cases defined by JSON and YAML files instead of Python test functions that were originally maintained in [ethereum/tests](https://github.com/ethereum/tests)).
16+
- In order to fill the static tests (which is not the case by default), please ensure that `solc` is located in your `PATH`.
17+
1318
#### 💥 Important Change for `fill` Users
1419

1520
The output behavior of `fill` has changed ([#1608](https://github.com/ethereum/execution-spec-tests/pull/1608)):
@@ -27,6 +32,12 @@ Users can select any of the artifacts depending on their testing needs for their
2732

2833
### 🛠️ Framework
2934

35+
#### 🔀 Refactoring
36+
37+
- 🔀 Move `TransactionType` enum from test file to proper module location in `ethereum_test_types.transaction_types` for better code organization and reusability.
38+
- ✨ Opcode classes now validate keyword arguments and raise `ValueError` with clear error messages.
39+
- 🔀 This PR removes the `solc` requirement to fill Python test cases. Regular test contributors no longer need to concern themselves with `solc` and, as such, the `solc-select` dependency has been removed. The remaining tests that used Yul have been ported to the EEST opcode wrapper mini-lang and the use of Yul in Python tests is no longer supported. Maintainers only: To fill the "static" JSON and YAML tests (`./tests/static/`) locally, `solc` (ideally v0.8.24) must be available in your PATH.
40+
3041
#### `fill`
3142

3243
- ✨ Add the `ported_from` test marker to track Python test cases that were converted from static fillers in [ethereum/tests](https://github.com/ethereum/tests) repository ([#1590](https://github.com/ethereum/execution-spec-tests/pull/1590)).

docs/dev/porting_legacy_tests.md

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -85,16 +85,14 @@ Follow the hyperlinks for lost base coverage (`LBC`) to address coverage gaps. H
8585

8686
It's important to note that coverage helps identify missing test paths. If you believe the coverage loss is due to differences in "setup" code between frameworks and doesn't impact the feature you're testing, explain this in your PR. A team member can help with the review.
8787

88-
Also note that yul tests and possibly other tests used `CALLDATALOAD` that might no longer needed when designing a test with python. But we must always investigate if an opcode is not covered anymore to see if its okay.
89-
90-
For example, review the [discussion in this PR.](https://github.com/ethereum/execution-spec-tests/pull/975#issuecomment-2528792289)
88+
For example, review the [discussion in this PR] to see an example of why and how coverage loss can occur.(https://github.com/ethereum/execution-spec-tests/pull/975#issuecomment-2528792289)
9189

9290
## Resolving Coverage Gaps from Yul Compilation
9391

9492
When porting tests from ethereum/tests, you may encounter coverage gaps that are false positives. This commonly occurs because:
9593

9694
- **Original tests** often used Yul to define smart contracts, and solc compilation introduces additional opcodes that aren't specifically under test
97-
- **EEST ports** typically use the explicit EEST Opcode mini-language, which is more precise in opcode definition
95+
- **EEST ports** use the explicit EEST Opcode mini-language, which is more precise in opcode definition
9896

9997
If coverage analysis shows missing opcodes that were only present due to Yul compilation artifacts (not the actual feature being tested), this can be resolved during PR review by adding the `coverage_missed_reason` parameter:
10098

docs/getting_started/installation.md

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -26,27 +26,17 @@ If installed via `curl`, `uv` will download Python for your target platform if o
2626

2727
Clone [execution-spec-tests](https://github.com/ethereum/execution-spec-tests) and install its dependencies. We recommend using Python 3.12, the following uses `uv` to download and configures 3.12 to be the Python version used in execution-spec-tests:
2828

29-
=== "All x86-64 platforms and ARM64 macOS"
29+
=== "All platforms"
3030

3131
```console
3232
git clone https://github.com/ethereum/execution-spec-tests
3333
cd execution-spec-tests
3434
uv python install 3.12
3535
uv python pin 3.12
3636
uv sync --all-extras
37-
uv run solc-select use 0.8.24 --always-install
3837
```
3938

40-
=== "ARM64 Linux"
41-
42-
```console
43-
git clone https://github.com/ethereum/execution-spec-tests
44-
uv python install 3.12
45-
uv python pin 3.12
46-
cd execution-spec-tests
47-
uv sync --all-extras
48-
```
49-
Then follow [this guide](./installation_troubleshooting.md#problem-exception-failed-to-compile-yul-source) to build the `solc` binary from source and copy it to the expected location.
39+
Static tests/maintainers only: To learn how to build the `solc` binary from source (optional) follow [this guide](./installation_troubleshooting.md#problem-exception-failed-to-compile-yul-source).
5040

5141
## Installation Troubleshooting
5242

docs/getting_started/installation_troubleshooting.md

Lines changed: 6 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -38,11 +38,11 @@ This page provides guidance on how to troubleshoot common issues that may arise
3838

3939
### Problem: `CERTIFICATE_VERIFY_FAILED`
4040

41-
!!! danger "Problem: `Failed to install solc ... CERTIFICATE_VERIFY_FAILED`"
42-
When running either `uv run solc-select use 0.8.24 --always-install` or `fill` you encounter the following error:
41+
!!! danger "Problem: `Failed to ... CERTIFICATE_VERIFY_FAILED`"
42+
When running `fill` you might encounter the following error:
4343

4444
```bash
45-
Exit: Failed to install solc version 0.8.24: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)>
45+
Exit: Failed to ...: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)>
4646
```
4747

4848
=== "Ubuntu"
@@ -66,15 +66,8 @@ This page provides guidance on how to troubleshoot common issues that may arise
6666

6767
### Problem: `Exception: failed to compile yul source`
6868

69-
!!! danger "Problem: `Running fill fails with tests that contain yul source code` on ARM platforms"
70-
If you're using `x86_64`, try to manually run `solc-select` to install the required version of the `solc` binary:
71-
72-
```shell
73-
uv run solc-select use 0.8.24 --always-install
74-
```
75-
76-
Otherwise, this can happen when you're using an ARM64 OS but followed the x86-64 installation guide.
77-
To resolve the issue you must build solidity from source (avoid 0.8.24 as it might require building z3 from source as well).
69+
!!! danger "Problem: `Running fill on static_tests fails with tests that contain yul source code` on ARM platforms"
70+
To resolve the issue you must acquire the `solc` binary e.g. by building solidity from source. The guide below installs v0.8.28 but you might prefer to choose a different version.
7871

7972
!!! success "Solution: Build solc from source"
8073
The following steps have been tested on Ubuntu 24.04.2 LTS ARM64:
@@ -88,7 +81,7 @@ This page provides guidance on how to troubleshoot common issues that may arise
8881
cp ./solc/solc $HOME/Documents/execution-spec-tests/.venv/bin/
8982
chmod +x $HOME/Documents/execution-spec-tests/.venv/bin/solc
9083
```
91-
Running `uv run solc --version` should now return the expected version. Verify that `fill` works by running `uv run fill -m yul_test` in the execution-spec-tests root folder.
84+
Running `uv run solc --version` should now return the expected version.
9285

9386
## Problem: `ValueError: unsupported hash type ripemd160`
9487

docs/running_tests/test_formats/blockchain_test_engine_x.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ The key optimization is that **clients need only be started once per group** ins
1414

1515
Instead of including large pre-allocation state in each test fixture, this format references a pre-allocation groups folder (`pre_alloc`) which contains all different pre-allocation combinations organized by group.
1616

17-
A single JSON fixture file is composed of a JSON object where each key-value pair is a different [`BlockchainTestEngineXFixture`](#BlockchainTestEngineXFixture) test object, with the key string representing the test name.
17+
A single JSON fixture file is composed of a JSON object where each key-value pair is a different [`BlockchainTestEngineXFixture`](#blockchaintestenginexfixture) test object, with the key string representing the test name.
1818

1919
The JSON file path plus the test name are used as the unique test identifier.
2020

@@ -48,7 +48,7 @@ Each file in the `pre_alloc` folder corresponds to a pre-allocation group identi
4848

4949
## Consumption
5050

51-
For each [`BlockchainTestEngineXFixture`](#BlockchainTestEngineXFixture) test object in the JSON fixture file, perform the following steps:
51+
For each [`BlockchainTestEngineXFixture`](#blockchaintestenginexfixture) test object in the JSON fixture file, perform the following steps:
5252

5353
1. **Load Pre-Allocation Group**:
5454
- Read the appropriate file from the `pre_alloc` folder in the same directory

docs/running_tests/test_formats/common_types.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,10 @@ State allocation represented as a JSON object, where the keys are the addresses
8686

8787
Maps forks to blob schedule configurations as defined by [EIP-7840](https://eips.ethereum.org/EIPS/eip-7840).
8888

89+
### `Environment`
90+
91+
Contains blockchain-related configuration that provides the context in which a test is run.
92+
8993
### `ForkBlobSchedule`
9094

9195
A fork blob schedule as defined by [EIP-7840](https://eips.ethereum.org/EIPS/eip-7840) as a JSON dictionary with the following entries:

docs/writing_tests/fork_methods.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -151,8 +151,6 @@ Methods for fork identification and comparison:
151151
```python
152152
fork.name() # Returns the name of the fork
153153
fork.transition_tool_name(block_number=0, timestamp=0) # Returns name for transition tools
154-
fork.solc_name() # Returns name for the solc compiler
155-
fork.solc_min_version() # Returns minimum solc version supporting this fork
156154
fork.is_deployed() # Returns whether the fork is deployed to mainnet
157155
```
158156

0 commit comments

Comments
 (0)