Skip to content

Commit 1e30af2

Browse files
docs(tests): update porting ethereum/tests instructions (#1872)
Co-authored-by: danceratopz <[email protected]>
1 parent 6e56978 commit 1e30af2

File tree

5 files changed

+28
-7
lines changed

5 files changed

+28
-7
lines changed

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,7 @@ We welcome contributions via pull requests! This section will guide you through
9999

100100
7. **For EVM Tests**: Review the cases in the [EIP checklist template](./docs/writing_tests/checklist_templates/eip_testing_checklist_template.md).
101101

102-
8. **For Porting Tests**: If you're porting tests from ethereum/tests, see the [porting guide](https://eest.ethereum.org/main/dev/porting_legacy_tests) for coverage analysis and using `--skip-coverage-missed-reason` when needed.
102+
8. **For Porting Tests**: If you're porting tests from ethereum/tests, see the [porting guide](https://eest.ethereum.org/main/writing_tests/porting_legacy_tests) for coverage analysis and using `--skip-coverage-missed-reason` when needed.
103103

104104
9. **Verify your changes** by running the appropriate checks:
105105

docs/dev/index.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@ This documentation is aimed at `execution-spec-tests` developers:
99
- [Logging](./logging.md): Documentation on using the custom logging system with enhanced features.
1010
- [Enabling pre-commit checks](./precommit.md): A guide for setting up pre-commit hooks to enforce code quality before commits.
1111
- [Running github actions locally](./test_actions_locally.md): Instructions for testing GitHub Actions workflows on your local machine to streamline development and debugging.
12-
- [Porting tests](./porting_legacy_tests.md): A guide to porting legacy ethereum tests to EEST.
1312

1413
These sections are primarily aimed at `execution-spec-tests` maintainers:
1514

docs/navigation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626
* [EIP Execution Layer Testing Checklist Template](writing_tests/checklist_templates/eip_testing_checklist_template.md)
2727
* [Post-mortems](writing_tests/post_mortems.md)
2828
* [Tutorial: Adding a State Test](writing_tests/tutorials/state_transition.md)
29+
* [Porting Legacy Tests](writing_tests/porting_legacy_tests.md)
2930
* [Filling Tests](filling_tests/index.md)
3031
* [Getting Started](filling_tests/getting_started.md)
3132
* [Filling Tests at a Prompt](filling_tests/filling_tests_command_line.md)
@@ -70,7 +71,6 @@
7071
* [Logging](dev/logging.md)
7172
* [Enabling Precommit Checks](dev/precommit.md)
7273
* [Running Github Actions Locally](dev/test_actions_locally.md)
73-
* [Porting Legacy Tests](dev/porting_legacy_tests.md)
7474
* [Changelog](CHANGELOG.md)
7575
* [Library Reference](library/index.md)
7676
* [EEST CLI Tools](library/cli/index.md)

docs/writing_tests/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,5 +25,6 @@ For help deciding which test format to select, see [Types of Tests](./types_of_t
2525
- [Adding a New Test](./adding_a_new_test.md) - Step-by-step guide to adding new tests
2626
- [Writing a New Test](./writing_a_new_test.md) - Detailed guide on writing different test types
2727
- [Using and Extending Fork Methods](./fork_methods.md) - How to use fork methods to write fork-adaptive tests
28+
- [Porting tests](./porting_legacy_tests.md): A guide to porting @ethereum/tests to EEST.
2829

2930
Please check that your code adheres to the repo's coding standards and read the other pages in this section for more background and an explanation of how to implement state transition and blockchain tests.

docs/dev/porting_legacy_tests.md renamed to docs/writing_tests/porting_legacy_tests.md

Lines changed: 25 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,14 +15,35 @@ While automating the conversion of the remaining YAML (or JSON) test cases to Py
1515

1616
## Porting an original test
1717

18-
1. Select one or more YAML test cases from ethereum/tests to port and create an issue in this repository AND comment on [this tracker issue.](https://github.com/ethereum/execution-spec-tests/issues/972)
18+
1. Select one or more test cases from `./tests/static/state_tests/` to port and create an issue in this repository AND comment on [this tracker issue.](https://github.com/ethereum/execution-spec-tests/issues/972)
1919

2020
2. [Add a new test](../writing_tests/index.md) in the appropriate fork folder, following the guidelines for [choosing a test type.](../writing_tests/types_of_tests.md#deciding-on-a-test-type)
2121

2222
3. Submit a PR with the ported tests:
2323

24-
1. Add the list of ported YAML files to [`converted-ethereum-tests.txt`](https://github.com/ethereum/execution-spec-tests/blob/1b30c336eae6b0746ea4db441ac74406f2fb2322/converted-ethereum-tests.txt).
25-
2. Open a PR to remove the ported tests from the _original tests_ repository.
24+
1. Add the list of ported files using python marker to the head of your python test.
25+
26+
Example:
27+
28+
```python
29+
@pytest.mark.ported_from(
30+
[
31+
"https://github.com/ethereum/tests/blob/v13.3/src/GeneralStateTestsFiller/stCreateTest/CREATE_ContractSuicideDuringInit_ThenStoreThenReturnFiller.json",
32+
"https://github.com/ethereum/tests/blob/v13.3/src/GeneralStateTestsFiller/stCreateTest/CREATE_ContractSuicideDuringInit_WithValueFiller.json",
33+
],
34+
pr=["https://github.com/ethereum/execution-spec-tests/pull/1871"],
35+
# coverage_missed_reason="Converting solidity code result in following opcode not being used:",
36+
```
37+
38+
Replace test names with your chosen tests and PR number.
39+
40+
Uncomment coverage_missed_reason when all the missed coverage lines are approved, usually some opcodes end up not used after translating test logic from lllc, yul.
41+
42+
But sometimes missed coverage line could hint that you forgot to account important test logic.
43+
44+
If no coverage is missed, you are good!
45+
46+
2. Remove the ported files from .tests/static/state_tests in your PR
2647

2748
> See also: 📄 [Getting started with EEST.](../getting_started/repository_overview.md)
2849

@@ -46,7 +67,7 @@ By default, EVM logs are stored in the `logs` folder at the repository root. You
4667

4768
It's crucial that ported tests maintain coverage parity with _original tests_. This ensures that no critical functions are left untested and prevents the introduction of bugs. A CI workflow automatically checks for coverage.
4869

49-
If coverage action fails (See: 📄 [An example of a failing test coverage](https://github.com/ethereum/execution-spec-tests/actions/runs/13037332959/job/36370897481)), it's recommended to run the coverage action locally (see: 📄 [How to run GitHub actions locally](./test_actions_locally.md)), which should generate a `evmtest_coverage` directory:
70+
If coverage action fails (See: 📄 [An example of a failing test coverage](https://github.com/ethereum/execution-spec-tests/actions/runs/13037332959/job/36370897481)), it's recommended to run the coverage action locally (see: 📄 [How to run GitHub actions locally](../dev/test_actions_locally.md)), which should generate a `evmtest_coverage` directory:
5071

5172
```console
5273
❯ tree evmtest_coverage -L 2

0 commit comments

Comments
 (0)