feat(fw): Add with_all_contract_creating_tx_types marker (#602)

* fix(evm_transition_tool): Allow formatting to="" for some tool types

* fix(fw,types): Make Transaction more permissible to always create a valid tx

* feat(forks): Add `contract_creating_tx_types` class method to forks

* feat(plugins): create "with_all_contract_creating_tx_types" marker

* docs: Add test markers page

* fix(fw): remove auto-fill blob_versioned_hashes

* chengelog

* Update src/evm_transition_tool/evmone.py

Co-authored-by: spencer <[email protected]>

* Update src/evm_transition_tool/transition_tool.py

Co-authored-by: spencer <[email protected]>

* Update src/evm_transition_tool/transition_tool.py

Co-authored-by: spencer <[email protected]>

* docs: lint

---------

Co-authored-by: spencer <[email protected]>

Author: marioevz

docs/CHANGELOG.md

@@ -37,6 +37,7 @@ Test fixtures for use by clients are available for each release on the [Github r
 - ✨ Add more Transaction and Block exceptions from existing ethereum/tests repo ([#572](https://github.com/ethereum/execution-spec-tests/pull/572)).
 - ✨ Add "description" and "url" fields containing test case documentation and a source code permalink to fixtures during `fill` and use them in `consume`-generated Hive test reports ([#579](https://github.com/ethereum/execution-spec-tests/pull/579)).
 - ✨ Add git workflow evmone coverage script for any new lines mentioned in converted_ethereum_tests.txt ([#503](https://github.com/ethereum/execution-spec-tests/pull/503)).
+- ✨ Add a new covariant marker `with_all_contract_creating_tx_types` that allows automatic parametrization of a test with all contract-creating transaction types at the current executing fork ([#602](https://github.com/ethereum/execution-spec-tests/pull/602)).
 
 ### 🔧 EVM Tools
 

docs/navigation.md

@@ -13,6 +13,7 @@
     * [Types of Test](writing_tests/types_of_tests.md)
     * [Adding a New Test](writing_tests/adding_a_new_test.md)
     * [Writing a New Test](writing_tests/writing_a_new_test.md)
+    * [Test Markers](writing_tests/test_markers.md)
     * [Referencing an EIP Spec Version](writing_tests/reference_specification.md)
     * [Verifying Changes Locally](writing_tests/verifying_changes.md)
     * [Exception Tests](writing_tests/exception_tests.md)

docs/writing_tests/test_markers.md

@@ -0,0 +1,111 @@
+# Test Markers
+
+Test markers are used to categorize tests and to run specific subsets of tests. They are defined in the test files using the `pytest.mark` decorator.
+
+## Fork Markers
+
+These markers are used to specify the forks for which a test is valid.
+
+### pytest.mark.valid_from("FORK_NAME")
+
+This marker is used to specify the fork from which the test is valid. The test will not be filled for forks before the specified fork.
+
+```python
+import pytest
+
+@pytest.mark.valid_from("London")
+def test_something_only_valid_after_london():
+    pass
+```
+
+In this example, the test will only be filled for the London fork and after, e.g. London, Paris, Shanghai, Cancun, etc.
+
+### pytest.mark.valid_until("FORK_NAME")
+
+This marker is used to specify the fork until which the test is valid. The test will not be filled for forks after the specified fork.
+
+```python
+import pytest
+
+@pytest.mark.valid_until("London")
+def test_something_only_valid_until_london():
+    pass
+```
+
+In this example, the test will only be filled for the London fork and before, e.g. London, Berlin, Istanbul, etc.
+
+### pytest.mark.valid_at_transition_to("FORK_NAME")
+
+This marker is used to specify that a test is only meant to be filled at the transition to the specified fork.
+
+The test usually starts at the fork prior to the specified fork at genesis and at block 5 (for pre-merge forks) or at timestamp 15,000 (for post-merge forks) the fork transition occurs.
+
+## Fork Covariant Markers
+
+These markers are used in conjunction with the fork markers to automatically parameterize tests with values that are valid for the fork being tested.
+
+### pytest.mark.with_all_tx_types
+
+This marker is used to automatically parameterize a test with all transaction types that are valid for the fork being tested.
+
+```python
+import pytest
+
+@pytest.mark.with_all_tx_types
+@pytest.mark.valid_from("Berlin")
+def test_something_with_all_tx_types(tx_type):
+    pass
+```
+
+In this example, the test will be parameterized for parameter `tx_type` with values `[0, 1]` for fork Berlin, but with values `[0, 1, 2]` for fork London (because of EIP-1559).
+
+### pytest.mark.with_all_contract_creating_tx_types
+
+This marker is used to automatically parameterize a test with all contract creating transaction types that are valid for the fork being tested.
+
+This marker only differs from `pytest.mark.with_all_tx_types` in that it does not include transaction type 3 (Blob Transaction type) on fork Cancun and after.
+
+### pytest.mark.with_all_precompiles
+
+This marker is used to automatically parameterize a test with all precompiles that are valid for the fork being tested.
+
+```python
+import pytest
+
+@pytest.mark.with_all_precompiles
+@pytest.mark.valid_from("Shanghai")
+def test_something_with_all_precompiles(precompile):
+    pass
+```
+
+In this example, the test will be parameterized for parameter `precompile` with values `[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]` for fork Shanghai, but with values `[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]` for fork Cancun (because of EIP-4844).
+
+## Other Markers
+
+### pytest.mark.slow
+
+This marker is used to mark tests that are slow to run. These tests are not run during tox testing, and are only run when a release is being prepared.
+
+### pytest.mark.skip("reason")
+
+This marker is used to skip a test with a reason.
+
+```python
+import pytest
+
+@pytest.mark.skip("Not implemented")
+def test_something():
+    pass
+```
+
+### pytest.mark.xfail("reason")
+
+This marker is used to mark a test as expected to fail.
+
+```python
+import pytest
+
+@pytest.mark.xfail("EVM binary doesn't support this opcode")
+def test_something():
+    pass
+```

src/ethereum_test_forks/base_fork.py

@@ -181,6 +181,14 @@ def tx_types(cls, block_number: int = 0, timestamp: int = 0) -> List[int]:
         """
         pass
 
+    @classmethod
+    @abstractmethod
+    def contract_creating_tx_types(cls, block_number: int = 0, timestamp: int = 0) -> List[int]:
+        """
+        Returns a list of the transaction types supported by the fork that can create contracts
+        """
+        pass
+
     @classmethod
     @abstractmethod
     def precompiles(cls, block_number: int = 0, timestamp: int = 0) -> List[int]:

src/ethereum_test_forks/forks/forks.py

@@ -156,6 +156,13 @@ def tx_types(cls, block_number: int = 0, timestamp: int = 0) -> List[int]:
         """
         return [0]
 
+    @classmethod
+    def contract_creating_tx_types(cls, block_number: int = 0, timestamp: int = 0) -> List[int]:
+        """
+        At Genesis, only legacy transactions are allowed
+        """
+        return [0]
+
     @classmethod
     def precompiles(cls, block_number: int = 0, timestamp: int = 0) -> List[int]:
         """
@@ -274,6 +281,13 @@ def tx_types(cls, block_number: int = 0, timestamp: int = 0) -> List[int]:
         """
         return [1] + super(Berlin, cls).tx_types(block_number, timestamp)
 
+    @classmethod
+    def contract_creating_tx_types(cls, block_number: int = 0, timestamp: int = 0) -> List[int]:
+        """
+        At Berlin, access list transactions are introduced
+        """
+        return [1] + super(Berlin, cls).contract_creating_tx_types(block_number, timestamp)
+
 
 class London(Berlin):
     """
@@ -294,6 +308,13 @@ def tx_types(cls, block_number: int = 0, timestamp: int = 0) -> List[int]:
         """
         return [2] + super(London, cls).tx_types(block_number, timestamp)
 
+    @classmethod
+    def contract_creating_tx_types(cls, block_number: int = 0, timestamp: int = 0) -> List[int]:
+        """
+        At London, dynamic fee transactions are introduced
+        """
+        return [2] + super(London, cls).contract_creating_tx_types(block_number, timestamp)
+
 
 # Glacier forks skipped, unless explicitly specified
 class ArrowGlacier(London, solc_name="london", ignore=True):

src/ethereum_test_tools/common/types.py

@@ -937,39 +937,37 @@ def model_post_init(self, __context):
             or self.max_fee_per_blob_gas is not None
         ):
             raise Transaction.InvalidFeePayment()
-
-        if (
-            self.gas_price is None
-            and self.max_fee_per_gas is None
-            and self.max_priority_fee_per_gas is None
-            and self.max_fee_per_blob_gas is None
-        ):
-            self.gas_price = 10
-
-        if self.v is not None and self.secret_key is not None:
-            raise Transaction.InvalidSignaturePrivateKey()
-
-        if self.v is None and self.secret_key is None:
-            self.secret_key = Hash(TestPrivateKey)
-
         if "ty" not in self.model_fields_set:
             # Try to deduce transaction type from included fields
-            if self.max_fee_per_blob_gas is not None:
+            if self.max_fee_per_blob_gas is not None or self.blob_kzg_commitments is not None:
                 self.ty = 3
-            elif self.max_fee_per_gas is not None:
+            elif self.max_fee_per_gas is not None or self.max_priority_fee_per_gas is not None:
                 self.ty = 2
             elif self.access_list is not None:
                 self.ty = 1
             else:
                 self.ty = 0
 
+        if self.v is not None and self.secret_key is not None:
+            raise Transaction.InvalidSignaturePrivateKey()
+
+        if self.v is None and self.secret_key is None:
+            self.secret_key = Hash(TestPrivateKey)
+
         # Set default values for fields that are required for certain tx types
+        if self.ty <= 1 and self.gas_price is None:
+            self.gas_price = 10
         if self.ty >= 1 and self.access_list is None:
             self.access_list = []
 
+        if self.ty >= 2 and self.max_fee_per_gas is None:
+            self.max_fee_per_gas = 7
         if self.ty >= 2 and self.max_priority_fee_per_gas is None:
             self.max_priority_fee_per_gas = 0
 
+        if self.ty == 3 and self.max_fee_per_blob_gas is None:
+            self.max_fee_per_blob_gas = 1
+
     def with_error(
         self, error: List[TransactionException] | TransactionException
     ) -> "Transaction":

src/evm_transition_tool/evmone.py

@@ -37,3 +37,11 @@ def is_fork_supported(self, fork: Fork) -> bool:
         Currently, evmone-t8n provides no way to determine supported forks.
         """
         return True
+
+    @classmethod
+    def empty_string_to(cls) -> bool:
+        """
+        Evmone requires an empty string within the `to` field for
+        contract-creating transactions.
+        """
+        return True

src/evm_transition_tool/transition_tool.py

@@ -225,6 +225,14 @@ def detect_binary(cls, binary_output: str) -> bool:
 
         return cls.detect_binary_pattern.match(binary_output) is not None
 
+    @classmethod
+    def empty_string_to(cls) -> bool:
+        """
+        Returns True if the tool requires an empty string `to` field
+        for contract creating transactions.
+        """
+        return False
+
     def version(self) -> str:
         """
         Return name and version of tool used to state transition
@@ -311,6 +319,15 @@ class TransitionToolData:
         fork_name: str
         chain_id: int = field(default=1)
         reward: int = field(default=0)
+        empty_string_to: bool = field(default=False)
+
+        def __post_init__(self):
+            """
+            Ensure that the `to` field of transactions is not None
+            """
+            if self.empty_string_to:
+                for tx in self.txs:
+                    tx["to"] = tx.get("to") or ""
 
     def _evaluate_filesystem(
         self,
@@ -571,7 +588,13 @@ def evaluate(
         if int(env["currentNumber"], 0) == 0:
             reward = -1
         t8n_data = TransitionTool.TransitionToolData(
-            alloc=alloc, txs=txs, env=env, fork_name=fork_name, chain_id=chain_id, reward=reward
+            alloc=alloc,
+            txs=txs,
+            env=env,
+            fork_name=fork_name,
+            chain_id=chain_id,
+            reward=reward,
+            empty_string_to=self.empty_string_to(),
         )
 
         if self.t8n_use_stream:

src/pytest_plugins/forks/forks.py

@@ -139,6 +139,13 @@ def add_values(self, metafunc: Metafunc, fork_parametrizer: ForkParametrizer) ->
         fork_attribute_name="tx_types",
         parameter_name="tx_type",
     ),
+    CovariantDescriptor(
+        marker_name="with_all_contract_creating_tx_types",
+        description="marks a test to be parametrized for all tx types that can create a contract"
+        " at parameter named tx_type of type int",
+        fork_attribute_name="contract_creating_tx_types",
+        parameter_name="tx_type",
+    ),
     CovariantDescriptor(
         marker_name="with_all_precompiles",
         description="marks a test to be parametrized for all precompiles at parameter named"