feat(specs,filler,ci): Merge static tests into stable/develop, make --generate-all-formats compatible with static tests (#2006)

* feat(ci): Embed static tests into stable and develop

* fix(cli): Remove the `--fill-static-tests` limitation from two-phase filling

* fix(specs,fixtures): Allow fixture formats to skip a test from being filled based on its markers

* feat(fixtures): Add FixtureFillingPhase

* refactor(fill): Fixture format selection in normal and static tests

* fix: tox

* docs: changelog

* Update src/pytest_plugins/filler/filler.py

Co-authored-by: Copilot <[email protected]>

* refactor(fill): extract phase management into testable components (#2010)

* feat(filler): add PhaseManager and FormatSelector classes with unit tests

Extract phase determination and format selection logic into clean,
testable classes. This prepares for simplifying the complex
conditional logic in the filler plugins.

- Add PhaseManager class with from_config() factory method.
- Add FormatSelector class with strategy-based format selection.
- Add comprehensive unit tests for both classes.
- Test all flag combinations including generate_all_formats edge case.
- Document two-phase execution model (separate pytest runs).
- Maintain backward compatibility with existing functions.

* feat(filler): add FillingSession class with unit tests

- Add FillingSession class to manage all state for a single pytest run.
- Encapsulates phase management, format selection, and pre-alloc groups.
- Handles loading/saving pre-alloc groups between phases.
- Provides xdist support for aggregating worker groups.
- Add comprehensive unit tests for all FillingSession functionality.
- Fix parameter naming to avoid shadowing Python builtin 'hash'.

* refactor(filler): Refactor pytest_configure and pytest_sessionstart

- Updated pytest_configure to use FillingSession instance
- Store phase info on config for backward compatibility
- Simplified pytest_sessionstart to delegate to FillingSession
- All pre-alloc group management now handled by FillingSession

* refactor(filler): Refactor pytest_generate_tests and format selection

- Updated pytest_generate_tests to use FillingSession.should_generate_format().
- Refactored base_test_parametrizer to use FillingSession for pre-alloc management.
- Replaced direct config access with session methods throughout.
- All format selection logic now centralized in FillingSession.

* refactor(filler): Update static_filler to use shared session

- Replaced select_format_by_phases import with get_filling_session.
- Updated format selection logic to use FillingSession.should_generate_format().
- Removed unused generate_all_formats variable.
- All phase management now delegated to FillingSession.

* refactor(filler): Remove old phase management functions and cleanup

- Removed _determine_filling_phases() function.
- Removed select_format_by_phases() function.
- Updated remaining direct config access to use FillingSession.
- Simplified config attribute management.
- Removed unused Tuple import.

* style(fill): use relative imports in test modules

Co-authored-by: Copilot <[email protected]>

Apply suggestion from @Copilot

Co-authored-by: Copilot <[email protected]>

Apply suggestion from @Copilot

Co-authored-by: Copilot <[email protected]>

style(fill): fix-up imports

* refactor(filler): Remove global session and use config-based approach

- Removed global _filling_session variable and get_filling_session() function.
- Store FillingSession directly on config object (config.filling_session).
- Updated all references to access session from config objects directly.
- Added type: ignore comments for mypy compatibility.
- Removed global session tests as they're no longer applicable.
- More aligned with standard pytest plugin patterns.

* refactor: typing

* fix: unit tests

* fix: docs issue

---------

Co-authored-by: Mario Vega <[email protected]>

* fix(ci): Ignore `stQuadraticComplexityTest` in fixture building process

* fix(ci): Up timeout of fixture building workflows

* fix(fixtures): Re-enable engine x for untagged static tests

---------

Co-authored-by: Copilot <[email protected]>
Co-authored-by: danceratopz <[email protected]>

Author: 3 people

.github/configs/feature.yaml

@@ -1,15 +1,11 @@
 # Unless filling for special features, all features should fill for previous forks (starting from Frontier) too
 stable:
   evm-type: stable
-  fill-params: --until=Prague
+  fill-params: --until=Prague --fill-static-tests --ignore=tests/static/state_tests/stQuadraticComplexityTest
 
 develop:
   evm-type: develop 
-  fill-params: --until=Osaka
-
-static: # TODO: remove and add `--fill-static-tests` to both develop & static
-  evm-type: static
-  fill-params: --until=Osaka --fill-static-tests ./tests/static
+  fill-params: --until=Osaka --fill-static-tests --ignore=tests/static/state_tests/stQuadraticComplexityTest
 
 benchmark:
   evm-type: benchmark # Evmone only fully supports up to Prague

.github/workflows/fixtures.yaml

@@ -60,6 +60,7 @@ jobs:
           release_name: ${{ matrix.name }}
           uv_version: ${{ vars.UV_VERSION }}
           python_version: ${{ vars.DEFAULT_PYTHON_VERSION }}
+        timeout-minutes: 720
   release:
     runs-on: ubuntu-latest
     needs: build

.github/workflows/fixtures_feature.yaml

@@ -55,6 +55,7 @@ jobs:
           release_name: ${{ matrix.feature }}
           uv_version: ${{ vars.UV_VERSION }}
           python_version: ${{ vars.DEFAULT_PYTHON_VERSION }}
+        timeout-minutes: 720
   release:
     runs-on: ubuntu-latest
     needs: build

docs/CHANGELOG.md

@@ -11,6 +11,7 @@ Test fixtures for use by clients are available for each release on the [Github r
 - Python 3.10 support was removed in this release ([#1808](https://github.com/ethereum/execution-spec-tests/pull/1808)).
 - Tests for the Prague fork are now marked as stable will be included in the `fixtures_stable.tar.gz` tarball from now on.
 - Tests for the Osaka fork are now included in the `fixtures_develop.tar.gz` tarball.
+- `fixtures_static.tar.gz` has been deprecated and filled static tests are now included in `fixtures_stable.tar.gz` and `fixtures_develop.tar.gz`.
 
 #### 💥 Important Change for EEST developers
 
@@ -80,6 +81,7 @@ Users can select any of the artifacts depending on their testing needs for their
 - ✨ Generate unique addresses with Python for compatible static tests, instead of using hard-coded addresses from legacy static test fillers ([#1781](https://github.com/ethereum/execution-spec-tests/pull/1781)).
 - ✨ Added support for the `--benchmark-gas-values` flag in the `fill` command, allowing a single genesis file to be used across different gas limit settings when generating fixtures. ([#1895](https://github.com/ethereum/execution-spec-tests/pull/1895)).
 - ✨ Static tests can now specify a maximum fork where they should be filled for ([#1977](https://github.com/ethereum/execution-spec-tests/pull/1977)).
+- ✨ Static tests can now be filled in every format using `--generate-all-formats` ([#2006](https://github.com/ethereum/execution-spec-tests/pull/2006)).
 
 #### `consume`
 

src/cli/pytest_commands/fill.py

@@ -153,9 +153,6 @@ def _should_use_two_phase_execution(self, args: List[str]) -> bool:
             "--generate-pre-alloc-groups" in args
             or "--generate-all-formats" in args
             or self._is_tarball_output(args)
-        ) and (
-            "--fill-static-tests"
-            not in args  # TODO: remove this once we have better grouping for static tests
         )
 
     def _ensure_generate_all_formats_for_tarball(self, args: List[str]) -> List[str]:

src/ethereum_test_fixtures/__init__.py

@@ -1,6 +1,6 @@
 """Ethereum test fixture format definitions."""
 
-from .base import BaseFixture, FixtureFormat, LabeledFixtureFormat
+from .base import BaseFixture, FixtureFillingPhase, FixtureFormat, LabeledFixtureFormat
 from .blockchain import (
     BlockchainEngineFixture,
     BlockchainEngineFixtureCommon,
@@ -25,6 +25,7 @@
     "EOFFixture",
     "FixtureCollector",
     "FixtureConsumer",
+    "FixtureFillingPhase",
     "FixtureFormat",
     "LabeledFixtureFormat",
     "PreAllocGroups",

src/ethereum_test_fixtures/base.py

@@ -2,9 +2,11 @@
 
 import hashlib
 import json
+from enum import Enum, auto
 from functools import cached_property
-from typing import Annotated, Any, ClassVar, Dict, Type, Union
+from typing import Annotated, Any, ClassVar, Dict, List, Set, Type, Union
 
+import pytest
 from pydantic import (
     Discriminator,
     Field,
@@ -38,6 +40,13 @@ def fixture_format_discriminator(v: Any) -> str | None:
     return fixture_format
 
 
+class FixtureFillingPhase(Enum):
+    """Execution phase for fixture generation."""
+
+    PRE_ALLOC_GENERATION = auto()
+    FILL = auto()
+
+
 class BaseFixture(CamelModel):
     """Represents a base Ethereum test fixture of any type."""
 
@@ -51,6 +60,7 @@ class BaseFixture(CamelModel):
     format_name: ClassVar[str] = ""
     output_file_extension: ClassVar[str] = ".json"
     description: ClassVar[str] = "Unknown fixture format; it has not been set."
+    format_phases: ClassVar[Set[FixtureFillingPhase]] = {FixtureFillingPhase.FILL}
 
     @classmethod
     def output_base_dir_name(cls) -> str:
@@ -145,6 +155,15 @@ def supports_fork(cls, fork: Fork) -> bool:
         """
         return True
 
+    @classmethod
+    def discard_fixture_format_by_marks(
+        cls,
+        fork: Fork,
+        markers: List[pytest.Mark],
+    ) -> bool:
+        """Discard a fixture format from filling if the appropriate marker is used."""
+        return False
+
 
 class LabeledFixtureFormat:
     """
@@ -179,9 +198,14 @@ def __init__(
 
     @property
     def format_name(self) -> str:
-        """Get the execute format name."""
+        """Get the filling format name."""
         return self.format.format_name
 
+    @property
+    def format_phases(self) -> Set[FixtureFillingPhase]:
+        """Get the filling format phases where it should be included."""
+        return self.format.format_phases
+
     def __eq__(self, other: Any) -> bool:
         """
         Check if two labeled fixture formats are equal.

src/ethereum_test_fixtures/blockchain.py

@@ -7,6 +7,7 @@
     ClassVar,
     List,
     Literal,
+    Set,
     Tuple,
     Union,
     cast,
@@ -43,7 +44,7 @@
 from ethereum_test_types.block_types import WithdrawalGeneric
 from ethereum_test_types.transaction_types import TransactionFixtureConverter, TransactionGeneric
 
-from .base import BaseFixture
+from .base import BaseFixture, FixtureFillingPhase
 from .common import FixtureAuthorizationTuple, FixtureBlobSchedule
 
 
@@ -556,6 +557,10 @@ class BlockchainEngineXFixture(BlockchainEngineFixtureCommon):
 
     format_name: ClassVar[str] = "blockchain_test_engine_x"
     description: ClassVar[str] = "Tests that generate a Blockchain Test Engine X fixture."
+    format_phases: ClassVar[Set[FixtureFillingPhase]] = {
+        FixtureFillingPhase.FILL,
+        FixtureFillingPhase.PRE_ALLOC_GENERATION,
+    }
 
     pre_hash: str
     """Hash of the pre-allocation group this test belongs to."""

src/ethereum_test_specs/blockchain.py

@@ -437,10 +437,14 @@ def discard_fixture_format_by_marks(
         markers: List[pytest.Mark],
     ) -> bool:
         """Discard a fixture format from filling if the appropriate marker is used."""
-        if "blockchain_test_only" in [m.name for m in markers]:
-            return fixture_format != BlockchainFixture
-        if "blockchain_test_engine_only" in [m.name for m in markers]:
-            return fixture_format != BlockchainEngineFixture
+        marker_names = [m.name for m in markers]
+        if fixture_format != BlockchainFixture and "blockchain_test_only" in marker_names:
+            return True
+        if (
+            fixture_format not in [BlockchainEngineFixture, BlockchainEngineXFixture]
+            and "blockchain_test_engine_only" in marker_names
+        ):
+            return True
         return False
 
     def get_genesis_environment(self, fork: Fork) -> Environment: