feat(consume): Add consume sync tests to sync clients after engine tests

[fselmo]

🗒️ Description

Add a new test type that will send consume engine tests to client under test and then have a separate client sync to it. Parametrizes both client under test and sync client using hive client config file.

• Fill e.g. uv run fill --fork=Osaka -k rlp_limit --clean (or narrow down to one or two tests)
• Consume e.g. uv run consume sync -v -s (-s for good logs, -v for good test ids)

---

TODO:

• Add documentation once we agree on the code after reviews
• Add entry to CHANGELOG
• Add some appropriate tests for the new command

✅ Checklist

• All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:

  uvx --with=tox-uv tox -e lint,typecheck,spellcheck,markdownlint

• All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
• All: Considered adding an entry to CHANGELOG.md.
• All: Considered updating the online docs in the ./docs/ directory.
• All: Set appropriate labels for the changes (only maintainers can apply labels).

[fselmo]

The logic for checking the verify_sync flag is a little tricky. We check if the flag exists at all on the fixture here (if so, we mark it with sync marker) and then later, inside generate(), when we can actually get the value here, we either skip the test if it's set to False, as in this case with invalid payload, or we generate it.

This way if you use uv run fill --fork=Osaka -k rlp_size_limit_boundary --clean -m blockchain_test_sync_only, it will skip the negative case and only fill the other two cases. Without the -m flag it will fill all of blockchain, blockchain_engine and blockchain_engine_sync and still skip that single invalid sync case but fill all 3 cases for blockchain and blockchain_engine. I just couldn't get it to not skip and just not collect it at all.

I couldn't figure out a good way to completely skip test filling, instead of pytest.skip(), if the value is False, because by the time we are checking the value we have the fixture already. I think this is fine but lmk if you see it another way!

[fselmo]

From Discord:

I'm realizing now that the -m flag doesn't work for fill command for blockchain_test_only or blockchain_test_engine_only. It isn't registered for filling, only for consuming. This is what I was (wrongly) trying to accomplish with sync and why I jumped through those hoops. It seems filling doesn't make the -m distinction so I should not try to for the blockchain sync tests.

Changes incoming 🙂

---

edit: It looks like a -m blockchain_test_sync flag is built dynamically from somewhere and can be used to fill. Perhaps I'm misunderstanding the blockchain_test_*_only flag. I will do some more reading.

edit2: It seems these are different markers altogether and meant to mark tests to only fill one type of fixture. The more you know 🌈 ...

[marioevz]

Looks fantastic! Just a couple of comments, main one being that we should make the refactor of the verify_sync field in the BlockchainTest spec.

[fselmo]

main one being that we should make the refactor of the verify_sync field in the BlockchainTest spec.

Ok yeah, this is the best way I think as well. I'm glad you agree. I was wondering if we wanted to try to keep it this way for backward compatibility but since no tests were turned on yet for this, I think this is the way to go. Will go this route 👍🏼

[fselmo]

@marioevz Updated this to reflect your comments. I had some unideal paths to get to where we got and rebasing main with those changes didn't really add anything useful to the git history so I squashed this all as a single commit. Hopefully that didn't make it too hard to track the changes here.

[marioevz]

Amazing! I feel this is pretty much ready to merge, just made two minor comments.

[marioevz]

Awesome! Thanks so much