framework: ability to use test or prod signature scheme in tests

[unnawut]

🗒️ Description

In this PR, there are no more independent references to TEST_SIGNATURE_SCHEME or PROD_SIGNATURE_SCHEME in the test vector framework nor the test cases. All usage go through lean_spec.config.LEAN_ENV. This should help ensure that the signature scheme setting is consistent across all tests.

Defaults around key managers and signature operations are removed so the configured scheme is always trickled down from the starting point than trying to default mid-way. The exception is in the client subspecs like signed_block_with_attestation.verify_signatures() and store.on_block() where they default to PROD_SIGNATURE_SCHEME for specs clarity.

Key changes:

1. Consolidated _get_shared_key_manager() from each fixture into consensus_testing/keys.py so there's only one place to set the signature scheme for the key manager
2. Added LEAN_ENV env var that takes in either "test" or "prod". I avoid using SIGNATURE_SCHEME or similar because it gets really ambiguous when it's used in the xmss subspecs.
3. LEAN_ENV parsing:
   • For subspecs, LEAN_ENV is parsed only once by xmss.constants. This ensures the entire subspec uses a consistent value.
   • For tests, LEAN_ENV is defaulted to LEAN_ENV="test" in tests/conftest.py. It's then parsed and consumed mainly by BaseFixture.lean_env() as a computed field. There's an additional parsing by test_types.genesis.generate_pre_state() because the function is independent from the fixtures.
4. If LEAN_ENV is not set:
   • For subspecs, it defaults to TARGET_CONFIG=PROD_CONFIG and TARGET_SIGNATURE_SCHEME=PROD_SIGNATURE_SCHEME, so specs are consumed with prod config/scheme by default.
   • For tests, it defaults LEAN_ENV="test", so tests are run in test config/scheme by default, which in turn would also configure subspecs to use test config.
5. Splitted test-keys.json into individual key files. This helps with managing prod keys that would've been ~200MB for 12 validators in a single file.
6. Added consensus.keys --download option in addition to the existing keygen so pre-generated prod keys can be downloaded.
   • The pre-generated keys currently live in https://github.com/leanEthereum/leansig-test-keys releases. Thinking this is the simplest solution without adding a new kind of infra.
7. Added CI workflow that triggers test filling with prod keys on a main branch commit, export the vectors as as a build artifact

✅ Checklist

• Ran tox checks to avoid unnecessary CI fails:

  uvx tox

• Considered adding appropriate tests for the changes.
• Considered updating the online docs in the ./docs/ directory.

[unnawut]

@fselmo @tcoratger Some updates to the PR:

1. I've replaced --scheme flag with LEAN_ENV env var that takes in either "test" or "prod". I avoid using SIGNATURE_SCHEME or similar because it gets really ambiguous when it's used in the xmss subspecs.
2. LEAN_ENV parsing:
   • For subspecs, LEAN_ENV is parsed only once by xmss.constants. This ensures the entire subspec uses a consistent value.
   • For tests, LEAN_ENV is defaulted to LEAN_ENV="test" in tests/conftest.py. It's then parsed and consumed mainly by BaseFixture.lean_env() as a computed field. There's an additional parsing by test_types.genesis.generate_pre_state() because the function is independent from the fixtures.
3. If LEAN_ENV is not set:
   • For subspecs, it defaults to TARGET_CONFIG=PROD_CONFIG and TARGET_SIGNATURE_SCHEME=PROD_SIGNATURE_SCHEME, so specs are consumed with prod config/scheme by default.
   • For tests, it defaults LEAN_ENV="test", so tests are run in test config/scheme by default, which in turn would also configure subspecs to use test config.
4. Splitted test-keys.json into individual key files. This helps with managing prod keys that would've been ~200MB for 12 validators in a single file.
5. Added consensus.keys --download option in addition to the existing keygen so pre-generated prod keys can be downloaded.
   • The pre-generated keys currently live in https://github.com/unnawut/leansig-test-keys releases. Thinking this is the simplest solution without adding a new kind of infra. Happy to transfer over the repo to leanEthereum if this approach makes sense
6. Added CI workflow that triggers test filling with prod keys on a main branch commit, export the vectors as as a build artifact

I havn't broken down to smaller PRs because the approach changes how different components interact. Happy to break down to smaller PRs if the overall approach looks okay.

[tcoratger]

Sounds super cool, thanks a lot, just two super small nits for the rest, looks good to me.

[fselmo]

This seems great 👌🏼. Very similar to what I was imagining as well. I just left some small things to think about.

I think also it could be nicer for DevEx if we had a flag for fill, something like --scheme that overrides the env var for that run perhaps. If you ran with --scheme=prod it would override any sort of test env var you had just for that run.

I think it would be as simple as adding a scheme flag to the click API for fill and doing something like:

 if scheme:
        os.environ["LEAN_ENV"] = scheme.lower()

So the order of precedence would be:

• CLI flag (sets os.environ["LEAN_ENV"] which is only used on current run)
• If no CLI flag, reads from LEAN_ENV from global user settings
• If no LEAN_ENV, defaults to test

Thoughts? Either way looks good from my end 👍🏼

---

edit: Ah, it looks like it was in there at one point but then removed in 6eb89e1.

Added --scheme flag to the test filler. This is where the scheme gets defined for usage

Curious on your thoughts on keeping it vs removing 👀

[unnawut]

edit: Ah, it looks like it was in there at one point but then removed in 6eb89e1.

Added --scheme flag to the test filler. This is where the scheme gets defined for usage

Curious on your thoughts on keeping it vs removing 👀

I also liked to use --scheme than setting the env var because it feels more natural to me. My thinking was that I didn't want to pre-emptively add multiple ways of achieving the same behavior unless suggested. And LEAN_ENV was the single way that could apply globally to both the subspecs and the tests and hence why I went with env var and removed the --scheme flag which is only applicable to tests.

But since you suggested + also my preference, let's add it back!

[unnawut]

@fselmo On your suggestions to remove the env default from conftest.py and adding --scheme back...

The xmss subspecs are assigning these env-dependent config on module init: e.g. HASH_DIGEST_LENGTH, vector length and LAYERS_LIMIT. I couldn't think of a way to make the env selection applies before these module init without conftest.py. Similarly, CLI flags are parsed in filler.py which afaik is also after module init.

I think the only way I can do your suggestions is to have the spec modules loaded after the filler parses the CLI args, but I'm not sure if there's a way to do that.

---

Btw. the --scheme worked in earlier commits because back then I changed the spec's verify_signature() to take the scheme as an additional function param so it can be passed in from the tests, but it didn't solve that the rest of the spec configs were still hardcoded to prod on module init.

[tcoratger]

Thanks a lot @unnawut good to merge on my side.

[fselmo]

@unnawut I think the flag is actually quite fine. I did some testing around it. If we trace it, we start at fill.py, we set the env var, then we load the config, then we load the types which read the config. The CLI flag works fine and we can default it to test for filler, and keep the conftest.py for the pytest tests (while those are still around). I wanted to just add my change here as a commit on yours so you can feel free to drop the single commit if you don't think it's worth it :)

Let me know what you think. I didn't want to rebase your branch bc it would require a force push and I didn't want to clean any of your history but we have a new typechecker, ty, so if you wanted to make sure we will pass the lint it might be good to rebase and fix conflicts locally.

Another thing is I noticed we still don't have prod config available, right? Did we want to generate those and add it here? I tried testing with --scheme=prod and realized everything breaks 😅

[unnawut]

@unnawut I think the flag is actually quite fine. I did some testing around it. If we trace it, we start at fill.py, we set the env var, then we load the config, then we load the types which read the config. The CLI flag works fine and we can default it to test for filler, and keep the conftest.py for the pytest tests (while those are still around). I wanted to just add my change here as a commit on yours so you can feel free to drop the single commit if you don't think it's worth it :)

Let me know what you think. I didn't want to rebase your branch bc it would require a force push and I didn't want to clean any of your history but we have a new typechecker, ty, so if you wanted to make sure we will pass the lint it might be good to rebase and fix conflicts locally.

Doing via fill.py looks good! I overlooked this file and only looked at filler.py.

Another thing is I noticed we still don't have prod config available, right? Did we want to generate those and add it here? I tried testing with --scheme=prod and realized everything breaks 😅

Because the test keys are over 200 MB, I didn't add them into the PR/repo and expected it to be downloaded via python -m consensus_testing.keys --download --scheme prod. I've just updated the code to download the keys automatically before running the fill here: d686e59

[unnawut]

@tcoratger @fselmo I think the PR features are quite stable now and remaining comments are quite small enough that if there're more changes needed I can do follow up PRs so I'll merge now

[fselmo]

Sounds good, yeah I approve :)