feat: add macros for contract consensus tests #6562
Conversation
| StacksEpochId::Epoch32 | ||
| } | ||
|
|
||
| pub const ALL_GTE_30: &'static [StacksEpochId] = &[ |
There was a problem hiding this comment.
the idea of adding this here is that, even if requires manual updating when we add a new epoch, most of the functions in this file will also need to be updated. Expecially the ones above this, fn latest(). I believe it will be difficult to miss it!
| tempfile = "3.3" | ||
| proptest = { version = "1.6.0", default-features = false, features = ["std"] } | ||
| insta = { version = "1.37.0", features = ["ron"] } | ||
| pinny = { git = "https://github.com/BitcoinL2-Labs/pinny-rs.git", rev = "54ba9d533a7b84525a5e65a3eae1a3ae76b9ea49" } #v0.0.2 |
There was a problem hiding this comment.
I added pinny to simplify discovery of consesus tests. The main reason was to easiliy identify the ones relying on insta to easily filter them. But after some discussions we realized that this is not currently needed. As all of our consensus tests are currently within the consensus file. I can remove it
Codecov Report❌ Patch coverage is
❌ Your project check has failed because the head coverage (61.51%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.
Additional details and impacted files@@ Coverage Diff @@
## develop #6562 +/- ##
============================================
- Coverage 74.70% 61.51% -13.20%
============================================
Files 568 568
Lines 347131 347050 -81
============================================
- Hits 259328 213474 -45854
- Misses 87803 133576 +45773
... and 347 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
| insta::allow_duplicates! { | ||
| for res in contract_deploy_results { | ||
| insta::assert_ron_snapshot!("contract_deploys", res, { | ||
| // redact contract name, as it changes per epoch and clarity version | ||
| r#".transactions[].tx[0].name"# => contract_name, | ||
| r#".transactions[].tx[0].code_body"# => "[contract code]", | ||
| // redact contract code, as it may take too much space needlessly | ||
| r#".transactions[].tx[1]"# => "[clarity version]", | ||
| }); | ||
| } | ||
| } | ||
| insta::allow_duplicates! { | ||
| for res in contract_call_results { | ||
| insta::assert_ron_snapshot!("contract_calls", res, { | ||
| // redact contract name, as it changes per epoch and clarity version | ||
| r#".transactions[].tx.*.name"# => contract_name, | ||
| }); | ||
| } | ||
| } |
There was a problem hiding this comment.
This doesn’t work yet, becasuse the MARF is included in the same struct and will always change between blocks. But I think it would be really helpful for a few reasons:
-
It could provide an additional (optional) guarantee that executing the same transaction across different epochs produces the same result — while still allowing certain transactions to behave differently between epochs when needed.
-
When a new epoch is added, the tests would automatically run for that epoch as well. Tests that produce the same result as before would be marked as “passed” automatically, while only those that yield a different output would be marked as failed or “to be updated.”
-
It will help prevent most snapshot become too big and difficult to review.
In order to make it work, we will have to decide if we want to move the list of MARF hashes into a separate snapshot instead of in the same data struct.
| pub struct ExpectedTransactionOutput { | ||
| /// The transaction that was executed. | ||
| /// `None` for bitcoin transactions. | ||
| pub tx: Option<TransactionPayload>, |
There was a problem hiding this comment.
when reviewing the snapshot to check if everything was correct for the matrix of contract deploy/contract call, I found it difficult to understand the mapping between the receipt and the original transaction. So I thought of adding it here. Maybe the TransactionPayload is too much, and would be enough to just save the transaction type?
…-consensus-test-api
Jiloc
changed the title
consensus_test macro1 participant
Description
This PR introduces an overhaul to the smart contract consensus testing suite, replacing manual test setups with a declarative framework. The goal is to make it significantly easier to write robust, multi-epoch, and multi-Clarity-version consensus tests.
We focus on ensuring we don't break consensus for state transitions that already happened, can happen now, or will happen in the future.
Past Events: For consensus-breaking changes in past epochs (e.g., 2.0, 3.1):
Current and Future Events:
New Macros:
contract_call_consensus_test!: Allows defining a full contract deployment and function call test scenario that automatically implements the strategy described above.contract_deploy_consensus_test!: A specialized version for testing only the contract deployment phase, focusing on current and future epochs.Applicable issues
Additional info (benefits, drawbacks, caveats)
Checklist
docs/rpc/openapi.yamlandrpc-endpoints.mdfor v2 endpoints,event-dispatcher.mdfor new events)clarity-benchmarkingrepobitcoin-tests.yml