Skip to content

Conversation

@fubuloubu
Copy link
Member

@fubuloubu fubuloubu commented Jan 21, 2026

What I did

Many people have given the feedback that they wish Ape supported more sophisticated smart contract testing paradigms, such as Foundry's contract-based testing workflow (using Solidity-based tests). This enables the testing of more complex behaviors like reentrancy, as well as more closely mimics the "look and feel" of being an enduser of such contracts (for example, library developers). Most frameworks have found a way to adopt this style of testing due to it's many benefits (including dramatic improvements in speed from executing in single-txn context within tests), so it's about time Ape supported this.

This is especially important to enable more advanced styles of testing like "fuzz" testing, and property/stateful tests (using Hypothesis ofc), which often catch complex bugs in user smart contracts. While it is possible to use Hypothesis with Ape, the experience of doing so creates a massive slow down. As an example, experiments with stateful tests often took almost an hour to execute. Adopting the contract-based testing paradigm cuts down on test execution time when creating sophisticated testing suites from hours to minutes.

fixes: #363
fixes: #351

How I did it

Ape already has excellent support for compiling smart contracts through it's plugin system, so a smart contract-based testing flow starts with detecting contracts in the tests/ folder to treat as "contract tests". In order to integrate well with other such frameworks (for example foundry, we don't want to co-opt the same tests from that framework, instead encouraging an easy migration path), we skip registering tests using the popular .t.{ext} naming style from our tests folder (Ape already supports "alternative" test folder names e.g. test/).

Secondly, Ape has very nice and broad integration with pytest including support for pytest features like fixtures, so it seems best to leverage that in our contract testing workflow to reduce the work of managing test setup and execution. For example, using pytest fixtures in our contract tests is done simply by looking up fixtures from those already registered when running the test (at that folder level) when a test has an argument that matches. Another example is things like markers, which we decided to support by leveraging custom Natspec declarations in our tests (e.g. @custom:test:mark:xfail {reason}).

Using Natspec turned out to be a very rich and easy way to enable this feature more broadly, so we added support for configuring both Pytest and Hypothesis this way. For example, we allow parametrizing contract tests across a range of values for particular argument(s), configuring hypothesis test settings, checking revert reasons when executing tests, or checking for the existence of contract logs (after executing our tests). Due to the use of Natspec, this works seamlessly out of the box with both Solidity and Vyper languages, meaning that other Ape compiler plugins can easily adapt this workflow by simply supporting custom Natspec-style declarations.

How to verify it

ape test tests/integration/*tests

Write some tests

Checklist

  • All changes are completed
  • Change is covered in tests
  • Documentation is complete

@fubuloubu fubuloubu requested a review from antazoey January 21, 2026 17:09
@fubuloubu fubuloubu force-pushed the feat/contract-tests branch 5 times, most recently from 39275c2 to 9cc7972 Compare January 22, 2026 03:55
def pytest_collect_file(parent, file_path):
from ape.utils.basemodel import ManagerAccessMixin

# NOTE: Avoid capturing "foundry tests", which follow a paradigm of `.t.sol`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How come?? This seems like a limitation in a few respects

  1. it'd be nice to run existing foundry tests in a project I use ape in a lot
  2. it may be confusing if people can't name their tests how they are used to naming them

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to allow "gracefully" migrating a larger test suite from Foundry to Ape, or using both side-by-side seamlessly. This requires some way to differentiate "Ape-flavored" contract tests from "Foundry-flavored" ones.

Since this naming convention is the easiest "Foundry-ism" to key off of (note we also ignore these types of files w/ ape compile, only compiling when the full suffix matches), and the solution is very simple (rename the file to try it with Ape), I like this approach best to create this effect

I think this is an important property to maintain with this feature!

@fubuloubu fubuloubu force-pushed the feat/contract-tests branch from 64488a6 to 991b4c7 Compare January 22, 2026 17:28
@fubuloubu fubuloubu force-pushed the feat/contract-tests branch from 991b4c7 to 81fed19 Compare January 22, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Testing: Hypothesis stateful test Testing: Hypothesis property test

2 participants