Skip to content

Comments

test: Orderbook TXE tests#16451

Merged
benesjan merged 1 commit intonextfrom
08-19-test_orderbook_txe_tests
Aug 29, 2025
Merged

test: Orderbook TXE tests#16451
benesjan merged 1 commit intonextfrom
08-19-test_orderbook_txe_tests

Conversation

@benesjan
Copy link
Contributor

@benesjan benesjan commented Aug 19, 2025

Fixes #14525

We didn't have the Orderbook thoroughly tested as when I implemented it TXE was not yet reasonable. Now it's reasonable so I decided to tackle this. Since AI is known to be good at tests I decided to gave it a try.

AI experiment

This was the prompt + well curated the context window by opening the relevant tabs in Cursor:

write the orderbook tests in noir-projects/noir-contracts/contracts/app/orderbook_contract/src/test.nr

note that the Orderbook contract accepts 2 token contracts on the input so you will need to deploy those first. Use the setup and setup and mint functionality from noir-projects/noir-contracts/contracts/app/token_contract/src/test/utils.nr.

From the orderbook you want to test the setup order and create order functions. There is also already a yarn-project/end-to-end/src/e2e_orderbook.test.ts test but that is written in TypeScript and noit Noir and it tests only the happy path. In the Noir tests I would also like to test unhappy paths. See tests in noir-projects/noir-contracts/contracts/app/token_contract/src/test/transfer_in_private.nr for inspiration on how to structure the Noir tests.

Overall it managed to implement it all. The only thing I need to help it with is that it didn't understand that it needs to refer the token_contract crate when deploying:

env.deploy("Token")
env.deploy("@token_contract/Token")

Would say this was caused by us not having enough examples and me not putting that into a context window.

Other than this it managed to resolve on its own one other issue (tried to mint with an account with token minter permissions).

The only other thing I did is that I nuked quite a few tests it did because it was for example testing that a function call fails when it doesn't have token transfer authwit provided. Felt like this is sufficient to have only in the token tests.

Update on AI

Code quality was not great and had to redo quite a lot of stuff. That could be improved though if we had a proper reference (plenty of the style issues were present in the rest of our codebase).

Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@benesjan benesjan force-pushed the 08-19-test_orderbook_txe_tests branch from 2e8fde0 to 0af57d4 Compare August 19, 2025 09:15
@benesjan benesjan marked this pull request as ready for review August 19, 2025 09:47
@benesjan benesjan force-pushed the 08-19-test_orderbook_txe_tests branch from f997615 to 400eb9d Compare August 19, 2025 09:59
@benesjan benesjan mentioned this pull request Aug 19, 2025
@benesjan benesjan requested review from Thunkar and nventuro August 19, 2025 12:29

// Utility function to setup orderbook with two tokens
unconstrained fn setup_orderbook_with_tokens(
with_account_contracts: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the only contract tests that exist are are the ones we wrote, I think it'd be good for these to serve as a sort of reference that can be copied by others (e.g. like you did here). Because of that I'd focus not just on coverage but also overall style etc.

I don't like passing this boolean here for a number of reasons. One, it is a bit obsure (setup(true)), two, the owner doesn't really need to be an account contract since all they'll do is mint, not sign authwits.

Copy link
Contributor Author

@benesjan benesjan Aug 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great point. The bool is ugly.

In this case I solved it by just moving the minter account creation out of the function. It made sense to do because the setup is only 1 line and we've already setup the taker account out of the setup in the full_flow test.

If you like how it looks now I can also update other tests were we have this ugly bool (Token and NFT tests). As you say seems to make sense to do as AI is able to do a useful job if you give it a good reference.

Also just renamed the function to setup as the original name didn't describe it well enough (it also sets up test env and the minter account).

@benesjan benesjan marked this pull request as draft August 25, 2025 09:15
@benesjan benesjan force-pushed the 08-19-test_orderbook_txe_tests branch from 400eb9d to b956650 Compare August 25, 2025 09:15
@benesjan benesjan marked this pull request as ready for review August 25, 2025 10:30
@benesjan benesjan requested a review from nventuro August 25, 2025 10:55
AztecBot pushed a commit that referenced this pull request Aug 26, 2025
Fixes #10289

Just like my [other recent PR](#16451) decided to add TXE tests for AMM.

## On using AI here
The only thing I did manually here is to merge a few test cases because they were too long-running as setting up the initial state is costly. I also told it to not have precise ratio in the second call to `add_liquidity` to test the refund. Other than this AI pretty much managed to nail it.

### Update on AI
Code quality was not great and had to redo quite a lot of stuff. That could be improved though if we had a proper reference (plenty of the style issues were present in the rest of our codebase).
github-merge-queue bot pushed a commit that referenced this pull request Aug 26, 2025
Fixes #10289

Just like my [other recent
PR](#16451) decided
to add TXE tests for AMM.

## On using AI here
The only thing I did manually here is to merge a few test cases because
they were too long-running as setting up the initial state is costly. I
also told it to not have precise ratio in the second call to
`add_liquidity` to test the refund. Other than this AI pretty much
managed to nail it.

### Update on AI
Code quality was not great and had to redo quite a lot of stuff. That
could be improved though if we had a proper reference (plenty of the
style issues were present in the rest of our codebase).
@benesjan benesjan marked this pull request as draft August 29, 2025 07:17
@benesjan benesjan force-pushed the 08-19-test_orderbook_txe_tests branch from b42b162 to 095e87a Compare August 29, 2025 07:19
@benesjan benesjan marked this pull request as ready for review August 29, 2025 07:35
@benesjan benesjan enabled auto-merge August 29, 2025 07:43
Fixes #14525

We didn't have the Orderbook thoroughly tested as when I implemented it TXE was not yet reasonable. Now it's reasonable so I decided to tackle this. Since AI is known to be good at tests I decided to gave it a try.

## AI experiment
This was the prompt + well curated the context window by opening the relevant tabs in Cursor:

> write the orderbook tests in noir-projects/noir-contracts/contracts/app/orderbook_contract/src/test.nr
>
> note that the Orderbook contract accepts 2 token contracts on the input so you will need to deploy those first. Use the setup and setup and mint functionality from noir-projects/noir-contracts/contracts/app/token_contract/src/test/utils.nr.
>
> From the orderbook you want to test the setup order and create order functions. There is also already a yarn-project/end-to-end/src/e2e_orderbook.test.ts test but that is written in TypeScript and noit Noir and it tests only the happy path. In the Noir tests I would also like to test unhappy paths. See tests in noir-projects/noir-contracts/contracts/app/token_contract/src/test/transfer_in_private.nr for inspiration on how to structure the Noir tests.

Overall it managed to implement it all. The only thing I need to help it with is that it didn't understand that it needs to refer the `token_contract` crate when deploying:

```diff
env.deploy("Token")
env.deploy("@token_contract/Token")
```

Would say this was caused by us not having enough examples and me not putting that into a context window.

Other than this it managed to resolve on its own one other issue (tried to mint with an account with token minter permissions).

The only other thing I did is that I nuked quite a few tests it did because it was for example testing that a function call fails when it doesn't have token transfer authwit provided. Felt like this is sufficient to have only in the token tests.

### Update on AI
Code quality was not great and had to redo quite a lot of stuff. That could be improved though if we had a proper reference (plenty of the style issues were present in the rest of our codebase).

Co-authored-by: Jan Beneš <janbenes1234@gmail.com>
Co-authored-by: benesjan <janbenes1234@gmail.com>
@AztecBot AztecBot force-pushed the 08-19-test_orderbook_txe_tests branch from 41319fd to 9439f5d Compare August 29, 2025 08:03
@benesjan benesjan added this pull request to the merge queue Aug 29, 2025
auto-merge was automatically disabled August 29, 2025 08:27

Pull Request is not mergeable

Merged via the queue into next with commit 62f076a Aug 29, 2025
16 checks passed
@benesjan benesjan deleted the 08-19-test_orderbook_txe_tests branch August 29, 2025 08:47
mralj pushed a commit that referenced this pull request Oct 13, 2025
Fixes #10289

Just like my [other recent PR](#16451) decided to add TXE tests for AMM.

## On using AI here
The only thing I did manually here is to merge a few test cases because they were too long-running as setting up the initial state is costly. I also told it to not have precise ratio in the second call to `add_liquidity` to test the refund. Other than this AI pretty much managed to nail it.

### Update on AI
Code quality was not great and had to redo quite a lot of stuff. That could be improved though if we had a proper reference (plenty of the style issues were present in the rest of our codebase).
ludamad pushed a commit that referenced this pull request Dec 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[TXE] Write thorough Orderbook tests

3 participants