Database benchmark

[kurtisc]

Closes RV-828

What

A benchmark for the durable storage system at the layer of the Database type, e.g.:

                    commands go in here
                  /
                 v
          Database
-------------------------------
Merkle layer | Persistence layer

Why

It's important to know that the current design of the durable storage system is feasible. When optimising parts of the system, we also want to know if they are effective.

How

1. Populates a large tree
2. Deserialises a list of operations defining a typical ERC-20 transaction
3. Runs the transaction 10000 times

A feature (bench_cloning) has been added to allow benchmarking with and without cloning the Database. This is due to a bug RV-855 where the implementation of cloning the persistence layer is too slow.

Manually Testing

cargo bench --features bench --bench "database"
cargo bench --features bench,bench_cloning --bench "database"

Tasks for the Author

• Link all Linear issues related to this MR using magic words (e.g. part of, relates to, closes).
• Eliminate dead code and other spurious artefacts introduced in your changes.
• Document new public functions, methods and types.
• Make sure the documentation for updated functions, methods, and types is correct.
• Add tests for bugs that have been fixed.
• Explain changes to regression test captures when applicable.
• Write commit messages in agreement with our guidelines.
• Self-review your changes to ensure they are high-quality.
• Complete all of the above before assigning this MR to reviewers.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.00%. Comparing base (0f488ae) to head (38bad96).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #597      +/-   ##
==========================================
+ Coverage   89.61%   91.00%   +1.38%     
==========================================
  Files         113      111       -2     
  Lines       20457    20101     -356     
  Branches    20457    20101     -356     
==========================================
- Hits        18332    18292      -40     
+ Misses       1749     1433     -316     
  Partials      376      376              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Benchmark results for revision 88421a9:

Metric	Duration	TPS
Mean	1.512094521s	26.454
Worst	1.525607802s	26.219
Best	1.490554923s	26.836
Standard Deviation	±7.767013ms	±0.136

Full results

Run	Transfers	Duration	TPS
1	40	1.512723318s	26.442
2	40	1.516678012s	26.373
3	40	1.490554923s	26.836
4	40	1.500020806s	26.666
5	40	1.517706699s	26.356
6	40	1.516130857s	26.383
7	40	1.525607802s	26.219
8	40	1.512564004s	26.445
9	40	1.52228952s	26.276
10	40	1.507029889s	26.542
11	40	1.516444895s	26.377
12	40	1.516021364s	26.385
13	40	1.50621355s	26.557
14	40	1.50878626s	26.511
15	40	1.509429223s	26.500
16	40	1.51097141s	26.473
17	40	1.51233239s	26.449
18	40	1.522425416s	26.274
19	40	1.509598756s	26.497
20	40	1.508361318s	26.519

Compare the results above with those for the default branch.

[danielpapp-trilitech]

Overall, it looks good. I think the large read size is off by a magnitude, and you might also want to include a large write, which has the same as the large read.

[kurtisc]

@danielpapp-trilitech Many changes and a rebase here, including renaming the main file, sorry it's all mixed up.

[kurtisc]

The poor performance of Clone is making this take too long to run, moving into draft briefly while I try to resolve this

[danielpapp-trilitech]

Looks good to me! Good job in using the json that Victor created! How much time does it take to run this?

[kurtisc]

How much time does it take to run this?

Currently it takes 2 minutes to run and the time for a measurement is 4.9s

Most of this is in persistence layer operations, which is preferable, but the erc20 transaction has a lot more operations than I was expecting (132). I think it would be good to add a tool which can generate the store_accesses.json file without Victor having to edit it manually.

[NSant215]

Overall I think looks good - I got a bit confused that methods written in one commit were immediately changed in the next between commits 2 and 3 - perhaps they can be squashed? or is there good reason to separate them?

[kurtisc]

How much time does it take to run this?

Currently it takes 2 minutes to run and the time for a measurement is 4.9s

Most of this is in persistence layer operations, which is preferable, but the erc20 transaction has a lot more operations than I was expecting (132). I think it would be good to add a tool which can generate the store_accesses.json file without Victor having to edit it manually.

Never mind, I've spoken to Joel and the 132 includes setup and block creation, which don't need to be done 10000 times like the transaction itself.

[kurtisc]

I got a bit confused that methods written in one commit were immediately changed in the next between commits 2 and 3 - perhaps they can be squashed? or is there good reason to separate them?

I don't see why not. I wanted to separate the json parsing into its own commit but I think it causes more confusion/temporary code than it is worth.