Specification of WiredTiger Storage API

Modeling the WiredTiger storage layer interface in Storage.tla (explore the spec here).

See operation types from C++ lightweight model-based verification harness here.

Model-Based Testing

We have experimented with automated, model-based test case generation for checking that the WiredTiger implementation conforms to the Storage.tla storage interface model. Essentially, we generate WiredTiger unit test cases by computing path coverings of the reachable state space of the Storage model. We then use these test cases to check that the WiredTiger implementation conforms to the Storage model.

The basic workflow to generate these test cases from the storage model is implemented in the testgen.py script e.g. to run a small model with 2 transactions and 2 keys, you can execute the following:

python3 testgen.py --parallel_test_split 4 --compact --constants TxnId "{t1,t2}" Keys "{k1,k2}" Timestamps "{1,2,3}" --coverage_pct 1.0

this will generate WiredTiger unit tests files in tests/test_txn_model_traces_*.py files, which can be directly run against a WiredTiger build.

There is also a testrun.sh script that executes the above test generation script, copies over the test files to a sibling wiredtiger build directory, and then executes all of these generated tests. For example, you can execute

./testrun.sh 0.5

to generate a set of unit test cases that aim to achieve 50% state coverage of the Storage model, and will then run these tests against a current WiredTiger build.

Note that this code also makes use of a modified build of TLC whose code lives on this branch.