Framework for setting up environment needed for ASV tests

General

Writing ASV tests against a single type of storage, moreover of none shared type, such as LMDB, is generally straightforward. All the moving parts, including the storage, are isolated on a specific developer machine or a GitHub runner. However, writing tests that need to use shared storage and can be executed concurrently in multiple locations can become challenging and lead to unpredictable results, as they will start to ovewrite each other's data. Additionally, we aim to reuse the same tests with minimal changes (such as modifying a single parameter) to run against different storage types like LMDB, Amazon S3, GCP, and Azure.

Furthermore, if we want to set up similar environments for other types of tests, we cannot use our current approach for writing tests and setups that are tightly coupled with the ASV test classes.

We need a safer approach that requires adding an additional layer of abstraction to help us achieve this safely and effectively.

What abstraction could solve the problem?

An abstraction that makes all storages appear as shared storages (including LMDB) and safeguards against potential issues, such as overwriting other test data.

To achieve this, we can use prefixes for Amazon S3 and paths for file locations in LMDB.

At the top level (root), there is:

• one Bucket for Amazon S3
• one root folder for LMDB storages

That will be dedicated space for ASV test

From there, we can create several different spaces, dedicated to different needs.

Persistant space

This shared space is meant to be accessed by all machines and clients. It is used to store data that is written once and then read by all clients. Therefore, this space is designated with a common prefix (PERSISTENT_LIBS_PREFIX) in the bucket.

Modifyiable space

This private space is meant for each client and test, isolated from others (all modifiable operations require such space). The prefix for this space is constructed from several parts:

• First, the common name for this prefix (MODIFIABLE_LIBS_PREFIX).
• Then, a unique part for each machine added via an environment variable (ARCTICDB_PERSISTENT_STORAGE_SHARED_PATH_PREFIX). This can be done for GitHub and individually by each person.
• The third part should be defined in each test case.

By concatenating these three parts, we create a separate unique space in the storage for tests that require private control.

Test Space

This space is exactly like the Persistent Space. It is used to test the framework to ensure that every operation working on the persistent space functions correctly. We can take the risk and execute unknown operations on the shared persistent space. Therefore, we need a safe space similar to the real one but not identical. Again, a shared space with a different prefix is needed, but this time, no concatenations are required to make it private.

Note, that above abstraction with prefixes works with LMDB too as this time it will be the folder structure and the name of folders.