You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/load-testing.md
+39-28Lines changed: 39 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,37 +11,48 @@ The overall goal with the benchmark tool is to simulate how the database would p
11
11
The ability to simulate different workloads is currently being worked on. This means that across releases, the default workload that the benchmark tool runs may vary. See the release notes to see if the benchmark workload has changed.
12
12
:::
13
13
14
-
## Benchmark workload
15
-
16
-
The workload that the engine currently uses (as of v0.8.1) is the following:
17
-
- A model `bench.bench` is created with a primary key (of type `string` and with a column of type `uint8`)
18
-
- Multiple clients are created (simulating "application servers")
19
-
- Queries are run against **unique rows**. This means that unlike `redis-benchmark`**every query group touches a different row, as it would generally do in the real-world**
20
-
- The following queries (collectively a "query group") are run for each unique row:
21
-
- The row is first created with an `INSERT`
22
-
- All columns of the row are returned with a `SELECT`
23
-
- The integer column is incremented with an `UPDATE`
24
-
- The row is finally removed with a `DELETE`
25
-
-**By default, 1,000,000 rows are created and manipulated**
26
-
-**The time taken for each row to be sent, read back and decoded into a readable form is recorded** (time taken to parse into actual language structures such as maps and lists) towards the total time taken, once again unlike many other benchmark tools
27
-
- In total 4,000,000 queries are run (by default)
28
-
29
-
:::caution
30
-
The benchmark tool will create a space `bench` and a model `bench` and will completely remove the space and associated data once the benchmark is complete. **Do not use this space!**
31
-
:::
14
+
## Using the benchmark tool
15
+
16
+
You will need to select a workload from the [below section on workloads](#benchmark-workloads). As workloads need `root` access to the database for creating and removing spaces and tables, you will need to also provide the `root` account password.
17
+
18
+
You may run workloads like this using an argument for the password:
You can also use the `SKYDB_PASSWORD` environment variable if you do not want to use `--password`. You can tune the number of threads, connections, rows created and such to your liking to simulate the environment that you think matches your production setting:
25
+
26
+
-`--connections <count>`: Set the number of client connections (defaults to `8 * number of logical CPUs`)
27
+
-`--threads <count>`: Total number of threads to use. (defaults to the logical CPU count)
28
+
-`--rowcount <count>`: Set the number of unique rows to run workload sequences on
29
+
-`--keysize <size>`: Set the number of bytes to use for the primary key
30
+
31
+
See the help menu using `sky-bench --help` to use additional configuration options.
32
+
33
+
## Benchmark workloads
34
+
35
+
Workloads are used to emulate various usage scenarios, for example by varying the read/write proportions, changing the distributions of keys and such. While we intend to add more workloads down the line, the default workload is currently `uniform_std_v1`.
36
+
37
+
### `uniform_std_v1`
38
+
39
+
This workload executes an uniform proportion of operations (hence called "uniform") for unique rows. It does the following:
32
40
33
-
## Running the benchmark
41
+
- Creates a space `db`
42
+
- Creates a model `db.db` with the following definition: `create model db.db(k: binary, v: uint64)`
43
+
- Now:
44
+
- 1,000,000 unique rows are inserted using `INSERT` (in parallel)
45
+
- 1,000,000 of the unique rows that were created in the previous step are modified using `UPDATE`
46
+
- 1,000,000 of the unique rows that were created and modified earlier are fetched using a `SELECT`
47
+
- 1,000,000 of the unique rows that were created are individually removed using `DELETE`
48
+
- Hence, a total of 4,000,000 queries are run
34
49
35
-
Now that you know how the benchmark engine works, go ahead and benchmark for yourself.
50
+
How to run:
36
51
37
-
1. Run:
38
-
```sh
39
-
sky-bench --password <root_password>
40
-
```
41
-
**Note**: You can ignore the `--password` argument if you have already set it using the `SKYDB_PASSWORD` environment variable
42
-
2. The benchmark engine will run the full workload (as described earlier)
43
-
3. Witness 4,000,000 queries being executed in real-time. Good luck and enjoy the results!
52
+
```sh
53
+
sky-bench --workload 'uniform_std_v1'
54
+
```
44
55
45
56
:::tip
46
-
You can tune the number of threads, connections, rows created and such to your liking to simulate the environment that you think matches your production setting.
57
+
Now go ahead and run your own benchmarks to see the performance of Skytable for yourself. We know you'll love it 🚀
0 commit comments