You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+15-15Lines changed: 15 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,24 +8,24 @@ The [dataset](https://clickhouse.com/blog/json-bench-clickhouse-vs-mongodb-elast
8
8
This was obtained using Jetstream to collect Bluesky events.
9
9
The dataset contains 1 billion Bluesky events and is currently hosted on a public S3 bucket.
10
10
11
-
We wrote a [detailed blog post](https://clickhouse.com/blog/json-bench-clickhouse-vs-mongodb-elasticsearch-duckdb-postgresql)on JSONBench, explaining how it works and showcasing benchmark results for the first five databases: ClickHouse, MongoDB, Elasticsearch, DuckDB, and PostgreSQL.
11
+
We wrote a [detailed blog post](https://clickhouse.com/blog/json-bench-clickhouse-vs-mongodb-elasticsearch-duckdb-postgresql)about JSONBench, explaining how it works and showcasing benchmark results for five databases: ClickHouse, MongoDB, Elasticsearch, DuckDB, and PostgreSQL.
12
12
13
13
## Principles
14
14
15
15
The [main principles](https://clickhouse.com/blog/json-bench-clickhouse-vs-mongodb-elasticsearch-duckdb-postgresql#benchmark-methodology) of this benchmark are:
16
16
17
17
### Reproducibility
18
18
19
-
You can easily reproduce every test (although for some systems it may take from several hours to days) in a semi-automated way.
19
+
It is easy to reproduce every test in a semi-automated way (although for some systems it may take from several hours to days).
20
20
The test setup is documented and uses inexpensive cloud VMs.
21
-
The test process is documented in the form of a shell script, covering the installation of every system, loading of the data, running the workload, and collecting the result numbers.
21
+
The test process is available in the form of a shell script, covering the installation of each database, loading of the data, running the workload, and collecting the result numbers.
22
22
The dataset is published and made available for download in multiple formats.
23
23
24
24
### Realism
25
25
26
-
[The dataset](https://clickhouse.com/blog/json-bench-clickhouse-vs-mongodb-elasticsearch-duckdb-postgresql#the-json-dataset---a-billion-bluesky-events)is represented by real-world production data.
27
-
The realistic data distributions allow for correctly accounting for compression, indices, codecs, custom data structures, etc., which is not possible with most of the random dataset generators.
28
-
It can test various aspects of hardware as well: some queries require high storage throughput; some queries benefit from a large number of CPU cores, and some benefit from single-core speed; some queries benefit from high main memory bandwidth.
26
+
[The dataset](https://clickhouse.com/blog/json-bench-clickhouse-vs-mongodb-elasticsearch-duckdb-postgresql#the-json-dataset---a-billion-bluesky-events)represents real-world production data.
27
+
The realistic data distribution allows to account appropriately for compression, indices, codecs, custom data structures, etc., something that is not possible with most random data generators.
28
+
JSONBench tests various aspects of the hardware as well: some queries require high storage throughput, some queries benefit from a large number of CPU cores, and some benefit from single-core speed, some queries benefit from high main memory bandwidth.
29
29
30
30
### Fairness
31
31
@@ -42,13 +42,12 @@ The dashboard allows to filter out databases which do not retain the document st
42
42
## Goals
43
43
44
44
The goal is to advance the possibilities of data analytics on semistructured data.
45
-
This benchmark is influenced by **[ClickBench](https://github.com/ClickHouse/ClickBench)** which was published in 2022 and has helped in improving performance, capabilities, and stability of many analytic databases.
46
-
We would like to see comparable influence from **JSONBench**.
45
+
This benchmark is influenced by **[ClickBench](https://github.com/ClickHouse/ClickBench)** which was published in 2022 and has helped in improving performance, capabilities, and stability of many analytics databases.
46
+
We would like to see **JSONBench** having a similar impact on the community.
47
47
48
48
## Limitations
49
49
50
-
The benchmark focuses on data analytics queries rather than search, single-value retrieval, or mutating operations.
51
-
50
+
The benchmark focuses on data analytics queries over JSON documents rather than single-value retrieval or data modification operations.
52
51
The benchmark does not record data loading times.
53
52
While it was one of the initial goals, many systems require a finicky multi-step data preparation process, which makes them difficult to compare.
54
53
@@ -58,11 +57,10 @@ To run the benchmark with 1 billion rows, it is important to provision a machine
58
57
The full compressed dataset takes 125 Gb of disk space, uncompressed it takes up to 425 Gb.
59
58
60
59
For reference, the initial benchmarks have been run on the following machines:
61
-
- AWS EC2 instance: m6i.8xlarge
62
-
- Disk: > 10Tb gp3
60
+
- Hardware: m6i.8xlarge AWS EC2 instance with 10Tb gp3 disks
63
61
- OS: Ubuntu 24.04
64
62
65
-
If you're interested in running the full benchmark, be aware that it will take several hours or days depending on the database.
63
+
If you're interested in running the full benchmark, be aware that it will take several hours or days, depending on the database.
66
64
67
65
## Usage
68
66
@@ -73,7 +71,9 @@ The full dataset contains 1 billion rows, but the benchmark runs for [different
73
71
### Download the data
74
72
75
73
Start by downloading the dataset using the script [`download_data.sh`](./download_data.sh).
76
-
When running the script, you will be prompted the dataset size you want to download, if you just want to test it out, I'd recommend starting with the default 1m rows, if you're interested to reproduce results at scale, go with the full dataset, 1 billion rows.
74
+
When running the script, you will be prompted the dataset size you want to download.
75
+
If you just want to test it out, we recommend starting with the default 1m rows.
76
+
If you are interested in reproducing the results at scale, go with the full dataset (1 billion rows).
77
77
78
78
```
79
79
./download_data.sh
@@ -88,7 +88,7 @@ Enter the number corresponding to your choice:
88
88
89
89
### Run the benchmark
90
90
91
-
Navigate to the folder corresponding to the database you want to run the benchmark for.
91
+
Navigate to folder corresponding to the database you want to run the benchmark for.
92
92
93
93
The script `main.sh` is the script to run each benchmark.
0 commit comments