Skip to content

Commit cb6033d

Browse files
kielfriedtkielfriedt
committed
Adding mongodb8 details. Modified how it is install and configured
1 parent d30e547 commit cb6033d

File tree

4 files changed

+337
-7
lines changed

4 files changed

+337
-7
lines changed

‎content/learning-paths/servers-and-cloud-computing/mongodb/benchmark_mongodb-7.0.md‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
---
2+
# User change
3+
title: "Benchmark MongoDB 7.0 on Arm with Yahoo Cloud Serving Benchmark (YCSB)"
4+
5+
weight: 4 # (intro is 1), 2 is first, 3 is second, etc.
6+
7+
# Do not modify these elements
8+
layout: "learningpathall"
9+
---
10+
To further measure the performance of MongoDB, you will run the [Yahoo Cloud Serving Benchmark](http://github.com/brianfrankcooper/YCSB).
11+
12+
YCSB is an open sourced project which provides the framework and common set of workloads to evaluate the performance of different "key-value" and "cloud" serving stores. Here are the steps to run YCSB to evaluate the performance of MongoDB running on 64-bit Arm machine.
13+
14+
## Additional software packages
15+
16+
To run YCSB, additional software packages are required, [Apache Maven](https://maven.apache.org/), and [Python](https://www.python.org) 2.7.
17+
18+
Installing Apache Maven:
19+
20+
```bash
21+
cd ~
22+
wget https://archive.apache.org/dist/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz
23+
sudo tar xzf apache-maven-*-bin.tar.gz -C /usr/local
24+
cd /usr/local
25+
sudo ln -s apache-maven-* maven
26+
cd ~/
27+
export M2_HOME=/usr/local/maven
28+
export PATH="$M2_HOME/bin:$PATH"
29+
```
30+
31+
Installing Python 2.7:
32+
33+
{{< tabpane code=true >}}
34+
{{< tab header="Ubuntu" >}}
35+
sudo apt-get update
36+
sudo apt install python -y
37+
{{< /tab >}}
38+
{{< tab header="RHE/Amazon" >}}
39+
sudo yum check-update
40+
sudo yum install python2
41+
{{< /tab >}}
42+
{{< /tabpane >}}
43+
{{% notice Python Note%}}
44+
45+
For Ubuntu 22.04 the `python` package may not be found. You can install Python 2.7 using:
46+
```console
47+
sudo apt install python2 -y
48+
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2 1
49+
```
50+
51+
For Red Hat you can configure `python2` to be the default `python` using:
52+
```console
53+
sudo alternatives --set python /usr/bin/python2
54+
```
55+
{{% /notice %}}
56+
57+
## Setup YCSB
58+
59+
Download the latest released YCSB zip file and uncompress it.
60+
61+
```bash { pre_cmd="sudo apt install -y python" }
62+
mkdir ycsb && cd ycsb
63+
curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.17.0/ycsb-0.17.0.tar.gz
64+
tar xfvz ycsb-0.17.0.tar.gz
65+
```
66+
Now `cd` into project folder and run the executable to print a description of how to use the benchmark.
67+
68+
```bash { env="M2_HOME=/usr/local/maven,PATH=/usr/local/maven/bin:$PATH",cwd="./ycsb",ret_code="2" }
69+
cd ycsb-0.17.0
70+
./bin/ycsb
71+
```
72+
## Load/Insert Test on MongoDB
73+
74+
To load and test the performance of loading data(INSERT) into default database `ycsb` at `localhost:27017` where MongoDB is running using the synchronous driver run the following command:
75+
76+
```console
77+
./bin/ycsb load mongodb -s -P workloads/workloada -p mongodb.url=mongodb://localhost:27017/ycsb?w=0 -threads 10
78+
```
79+
The "-P" parameter is used to load property files. In this example, you used it load the workloada parameter file which sets the recordcount to 1000 in addition to other parameters. The "-threads" parameter indicates the number of threads and is set to 1 by default.
80+
81+
## Update/Read/Read Modify Write Test on MongoDB
82+
83+
To test the performance of executing a workload which includes running UPDATE, Read Modify Write(RMW) and/or READ operations on the data using 10 threads for example, use the following command:
84+
85+
```console
86+
./bin/ycsb load mongodb -s -P workloads/workloada -p mongodb.url=mongodb://localhost:27017/ycsb?w=0
87+
```
88+
89+
The workloads/workloada file in this example sets the following values `readproportion=0.5` and `updateproportion=0.5` which means there is an even split between the number of READ and UPDATE operations performed. You can change the type of operations and the splits by providing your own workload parameter file.
90+
91+
For more detailed information on all the parameters for running a workload refer to [this section](https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload).
92+
93+
## View the results
94+
95+
At the end of each test, statistics are printed to the console. Shown below is the output from the end of Load/Insert test
96+
97+
```output
98+
2022-07-06 15:50:18:917 1 sec: 1000 operations; 542.01 current ops/sec; [CLEANUP: Count=10, Max=12951, Min=0, Avg=1295.2, 90=4, 99=12951, 99.9=12951, 99.99=12951] [INSERT: Count=1000, Max=134655, Min=561, Avg=8506.37, 90=10287, 99=39903, 99.9=134015, 99.99=134655]
99+
[OVERALL], RunTime(ms), 1849
100+
[OVERALL], Throughput(ops/sec), 540.8328826392644
101+
[TOTAL_GCS_Copy], Count, 5
102+
[TOTAL_GC_TIME_Copy], Time(ms), 23
103+
[TOTAL_GC_TIME_%_Copy], Time(%), 1.2439156300703083
104+
[TOTAL_GCS_MarkSweepCompact], Count, 0
105+
[TOTAL_GC_TIME_MarkSweepCompact], Time(ms), 0
106+
[TOTAL_GC_TIME_%_MarkSweepCompact], Time(%), 0.0
107+
[TOTAL_GCs], Count, 5
108+
[TOTAL_GC_TIME], Time(ms), 23
109+
[TOTAL_GC_TIME_%], Time(%), 1.2439156300703083
110+
[CLEANUP], Operations, 10
111+
[CLEANUP], AverageLatency(us), 1295.2
112+
[CLEANUP], MinLatency(us), 0
113+
[CLEANUP], MaxLatency(us), 12951
114+
[CLEANUP], 95thPercentileLatency(us), 12951
115+
[CLEANUP], 99thPercentileLatency(us), 12951
116+
[INSERT], Operations, 1000
117+
[INSERT], AverageLatency(us), 8506.367
118+
[INSERT], MinLatency(us), 561
119+
[INSERT], MaxLatency(us), 134655
120+
[INSERT], 95thPercentileLatency(us), 11871
121+
[INSERT], 99thPercentileLatency(us), 39903
122+
[INSERT], Return=OK, 1000
123+
...
124+
```
125+
## Other tests
126+
127+
For instructions on running any other tests or more details on the metrics reported, refer to the [GitHub project for the YCSB](https://github.com/brianfrankcooper/YCSB/wiki/).
128+

‎content/learning-paths/servers-and-cloud-computing/mongodb/benchmark_mongodb-8.0.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
---
2+
# User change
3+
title: "Benchmark MongoDB 8.0 on Arm with Yahoo Cloud Serving Benchmark (YCSB)"
4+
5+
weight: 4 # (intro is 1), 2 is first, 3 is second, etc.
6+
7+
# Do not modify these elements
8+
layout: "learningpathall"
9+
---
10+
To further measure the performance of MongoDB, you will run the [Yahoo Cloud Serving Benchmark](http://github.com/brianfrankcooper/YCSB).
11+
12+
YCSB is an open sourced project which provides the framework and common set of workloads to evaluate the performance of different "key-value" and "cloud" serving stores. Here are the steps to run YCSB to evaluate the performance of MongoDB running on 64-bit Arm machine.
13+
14+
## Additional software packages
15+
16+
To run YCSB, additional software packages are required: default-jdk, default-jre, maven, make and Python.
17+
18+
19+
Install all other packages:
20+
21+
{{< tabpane code=true >}}
22+
{{< tab header="Ubuntu" >}}
23+
sudo apt-get update
24+
sudo apt install -y default-jre default-jdk maven make gcc
25+
{{< /tab >}}
26+
{{< /tabpane >}}
27+
{{% notice Python Note%}}
28+
29+
For Ubuntu 22.04 and 24.04 the `python` package may not be found. You can install Python 2.7 using:
30+
```console
31+
wget https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tgz
32+
cd Python-2.7.18
33+
sudo ./configure --enable-optimizations
34+
make altinstall
35+
ln -s /usr/local/bin/python2.7 /usr/bin/python
36+
```
37+
{{% /notice %}}
38+
39+
## Setup YCSB
40+
41+
Download the latest released YCSB zip file and uncompress it.
42+
43+
```bash
44+
mkdir ycsb && cd ycsb
45+
curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.17.0/ycsb-0.17.0.tar.gz
46+
tar xfvz ycsb-0.17.0.tar.gz
47+
48+
```
49+
Now `cd` into project folder and run the executable to print a description of how to use the benchmark.
50+
51+
```bash
52+
cd ycsb-0.17.0
53+
./bin/ycsb
54+
```
55+
56+
## Most Common MongoDB Test Setup
57+
58+
The recommended test setup is a relica set. Which contains three nodes each of equal size. A primary will be the node you send the YCSB traffic to.
59+
60+
## Recommended Tests on MongoDB
61+
62+
The most common real world test to run is a 95/5 test, 95% read and 5% update. 100/0 and 90/10 are also popular. Run the following commands for about 5 mins before collecting data.
63+
64+
Load the dataset
65+
```console
66+
./bin/ycsb load mongodb -s -P workloads/workloadb -p mongodb.url=mongodb://localhost:27017 -p compressibility=2 -p fieldlengthdistribution=zipfian -p minfieldlength=50 -threads 64 -p recordcount=20000000
67+
```
68+
69+
95/5
70+
```console
71+
./bin/ycsb run mongodb -s -P workloads/workloadb -p mongodb.url=mongodb://localhost:27017 -p minfieldlength=50 -p compressibility=2 -p maxexecutiontime=120 -threads 64 -p operationcount=40000000 -p recordcount=20000000 -p requestdistribution=zipfian -p readproportion=0.95 -p updateproportion=0.05
72+
73+
```
74+
75+
100/0
76+
```console
77+
./bin/ycsb run mongodb -s -P workloads/workloadc -p mongodb.url=mongodb://Localhost:27017 -p minfieldlength=50 -p compressibility=2 -p maxexecutiontime=120 -threads 64 -p operationcount=40000000 -p recordcount=20000000 -p requestdistribution=zipfian -p readproportion=1.0 -p updateproportion=0.0
78+
79+
```
80+
81+
90/10
82+
```console
83+
./bin/ycsb run mongodb -s -P workloads/workloadb -p mongodb.url=mongodb://localhost:27017 -p minfieldlength=50 -p compressibility=2 -p maxexecutiontime=120 -threads 64 -p operationcount=40000000 -p recordcount=20000000 -p requestdistribution=zipfian -p readproportion=0.90 -p updateproportion=0.10
84+
85+
```
86+
87+
For more detailed information on all the parameters for running a workload refer to [this section](https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload).
88+
89+
## View the results
90+
91+
At the end of each test, statistics are printed to the console. Shown below is the output from the end of Load/Insert test
92+
93+
```output
94+
2022-07-06 15:50:18:917 1 sec: 1000 operations; 542.01 current ops/sec; [CLEANUP: Count=10, Max=12951, Min=0, Avg=1295.2, 90=4, 99=12951, 99.9=12951, 99.99=12951] [INSERT: Count=1000, Max=134655, Min=561, Avg=8506.37, 90=10287, 99=39903, 99.9=134015, 99.99=134655]
95+
[OVERALL], RunTime(ms), 1849
96+
[OVERALL], Throughput(ops/sec), 540.8328826392644
97+
[TOTAL_GCS_Copy], Count, 5
98+
[TOTAL_GC_TIME_Copy], Time(ms), 23
99+
[TOTAL_GC_TIME_%_Copy], Time(%), 1.2439156300703083
100+
[TOTAL_GCS_MarkSweepCompact], Count, 0
101+
[TOTAL_GC_TIME_MarkSweepCompact], Time(ms), 0
102+
[TOTAL_GC_TIME_%_MarkSweepCompact], Time(%), 0.0
103+
[TOTAL_GCs], Count, 5
104+
[TOTAL_GC_TIME], Time(ms), 23
105+
[TOTAL_GC_TIME_%], Time(%), 1.2439156300703083
106+
[CLEANUP], Operations, 10
107+
[CLEANUP], AverageLatency(us), 1295.2
108+
[CLEANUP], MinLatency(us), 0
109+
[CLEANUP], MaxLatency(us), 12951
110+
[CLEANUP], 95thPercentileLatency(us), 12951
111+
[CLEANUP], 99thPercentileLatency(us), 12951
112+
[INSERT], Operations, 1000
113+
[INSERT], AverageLatency(us), 8506.367
114+
[INSERT], MinLatency(us), 561
115+
[INSERT], MaxLatency(us), 134655
116+
[INSERT], 95thPercentileLatency(us), 11871
117+
[INSERT], 99thPercentileLatency(us), 39903
118+
[INSERT], Return=OK, 1000
119+
...
120+
```
121+
## Other tests
122+
123+
For instructions on running any other tests or more details on the metrics reported, refer to the [GitHub project for the YCSB](https://github.com/brianfrankcooper/YCSB/wiki/).
124+

‎content/learning-paths/servers-and-cloud-computing/mongodb/mongodb_configuration.md‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
---
2+
# User change
3+
title: "MongoDB test configuration and setup"
4+
5+
weight: 3 # 1 is first, 2 is second, etc.
6+
7+
# Do not modify these elements
8+
layout: "learningpathall"
9+
---
10+
The most popular test setup for read world testing is a replica set. A replica set of three equal sized nodes is created and initiated.
11+
12+
## What is a replica Set?
13+
A replica set is a group of instances that maintain the same data set. A replica set contains many nodes and in our test three nodes. Out of the three nodes, one and only one member is the primary node, while the other nodes are secondary nodes.
14+
15+
## What node size should I use?
16+
The most common size for testing MongoDB is a 8vCPU instance. You are welcome to test with any sized machine. But If you are looking for ideal testing conditionals 8 is more than enough. 32 Gb of ram is recommended for testing.
17+
18+
## How should I run this test?
19+
It is recommended to avoid disk and keep the complete date set within memory. The recommended configuration is below and will be explains in detail.
20+
21+
## Mongod.conf
22+
23+
```console
24+
# Configuration Options: https://docs.mongodb.org/manual/reference/configuration-options/
25+
# Log Messages/Components: https://docs.mongodb.com/manual/reference/log-messages/index.html
26+
27+
systemLog:
28+
destination: file
29+
logAppend: true
30+
path: /var/log/mongodb/mongodb.log
31+
32+
storage:
33+
dbPath: /mnt/mongodb # Mounting point selected
34+
engine: wiredTiger
35+
wiredTiger:
36+
engineConfig:
37+
configString: "cache_size=16484MB" # 50% of your ram is recommened. Adding more helps depending on dataset.
38+
39+
replication:
40+
replSetName: "rs0" # Name of your replicaset
41+
oplogSizeMB: 5000
42+
43+
# network interfaces
44+
net:
45+
port: 27017
46+
bindIp: 0.0.0.0
47+
maxIncomingConnections: 16000
48+
setParameter:
49+
diagnosticDataCollectionDirectorySizeMB: 400
50+
honorSystemUmask: false
51+
lockCodeSegmentsInMemory: true
52+
reportOpWriteConcernCountersInServerStatus: true
53+
suppressNoTLSPeerCertificateWarning: true
54+
tlsWithholdClientCertificate: true
55+
```
56+
**systemLog:** Contains locations and details of where logging should be contained.
57+
- **path:** Location for logging
58+
59+
**storage:** Its recommended to run test within memory to get achieve the best performance. This contains details on the engine used and location of storage.
60+
- **engine:** Wiredtiger is used in this case. Using a disk will add latency.
61+
- **cache_size:** The minimum if using the recommend instance size is 50% of 32(16gb). But in testing using 18gb produced better results.
62+
63+
**replication:** This is used for replica set setup.
64+
- **replSetName:** This is the name of the replica set.
65+
- **oplogSizeMB:** 5% of the disk size is recommended.
66+
67+
**net:** Contains details of networking on the node.
68+
- **port:** 27017 is the port used for replica sets
69+
- **maxIncomingConnections:** The maximum number of incoming connections supported by MongoDB
70+
71+
**setParameter:** Addtional options
72+
- **diagnosticDataCollectionDirectorySizeMB:** 400 is based on the docs.
73+
- **honorSystemUmask:** Sets read and write permissions only to the owner of new files
74+
- **lockCodeSegmentsInMemory:** Locks code into memory and prevents it from being swapped.
75+
- **suppressNoTLSPeerCertificateWarning:** allows clients to connect without a certificate. (Only for testing purposes)
76+
- **tlsWithholdClientCertificate:** Will not send the certification during communication. (Only for testing purposes)
77+
78+
If you would like to use encryption you will need to add the security and keyFile to your configuration. As well as change some of the parameters in the mongod.conf.

‎content/learning-paths/servers-and-cloud-computing/mongodb/run_mongodb.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -9,17 +9,17 @@ layout: "learningpathall"
99
---
1010
[MongoDB](https://www.mongodb.com/) is a popular free-to-use document database.
1111

12-
The latest released version of MongoDB Community Edition (7.0) is supported on the following Linux distributions:
12+
The latest released version of MongoDB Community Edition (8.0) is supported on the following Linux distributions:
1313

14-
* Ubuntu Versions - 20.04, 22.04
14+
* Ubuntu Versions - 20.04, 22.04, 24.04
1515
* RHEL/CentOS 8, 9
16-
* Amazon Linux 2, 2023
16+
* Amazon Linux 2023
1717

1818
Refer to this [page](https://www.mongodb.com/docs/manual/administration/production-notes/#platform-support-matrix) for the complete platform support matrix.
1919

2020
## Install and Run MongoDB on Ubuntu
2121

22-
Launch an [Arm based instance](/learning-paths/servers-and-cloud-computing/csp/) running either Ubuntu versions 20.04 or 22.04.
22+
Launch an [Arm based instance](/learning-paths/servers-and-cloud-computing/csp/) running either Ubuntu versions 20.04, 22.04 or 24.04.
2323

2424
Follow [Install MongoDB Community Edition on Ubuntu](https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-ubuntu/) to install and run MongoDB on your instance.
2525

@@ -29,8 +29,8 @@ Launch an [Arm based instance](/learning-paths/servers-and-cloud-computing/csp/)
2929

3030
Follow [Install MongoDB Community Edition on Red Hat or CentOS](https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-red-hat/) to install and run MongoDB on your instance.
3131

32-
## Install and Run MongoDB on Amazon Linux 2
32+
## Install and Run MongoDB on Amazon Linux 2023
3333

34-
Launch an [Arm based instance](/learning-paths/servers-and-cloud-computing/csp/) running Amazon Linux 2 or Amazon Linux 2023.
34+
Launch an [Arm based instance](/learning-paths/servers-and-cloud-computing/csp/) running Amazon Linux 2023.
3535

36-
Follow [Install MongoDB Community Edition on Amazon Linux](https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-amazon/) to install and run MongoDB on your instance.
36+
Follow [Install MongoDB Community Edition on Amazon Linux 2023](https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-amazon/) to install and run MongoDB on your instance.

0 commit comments

Comments
 (0)