Skip to content

Commit bedb3d7

Browse files
Merge pull request #1471 from jasonrandrews/review2
MongoDB udpates
2 parents f2559ca + 5734a58 commit bedb3d7

File tree

5 files changed

+74
-189
lines changed

5 files changed

+74
-189
lines changed

content/learning-paths/servers-and-cloud-computing/mongodb/_index.md

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,5 @@
11
---
2-
title: Test the performance of MongoDB on Arm servers
3-
4-
draft: true
5-
cascade:
6-
draft: true
2+
title: Analyze the performance of MongoDB on Arm servers
73

84
author_primary: Pareena Verma
95

@@ -15,7 +11,7 @@ learning_objectives:
1511
- Measure and compare the performance of MongoDB on Arm versus other architectures with Yahoo Cloud Serving Benchmark (YCSB).
1612

1713
prerequisites:
18-
- An Arm based instance from a cloud service provider. The recommend YCSB configuration requires a cluster of 3 Arm servers.
14+
- An Arm based instance from a cloud service provider.
1915

2016
armips:
2117
- Neoverse

content/learning-paths/servers-and-cloud-computing/mongodb/automate_setup_pulumi.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,6 @@ title: "Automate MongoDB Performance Benchmarking Infrastructure Setup with Pulu
44

55
weight: 8 # (intro is 1), 2 is first, 3 is second, etc.
66

7-
draft: true
8-
97
# Do not modify these elements
108
layout: "learningpathall"
119
---

content/learning-paths/servers-and-cloud-computing/mongodb/benchmark_mongodb-7.0.md

Lines changed: 0 additions & 130 deletions
This file was deleted.

content/learning-paths/servers-and-cloud-computing/mongodb/benchmark_mongodb-8.0.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ To test the performance of executing a workload which includes running UPDATE, R
8080

8181
The workloads/workloada file in this example sets the following values `readproportion=0.5` and `updateproportion=0.5` which means there is an even split between the number of READ and UPDATE operations performed. You can change the type of operations and the splits by providing your own workload parameter file.
8282

83-
For more detailed information on all the parameters for running a workload refer to [this section](https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload).
83+
For more detailed information on all the parameters for running a workload refer to [Running a Workload.](https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload)
8484

8585
## View the results
8686

content/learning-paths/servers-and-cloud-computing/mongodb/replica.md

Lines changed: 71 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22
# User change
33
title: "Run YCSB using a 3 node replica set"
44

5+
draft: true
6+
57
weight: 5 # (intro is 1), 2 is first, 3 is second, etc.
68

79
# Do not modify these elements
@@ -10,19 +12,70 @@ layout: "learningpathall"
1012

1113
The recommended MongoDB YCSB test setup is a relica set containing three nodes of equal size. The primary node is the node you send the YCSB traffic to and the others are secondary nodes.
1214

13-
## What is a replica Set?
15+
## What is a replica set?
1416

1517
A replica set is a group of instances that maintain the same data set. A replica set contains many nodes, but 3 nodes are used for testing.
1618

1719
## What node size should I use?
1820

1921
The most common size for testing MongoDB is an 8 vCPU instance. You can test with any sized machine, but if you are looking for ideal testing conditions 8 vCPUs is enough. Each node should have 32GB of RAM.
2022

21-
## How should I run this test?
22-
2323
You should keep the complete data set in memory. Additional details abut the recommended configuration are provided below.
2424

25-
## Mongod.conf
25+
## Create a replica set
26+
27+
Create a 3 node replica set by starting 3 Arm instances with the specifications above.
28+
29+
Install MongoDB on each node using the previously provided instructions.
30+
31+
Select 1 instance as the primary node and install YCSB on the instance.
32+
33+
## Initialize the replica set
34+
35+
1. Set variables with the IP addresses of each node:
36+
37+
```bash
38+
PRIMARY_NODE_IP="<primary-node-ip>"
39+
SECONDARY_NODE1_IP="<secondary-node1-ip>"
40+
SECONDARY_NODE2_IP="<secondary-node2-ip>"
41+
```
42+
43+
2. Connect to the primary node using the MongoDB shell:
44+
45+
```bash
46+
mongosh --host <primary-node-ip>:27017
47+
```
48+
49+
3. Initialize the replica set with the following command:
50+
51+
```bash
52+
PRIMARY_NODE_IP="<primary-node-ip>"
53+
SECONDARY_NODE1_IP="<secondary-node1-ip>"
54+
SECONDARY_NODE2_IP="<secondary-node2-ip>"
55+
56+
mongosh --host $PRIMARY_NODE_IP:27017 <<EOF
57+
rs.initiate({
58+
_id: "rs0",
59+
members: [
60+
{ _id: 0, host: "$PRIMARY_NODE_IP:27017" },
61+
{ _id: 1, host: "$SECONDARY_NODE1_IP:27017" },
62+
{ _id: 2, host: "$SECONDARY_NODE2_IP:27017" }
63+
]
64+
})
65+
EOF
66+
```
67+
68+
3. Verify the replica set status:
69+
70+
```bash
71+
mongosh --host $PRIMARY_NODE_IP:27017 <<EOF
72+
rs.status()
73+
EOF
74+
```
75+
76+
## Modify the MongoDB configuration
77+
78+
Use a text editor to edit the file `/etc/mongodb.conf` file and replace the contents of the file with the text below.
2679
2780
```console
2881
# Configuration Options: https://docs.mongodb.org/manual/reference/configuration-options/
@@ -57,6 +110,7 @@ setParameter:
57110
suppressNoTLSPeerCertificateWarning: true
58111
tlsWithholdClientCertificate: true
59112
```
113+
60114
**systemLog:** Contains locations and details of where logging should be contained.
61115
- **path:** Location for logging
62116
@@ -79,74 +133,41 @@ setParameter:
79133
- **suppressNoTLSPeerCertificateWarning:** allows clients to connect without a certificate. (Only for testing purposes)
80134
- **tlsWithholdClientCertificate:** Will not send the certification during communication. (Only for testing purposes)
81135
82-
If you would like to use encryption you will need to add the security and keyFile to your configuration. As well as change some of the parameters in the mongod.conf.
136+
If you want to use encryption you will need to add the security and keyFile to your configuration. As well as change some of the parameters in the `mongod.conf` file.
83137
84-
## Most Common MongoDB Test Setup
85-
86-
The recommended test setup is a relica set. Which contains three nodes each of equal size. A primary will be the node you send the YCSB traffic to.
87138
88139
## Recommended Tests on MongoDB
89140
90-
The most common real world test to run is a 95/5 test, 95% read and 5% update. 100/0 and 90/10 are also popular. Run the following commands for about 5 mins before collecting data.
141+
The most common real world test to run is a 95/5 test, 95% read and 5% update. 100/0 and 90/10 are also popular.
142+
143+
Run the following commands for about 5 mins before collecting data.
144+
145+
Load the dataset:
91146
92-
Load the dataset
93147
```console
94148
./bin/ycsb load mongodb -s -P workloads/workloadb -p mongodb.url=mongodb://localhost:27017 -p compressibility=2 -p fieldlengthdistribution=zipfian -p minfieldlength=50 -threads 64 -p recordcount=20000000
95149
```
96150
97-
95/5
151+
Run the 95/5 test:
152+
98153
```console
99154
./bin/ycsb run mongodb -s -P workloads/workloadb -p mongodb.url=mongodb://localhost:27017 -p minfieldlength=50 -p compressibility=2 -p maxexecutiontime=120 -threads 64 -p operationcount=40000000 -p recordcount=20000000 -p requestdistribution=zipfian -p readproportion=0.95 -p updateproportion=0.05
100155
101156
```
102157
103-
100/0
158+
Run the 100/0 test:
159+
104160
```console
105161
./bin/ycsb run mongodb -s -P workloads/workloadc -p mongodb.url=mongodb://Localhost:27017 -p minfieldlength=50 -p compressibility=2 -p maxexecutiontime=120 -threads 64 -p operationcount=40000000 -p recordcount=20000000 -p requestdistribution=zipfian -p readproportion=1.0 -p updateproportion=0.0
106-
107162
```
108163
109-
90/10
164+
Run the 90/10 test:
165+
110166
```console
111167
./bin/ycsb run mongodb -s -P workloads/workloadb -p mongodb.url=mongodb://localhost:27017 -p minfieldlength=50 -p compressibility=2 -p maxexecutiontime=120 -threads 64 -p operationcount=40000000 -p recordcount=20000000 -p requestdistribution=zipfian -p readproportion=0.90 -p updateproportion=0.10
112-
113168
```
114169
115-
For more detailed information on all the parameters for running a workload refer to [this section](https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload).
116-
117-
## View the results
118-
119-
At the end of each test, statistics are printed to the console. Shown below is the output from the end of Load/Insert test
120-
121-
```output
122-
2022-07-06 15:50:18:917 1 sec: 1000 operations; 542.01 current ops/sec; [CLEANUP: Count=10, Max=12951, Min=0, Avg=1295.2, 90=4, 99=12951, 99.9=12951, 99.99=12951] [INSERT: Count=1000, Max=134655, Min=561, Avg=8506.37, 90=10287, 99=39903, 99.9=134015, 99.99=134655]
123-
[OVERALL], RunTime(ms), 1849
124-
[OVERALL], Throughput(ops/sec), 540.8328826392644
125-
[TOTAL_GCS_Copy], Count, 5
126-
[TOTAL_GC_TIME_Copy], Time(ms), 23
127-
[TOTAL_GC_TIME_%_Copy], Time(%), 1.2439156300703083
128-
[TOTAL_GCS_MarkSweepCompact], Count, 0
129-
[TOTAL_GC_TIME_MarkSweepCompact], Time(ms), 0
130-
[TOTAL_GC_TIME_%_MarkSweepCompact], Time(%), 0.0
131-
[TOTAL_GCs], Count, 5
132-
[TOTAL_GC_TIME], Time(ms), 23
133-
[TOTAL_GC_TIME_%], Time(%), 1.2439156300703083
134-
[CLEANUP], Operations, 10
135-
[CLEANUP], AverageLatency(us), 1295.2
136-
[CLEANUP], MinLatency(us), 0
137-
[CLEANUP], MaxLatency(us), 12951
138-
[CLEANUP], 95thPercentileLatency(us), 12951
139-
[CLEANUP], 99thPercentileLatency(us), 12951
140-
[INSERT], Operations, 1000
141-
[INSERT], AverageLatency(us), 8506.367
142-
[INSERT], MinLatency(us), 561
143-
[INSERT], MaxLatency(us), 134655
144-
[INSERT], 95thPercentileLatency(us), 11871
145-
[INSERT], 99thPercentileLatency(us), 39903
146-
[INSERT], Return=OK, 1000
147-
...
148-
```
149170
## Other tests
150171
151-
For instructions on running any other tests or more details on the metrics reported, refer to the [GitHub project for the YCSB](https://github.com/brianfrankcooper/YCSB/wiki/).
172+
For instructions on running any other tests or more details on the metrics reported, refer to the [GitHub project for the YCSB.](https://github.com/brianfrankcooper/YCSB/wiki/).
152173

0 commit comments

Comments
 (0)