Skip to content

Commit 7a3a2ec

Browse files
witold-swierzywitold-swierzy
authored
Create README.md
1 parent 33557aa commit 7a3a2ec

File tree

1 file changed

+149
-0
lines changed
  • data-platform/autonomous-database/autonomous-json/MongoDBReplay

1 file changed

+149
-0
lines changed

‎data-platform/autonomous-database/autonomous-json/MongoDBReplay/README.md

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
## MongoDB Replay
2+
3+
Purpose of this tool is to provide support with assessment of a MongoDB application with Oracle MongoDB API.
4+
5+
The repository contains source code of two artifacts
6+
1. MDBExtractor.jar
7+
- this is the tool, which extracts stream of commands from a log file generated by a MongoDB instance.
8+
9+
2. MDBApplier.jar
10+
- this is the tool, which executes commands extracted during the extraction process against an Oracle MongoDB API instance
11+
12+
## License
13+
14+
Copyright (c) 2025 Oracle and/or its affiliates.
15+
16+
Licensed under the Universal Permissive License (UPL), Version 1.0.
17+
18+
See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details.
19+
20+
## WARNING
21+
#### This tool has been developed for test purposes only!
22+
#### Do not use this tool against production databases! It may drive to serious damages of a database or complete data loss! Oracle does not guarantee and does not take any responsibility of possible problems/issues/damages/data corruptions caused by using this tool.
23+
24+
25+
## Usage
26+
1. Generic requirements
27+
- MR_CONFIG_DIR variable has to be set to a directory, where configuration and log files are/will be stored
28+
- JDK 21 (it can be Oracle or OpenJDK)
29+
- Oracle JDK can be downloaded from the following link: https://www.oracle.com/java/technologies/downloads/#java21
30+
- OpenJDK is usually available in a Linux packates repository, but also can be downloaded and installed independly from the following link: https://openjdk.org/
31+
32+
1.MDBExtractor
33+
requirements:
34+
- Source MongoDB instance up and running
35+
- Profiling at the MongoDB instance
36+
- to extract data from traditional Mongod log file, there is need to enable logging so called slow operations at the Mongod instance level. It can be done by executing the following command:
37+
38+
db.setProfilingLevel(0, -1)
39+
- to extract data from db.system.profile collection or its dump there is need to enable traditional profiling, by executing the following command:
40+
41+
db.setProfilingLevel(2)
42+
43+
- The following configuration file $MR_CONFIG_DIR/MDBExtractConfig.json needs to be existing with the following parameters
44+
45+
OUTPUT_DIR : optional; points to directory where output files will be created;
46+
if this parameter is not set, then MDBExtractor will print out results into standard output
47+
48+
INPUT_FILE : optional; points to an input log file generated by a MongoDB Instance with profiling enabled
49+
if this parameter is not set, then MDBExtractor will read data from standard input
50+
51+
INPUT_FILE_FORMAT : optional; provides information about from which file the data will be read. It accepts two values
52+
MONGO_LOG : in that case Extractor will be reading data from traditional Mongod log file. Profiling needs to be set to (0,-1)
53+
MONGO_PROFILE : in that case Extractor will be reading data from dump of db.system.profile collection. Profiling needs to be set to (2)
54+
55+
INPUT_CONNECT_STRING : optional; if set then it points to a MongoDB instance, from which input data will be read directly. Profiling needs to be set to (2). When this parameter is set, then INPUT_FILE and INPUT_FILE_FORMAT cannot be set.
56+
57+
COMMANDS_LOGGING : optional; default value "true", if set to "true" then every command found by the extractor will be
58+
additional logged with console.log(...). Useful, when output will be processed by mongosh tool, which
59+
has limited capabilities related to error reporting. When set to "false" only commands will be reported.
60+
61+
INCLUDE_COMMANDS : optional; JSON array allowing for providing list of commands, which will be extracted;
62+
all commands not listed here will not be printed out into output; this parameter CANNOT be used
63+
simultanously with EXCLUDE_COMMANDS
64+
65+
EXCLUDE_COMMANDS : optional; JSON array allowing for providing list of commands, which will be ignored during extraction process;
66+
all commands listed here will not be printed out into output; this parameter CANNOT be used
67+
simultanously with INCLUDE_COMMANDS
68+
if none of above parameters is set, then all commands will be traced
69+
70+
INCLUDE_DATABASES : optional; JSON array allowing for providing list of databases, which will be traced.
71+
all databases not listed here will not be traced.
72+
This parameter cannot be used simultaneously with EXCLUDE_DATABASES
73+
74+
EXCLUDE_DATABASES : optional; JSON array allowing for providing list of databases, which will not be traced.
75+
all databases not listed here will be traced.
76+
This parameter cannot be used simultaneously with INCLUDE_DATABASES
77+
if none of above parameters is set, then all databases will be traced
78+
79+
EXECUTION_PLAN_TRACING : if set to a non-zero value, then output will contain explain() commands instead of runCommand;
80+
useful, when there is need to check/compare execution plans between a source and a target system
81+
82+
OUTPUT_MODE : can be set to JSON, which result in that the output data will be formated into sequence of JSON documents
83+
or to SCRIPT, wich will result in that the output data will be formated into NodeJS script, which can be
84+
consumed by mongosh
85+
LOG_FILE : name of log file used by the tool; optional; if not set then all diangostic messages will be redirected to
86+
standard diagnostic output
87+
88+
LOG_LEVEL : optional; can be set to 0,1 or 2. Default value : 0
89+
0 means that only summary of initialization and processing will be logged
90+
1 means that additionally to summaries also errors will be logged
91+
2 means that additionally to summaries and errors all commands will be logged
92+
93+
Configuration File example:
94+
```
95+
{
96+
"OUTPUT_DIR" : "/opt/mongo_tests/output",
97+
"INPUT_FILE" : "/opt/mongo_tests/mongod01.log",
98+
"COMMANDS_LOGGING" : "false",
99+
"INCLUDE_DATABASES" : ["oradev","test"],
100+
"EXCLUDE_COMMANDS" : ["insert"],
101+
"EXECUTION_PLAN_TRACING" : 0,
102+
"OUTPUT_MODE" : "JSON"
103+
}
104+
examples of using the tool
105+
1. ```
106+
java -jar ./MDBExtract.jar
107+
```
108+
this example reads data from a file pointed by INPUT_FILE parameter and generates output into set of files created in
109+
OUTPUT_DIR (every traced database will use a separate output file)
110+
111+
2. ```
112+
tail -n 10000 -f ./mongod.log|java -jar ./MDBExtract.jar
113+
```
114+
this example assumes, that INPUT_FILE parameter IS NOT SET. Instead of the tool reads data from its standard input
115+
116+
2.MDBApplier
117+
requirements
118+
- target MongoDB instance up and running
119+
- the following configuration file $MR_CONFIG_DIR/MDBApplierConfig.json needs to be existing with the following parameters
120+
INPUT_FILE : optional; points to an input log file generated by MDBExtractor
121+
if this parameter is not set, then MDBApplier will read data from standard input
122+
DB_NAME : optional; allows for providing target database name
123+
CONNECT_STRING : mandatoryl; connect string to a target MongoDB or Oracle API for MongoDB instance
124+
LOG_FILE : name of log file used by the tool; optional; if not set then all diangostic messages will be redirected to
125+
standard diagnostic output
126+
127+
LOG_LEVEL : optional; can be set to 0,1 or 2. Default value : 0
128+
0 means that only summary of initialization and processing will be logged
129+
1 means that additionally to summaries also errors will be logged
130+
2 means that additionally to summaries and errors all commands will be logged
131+
132+
Configuration file example:
133+
```
134+
{
135+
"INPUT_FILE" : "/opt/mongo_tests/output/oradev.json",
136+
"DB_NAME" : "test",
137+
"LOG_FILE" : "/opt/mongo_tests/MDBApply.log",
138+
"CONNECT_STRING" : "mongodb://mongo-oci.acompany.com/test"
139+
}
140+
```
141+
examples of using applier:
142+
1. ```
143+
java -jar ./MDBApplier.jar
144+
```
145+
this example reads data from a file pointed by INPUT_FILE parameter and executes found commands against the database pointed by
146+
CONNECT_STRING and DB_NAME
147+
148+
149+

0 commit comments

Comments
 (0)