@@ -25,7 +25,7 @@ You can build the JAR with both Maven and SBT.
25
25
* To assemble the JAR file with Maven, run:
26
26
27
27
``` bash
28
- tools/build.sh
28
+ ./ tools/build.sh
29
29
```
30
30
31
31
* For faster builds during development, consider using SBT. To assemble the JAR file with SBT, run:
@@ -51,7 +51,7 @@ pip install ./tools
51
51
` ` `
52
52
# ## Running locally
53
53
54
- The ` tools/run.py` is intended for ** local runs** . To use it, download and extract Spark as follows.
54
+ The ` ./ tools/run.py` script is intended for ** local runs** . To use it, download and extract Spark as follows.
55
55
56
56
# ### Spark 3.1.x
57
57
@@ -75,26 +75,20 @@ export PATH="$SPARK_HOME/bin":"$PATH"
75
75
76
76
Both Java 8 and Java 11 are supported.
77
77
78
- To build, run
79
-
80
- ` ` ` bash
81
- tools/build.sh
82
- ` ` `
83
-
84
- Run the script with:
78
+ Once you have Spark in place and built the JAR file, run the generator as follows:
85
79
86
80
` ` ` bash
87
81
export PLATFORM_VERSION=2.12_spark3.1
88
82
export DATAGEN_VERSION=0.5.0-SNAPSHOT
89
- tools/run.py ./target/ldbc_snb_datagen_${PLATFORM_VERSION} -${DATAGEN_VERSION} .jar < runtime configuration arguments> -- < generator configuration arguments>
83
+ ./ tools/run.py ./target/ldbc_snb_datagen_${PLATFORM_VERSION} -${DATAGEN_VERSION} .jar < runtime configuration arguments> -- < generator configuration arguments>
90
84
` ` `
91
85
92
86
# ### Runtime configuration arguments
93
87
94
88
The runtime configuration arguments determine the amount of memory, number of threads, degree of parallelism. For a list of arguments, see:
95
89
96
90
` ` ` bash
97
- tools/run.py --help
91
+ ./ tools/run.py --help
98
92
` ` `
99
93
100
94
To generate a single ` part-* .csv` file, reduce the parallelism (number of Spark partitions) to 1.
@@ -158,18 +152,18 @@ It is also possible to pass a parameter file:
158
152
The Docker image can be built with the provided Dockerfile. To build, execute the following command from the repository directory:
159
153
160
154
` ` ` bash
161
- tools/docker-build.sh
155
+ ./ tools/docker-build.sh
162
156
` ` `
163
157
164
- See [Build the JAR](# build-the-jar) to build the library. Then, run the following:
158
+ See [Build the JAR](# build-the-jar) to build the library (e.g. by invoking `./tools/build.sh`) . Then, run the following:
165
159
166
160
` ` ` bash
167
- tools/docker-run.sh
161
+ ./ tools/docker-run.sh
168
162
` ` `
169
163
170
164
# ## Elastic MapReduce
171
165
172
- We provide scripts to run Datagen on AWS EMR. See the README in the [` tools/emr` ](tools/emr) directory for details.
166
+ We provide scripts to run Datagen on AWS EMR. See the README in the [` ./ tools/emr` ](tools/emr) directory for details.
173
167
174
168
# # Larger scale factors
175
169
0 commit comments