File tree Expand file tree Collapse file tree 3 files changed +10
-10
lines changed Expand file tree Collapse file tree 3 files changed +10
-10
lines changed Original file line number Diff line number Diff line change 29
29
test :
30
30
executor : my-executor
31
31
environment :
32
- PLATFORM_VERSION : 2.12_spark3.1
32
+ PLATFORM_VERSION : 2.12_spark3.2
33
33
DATAGEN_VERSION : 0.5.0-SNAPSHOT
34
34
steps :
35
35
- checkout
Original file line number Diff line number Diff line change @@ -48,20 +48,20 @@ E.g. with [pyenv](https://github.com/pyenv/pyenv) and [pyenv-virtualenv](https:/
48
48
pyenv install 3.7.7
49
49
pyenv virtualenv 3.7.7 ldbc_datagen_tools
50
50
pyenv local ldbc_datagen_tools
51
- pip install -U pip
51
+ pip install -U pip
52
52
pip install ./tools
53
53
` ` `
54
54
# ## Running locally
55
55
56
56
The ` tools/run.py` is intended for ** local runs** . To use it, download and extract Spark as follows.
57
57
58
- # ### Spark 3.1 .x
58
+ # ### Spark 3.2 .x
59
59
60
- Spark 3.1 .x is the recommended runtime to use. The rest of the instructions are provided assuming Spark 3.1 .x.
60
+ Spark 3.2 .x is the recommended runtime to use. The rest of the instructions are provided assuming Spark 3.2 .x.
61
61
62
62
` ` ` bash
63
- curl https://downloads.apache.org/spark/spark-3.1.2 /spark-3.1.2 -bin-hadoop3.2.tgz | sudo tar -xz -C /opt/
64
- export SPARK_HOME=" /opt/spark-3.1.2 -bin-hadoop3.2"
63
+ curl https://downloads.apache.org/spark/spark-3.2.0 /spark-3.2.0 -bin-hadoop3.2.tgz | sudo tar -xz -C /opt/
64
+ export SPARK_HOME=" /opt/spark-3.2.0 -bin-hadoop3.2"
65
65
export PATH=" $SPARK_HOME /bin" :" $PATH "
66
66
` ` `
67
67
@@ -76,7 +76,7 @@ tools/build.sh
76
76
Run the script with:
77
77
78
78
` ` ` bash
79
- export PLATFORM_VERSION=2.12_spark3.1
79
+ export PLATFORM_VERSION=2.12_spark3.2
80
80
export DATAGEN_VERSION=0.5.0-SNAPSHOT
81
81
tools/run.py ./target/ldbc_snb_datagen_${PLATFORM_VERSION} -${DATAGEN_VERSION} .jar < runtime configuration arguments> -- < generator configuration arguments>
82
82
` ` `
Original file line number Diff line number Diff line change 16
16
<maven .compiler.target>${java.version} </maven .compiler.target>
17
17
<scala .version>2.12.15</scala .version>
18
18
<scala .compat.version>2.12</scala .compat.version>
19
- <spark .version>3.1.2 </spark .version>
20
- <spark .compat.version>3.1 </spark .compat.version>
19
+ <spark .version>3.2.0 </spark .version>
20
+ <spark .compat.version>3.2 </spark .compat.version>
21
21
<spec2 .version>4.2.0</spec2 .version>
22
22
</properties >
23
23
294
294
295
295
<profiles >
296
296
<profile >
297
- <id >spark3.1 </id >
297
+ <id >spark3.2 </id >
298
298
<properties >
299
299
<!-- This is the default profile. -->
300
300
</properties >
You can’t perform that action at this time.
0 commit comments