Skip to content

Commit aa696dd

Browse files
committed
Bump Spark version to 3.2
1 parent 5474289 commit aa696dd

File tree

3 files changed

+10
-10
lines changed

3 files changed

+10
-10
lines changed

.circleci/config.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ jobs:
2929
test:
3030
executor: my-executor
3131
environment:
32-
PLATFORM_VERSION: 2.12_spark3.1
32+
PLATFORM_VERSION: 2.12_spark3.2
3333
DATAGEN_VERSION: 0.5.0-SNAPSHOT
3434
steps:
3535
- checkout

README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -48,20 +48,20 @@ E.g. with [pyenv](https://github.com/pyenv/pyenv) and [pyenv-virtualenv](https:/
4848
pyenv install 3.7.7
4949
pyenv virtualenv 3.7.7 ldbc_datagen_tools
5050
pyenv local ldbc_datagen_tools
51-
pip install -U pip
51+
pip install -U pip
5252
pip install ./tools
5353
```
5454
### Running locally
5555

5656
The `tools/run.py` is intended for **local runs**. To use it, download and extract Spark as follows.
5757

58-
#### Spark 3.1.x
58+
#### Spark 3.2.x
5959

60-
Spark 3.1.x is the recommended runtime to use. The rest of the instructions are provided assuming Spark 3.1.x.
60+
Spark 3.2.x is the recommended runtime to use. The rest of the instructions are provided assuming Spark 3.2.x.
6161

6262
```bash
63-
curl https://downloads.apache.org/spark/spark-3.1.2/spark-3.1.2-bin-hadoop3.2.tgz | sudo tar -xz -C /opt/
64-
export SPARK_HOME="/opt/spark-3.1.2-bin-hadoop3.2"
63+
curl https://downloads.apache.org/spark/spark-3.2.0/spark-3.2.0-bin-hadoop3.2.tgz | sudo tar -xz -C /opt/
64+
export SPARK_HOME="/opt/spark-3.2.0-bin-hadoop3.2"
6565
export PATH="$SPARK_HOME/bin":"$PATH"
6666
```
6767

@@ -76,7 +76,7 @@ tools/build.sh
7676
Run the script with:
7777

7878
```bash
79-
export PLATFORM_VERSION=2.12_spark3.1
79+
export PLATFORM_VERSION=2.12_spark3.2
8080
export DATAGEN_VERSION=0.5.0-SNAPSHOT
8181
tools/run.py ./target/ldbc_snb_datagen_${PLATFORM_VERSION}-${DATAGEN_VERSION}.jar <runtime configuration arguments> -- <generator configuration arguments>
8282
```

pom.xml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,8 @@
1616
<maven.compiler.target>${java.version}</maven.compiler.target>
1717
<scala.version>2.12.15</scala.version>
1818
<scala.compat.version>2.12</scala.compat.version>
19-
<spark.version>3.1.2</spark.version>
20-
<spark.compat.version>3.1</spark.compat.version>
19+
<spark.version>3.2.0</spark.version>
20+
<spark.compat.version>3.2</spark.compat.version>
2121
<spec2.version>4.2.0</spec2.version>
2222
</properties>
2323

@@ -294,7 +294,7 @@
294294

295295
<profiles>
296296
<profile>
297-
<id>spark3.1</id>
297+
<id>spark3.2</id>
298298
<properties>
299299
<!-- This is the default profile. -->
300300
</properties>

0 commit comments

Comments
 (0)