Skip to content

Commit a24a700

Browse files
committed
Updated README
1 parent f26138d commit a24a700

File tree

2 files changed

+11
-6
lines changed

2 files changed

+11
-6
lines changed

README.md

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@ Spark jobs in this repo can be used for data migration and data validation.
44

55
> :warning: Please note this job has been tested with spark version [2.4.8](https://archive.apache.org/dist/spark/spark-2.4.8/)
66
7+
## Build
8+
1. Clone this repo
9+
2. Move to the repo folder `cd cassandra-data-migrator`
10+
3. Run the build `mvn clean package`
11+
4. The fat jar (`cassandra-data-migrator-2.x.jar`) file should now be present in the `target` folder
12+
713
## Prerequisite
814

915
Install Java8 as spark binaries are compiled with it.
@@ -19,13 +25,12 @@ tar -xvzf <spark downloaded file name>
1925
1. `sparkConf.properties` file needs to be configured as applicable for the environment
2026
> A sample Spark conf file configuration can be [found here](./src/resources/sparkConf.properties)
2127
2. Place the conf file where it can be accessed while running the job via spark-submit.
22-
3. Generate a fat jar (`cassandra-data-migrator-1.x.jar`) using command `mvn clean package`
23-
4. Run the 'Data Migration' job using `spark-submit` command as shown below:
28+
3. Run the 'Data Migration' job using `spark-submit` command as shown below:
2429

2530
```
2631
./spark-submit --properties-file sparkConf.properties /
2732
--master "local[*]" /
28-
--class datastax.astra.migrate.Migrate cassandra-data-migrator-1.x.jar &> logfile_name.txt
33+
--class datastax.astra.migrate.Migrate cassandra-data-migrator-2.x.jar &> logfile_name.txt
2934
```
3035

3136
Note: Above command also generates a log file `logfile_name.txt` to avoid log output on the console.
@@ -38,7 +43,7 @@ Note: Above command also generates a log file `logfile_name.txt` to avoid log ou
3843
```
3944
./spark-submit --properties-file sparkConf.properties /
4045
--master "local[*]" /
41-
--class datastax.astra.migrate.DiffData cassandra-data-migrator-1.x.jar &> logfile_name.txt
46+
--class datastax.astra.migrate.DiffData cassandra-data-migrator-2.x.jar &> logfile_name.txt
4247
```
4348

4449
- Validation job will report differences as “ERRORS” in the log file as shown below
@@ -67,7 +72,7 @@ spark.target.autocorrect.mismatch true|false
6772
```
6873
./spark-submit --properties-file sparkConf.properties /
6974
--master "local[*]" /
70-
--class datastax.astra.migrate.MigratePartitionsFromFile cassandra-data-migrator-1.x.jar &> logfile_name.txt
75+
--class datastax.astra.migrate.MigratePartitionsFromFile cassandra-data-migrator-2.x.jar &> logfile_name.txt
7176
```
7277

7378
When running in above mode the tool assumes a `partitions.csv` file to be present in the current folder in the below format, where each line (`min,max`) represents a partition-range

pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33

44
<groupId>datastax.astra.migrate</groupId>
55
<artifactId>cassandra-data-migrator</artifactId>
6-
<version>2.3</version>
6+
<version>2.4</version>
77
<packaging>jar</packaging>
88

99
<properties>

0 commit comments

Comments
 (0)