Skip to content

Commit 6ac18cc

Browse files
authored
Merge pull request #163 from datastax/issue/CDM-71
create simplified version of cdm.properties
2 parents 9924a8e + 6292942 commit 6ac18cc

File tree

3 files changed

+371
-246
lines changed

3 files changed

+371
-246
lines changed

README.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,9 @@ tar -xvzf spark-3.3.1-bin-hadoop3.tgz
2626

2727
> :warning: Note that Version 4 of the tool is not backward-compatible with .properties files created in previous versions, and that package names have changed.
2828
29-
1. `cdm.properties` file needs to be configured as applicable for the environment. Parameter descriptions and defaults are described in the file.
30-
> A sample properties file configuration can be [found here](./src/resources/cdm.properties)
29+
1. `cdm.properties` file needs to be configured as applicable for the environment. Parameter descriptions and defaults are described in the file. The file can have any name, it does not need to be `cdm.properties`.
30+
> * A simplified sample properties file configuration can be found here as [cdm.properties](./src/resources/cdm.properties)
31+
> * A complete sample properties file configuration can be found here as [cdm-detailed.properties](./src/resources/cdm-detailed.properties)
3132
2. Place the properties file where it can be accessed while running the job via spark-submit.
3233
3. Run the below job using `spark-submit` command as shown below:
3334

@@ -119,12 +120,12 @@ When running in above mode the tool assumes a `partitions.csv` file to be presen
119120
# Perform large-field Guardrail violation checks
120121
- The tool can be used to identify large fields from a table that may break you cluster guardrails (e.g. AstraDB has a 10MB limit for a single large field) `--class datastax.astra.migrate.Guardrail` as shown below
121122
```
122-
./spark-submit --properties-file cdmGuardrail.properties /
123+
./spark-submit --properties-file cdm.properties /
123124
--conf spark.origin.keyspaceTable="<keyspace-name>.<table-name>" /
125+
--conf spark.cdm.feature.guardrail.colSizeInKB=10000 /
124126
--master "local[*]" /
125127
--class datastax.cdm.job.GuardrailCheck cassandra-data-migrator-4.x.x.jar &> logfile_name_$(date +%Y%m%d_%H_%M).txt
126128
```
127-
> A sample Guardrail properties file can be [found here](./src/resources/cdmGuardrail.properties)
128129

129130
# Features
130131
- Auto-detects table schema (column names, types, keys, collections, UDTs, etc.)

0 commit comments

Comments
 (0)