Address PR comments; more details on pyspark shell v submitting apps

Signed-off-by: Jason T. Brown <[email protected]>

Author: vpipkt

pyrasterframes/src/main/python/docs/getting-started.pymd

@@ -3,7 +3,7 @@
 There are @ref:[several ways](getting-started.md#other-options) to use RasterFrames, and @ref:[several languages](languages.md) with which you can use it. Let's start with the simplest: the Python shell.  To get started you will need:
 
 1. [Python](https://www.python.org/) installed. Version 3.6 or greater is recommended.
-1. `pip` or `pip3` (recommended) installed. If you are using Python 3, `pip3` may already be installed.
+1. [`pip`](https://pip.pypa.io/en/stable/installing/) installed. If you are using Python 3, `pip` may already be installed.
 1. Java [JDK 8](https://openjdk.java.net/install/index.html) installed on your system and `java` on your system `PATH` or `JAVA_HOME` pointing to a Java installation.
 
 ## pip install pyrasterframes
@@ -30,11 +30,12 @@ df = spark.read.raster('https://modis-pds.s3.amazonaws.com/MCD43A4.006/11/08/201
 
 # Add 3 element-wise, show some rows of the dataframe
 df.select(rf_local_add(df.proj_raster, lit(3))).show(5, False)
-
 ```
 
 This example is extended in the [getting started Jupyter notebook](https://nbviewer.jupyter.org/github/locationtech/rasterframes/blob/develop/rf-notebook/src/main/notebooks/Getting%20Started.ipynb).
 
+## Next Steps
+
 To understand more about how and why RasterFrames represents Earth observation in DataFrames, read about the @ref:[core concepts](concepts.md) and the project @ref:[description](description.md). For more hands-on examples, see the chapters about @ref:[reading](raster-io.md) and @ref:[processing](raster-processing.md) with RasterFrames.
 
 ## Other Options
@@ -60,35 +61,74 @@ See [RasterFrames Notebook README](https://github.com/locationtech/rasterframes/
 
 ### `pyspark` shell or app
 
-To initialize RasterFrames in a `pyspark` shell, prepare to call pyspark with the appropriate `--master` and other `--conf` arguments for your cluster manager and environment. To these you will add the PyRasterFrames assembly JAR and the python source zip. 
+You can use RasterFrames in a `pyspark` shell or when submitting a `pyspark` app via a Python script. To set up the `pyspark` environment, prepare your call with the appropriate `--master` and other `--conf` arguments for your cluster manager and environment. To these you will add the PyRasterFrames assembly JAR and the python source zip. 
 
 You can either [build](https://github.com/locationtech/rasterframes/blob/develop/README.md) the artifacts or download them:
 
- * Assembly JAR: https://repo1.maven.org/maven2/org/locationtech/rasterframes/pyrasterframes_2.11/${VERSION}/pyrasterframes-assembly-${VERSION}.jar
  * Python zip: https://repo1.maven.org/maven2/org/locationtech/rasterframes/pyrasterframes_2.11/${VERSION}/pyrasterframes_2.11-${VERSION}-python.zip
+ * Assembly JAR:
+    * The assembly JAR is embedded in the wheel file publised on pypi. Download the wheel from https://pypi.org/project/pyrasterframes/#files
+    * The wheel file is just a [zip file with .whl extension](https://www.python.org/dev/peps/pep-0427/); you can extract the assembly JAR with a command like this: `unzip -j  $PYRF_WHEEL  $(zipinfo -1 $PYRF_WHEEL | grep jar)`
+
+
+#### Shell 
 
+The `pyspark` shell command will look something like this, replacing the `--jars` argument with the assembly jar and the `--py-files` with the source zip (not the wheel). To submit a script, add a .py file as the final argument
 
 ```bash
    pyspark \
+    --master local[*] \
     --conf spark.serializer=org.apache.spark.serializer.KryoSerializer \
     --conf spark.kryo.registrator=org.locationtech.rasterframes.util.RFKryoRegistrator \
     --conf spark.kryoserializer.buffer.max=500m \
     --jars pyrasterframes/target/scala-2.11/pyrasterframes-assembly-${VERSION}.jar \
     --py-files pyrasterframes/target/scala-2.11/pyrasterframes-python-${VERSION}.zip
 ```
 
-Then in the pyspark shell, import the module and call `withRasterFrames` on the SparkSession.
+Then in the `pyspark` shell, import the module and call `withRasterFrames` on the SparkSession.
 
 ```python, evaluate=False
-import pyrasterframes
-spark = spark.withRasterFrames()
-df = spark.read.raster('https://landsat-pds.s3.amazonaws.com/c1/L8/158/072/LC08_L1TP_158072_20180515_20180604_01_T1/LC08_L1TP_158072_20180515_20180604_01_T1_B5.TIF')
+Welcome to
+      ____              __
+     / __/__  ___ _____/ /__
+    _\ \/ _ \/ _ `/ __/  '_/
+   /__ / .__/\_,_/_/ /_/\_\   version 2.3.2
+      /_/
+
+Using Python version 3.7.3 (default, Mar 27 2019 15:43:19)
+SparkSession available as 'spark'.
+>>> import pyrasterframes
+>>> spark = spark.withRasterFrames()
+>>> df = spark.read.raster('https://landsat-pds.s3.amazonaws.com/c1/L8/158/072/LC08_L1TP_158072_20180515_20180604_01_T1/LC08_L1TP_158072_20180515_20180604_01_T1_B5.TIF')
 ```
 
 Now you have the configured SparkSession with RasterFrames enabled.
 
-```python, echo=False 
-spark.stop()
+#### Submitting Apps
+
+Prepare the call to `spark-submit` in much the same way as using the `pyspark` shell. In the python script you submit, you will use the SparkSession builder pattern and add some RasterFrames extras to it. You have more flexibility in setting up configurations in either your script or in the `spark-submit` call. 
+
+```python, evaluate=False
+# contents of app.py
+
+from pyspark.sql import SparkSession
+import pyrasterframes
+spark = (SparkSession.builder
+             .appName("My RasterFrames app")
+             .config('spark.some_config', some_val) # app configurations
+             .withKryoSerialization() # sets spark.serializer and spark.kryo configs
+             .getOrCreate()).withRasterFrames()
+df = spark.read.raster('...')
+```
+
+To submit, use a call like:
+
+```bash 
+$ spark-submit \
+    --master spark://sparkmaster:7077 \
+    --jars pyrasterframes/target/scala-2.11/pyrasterframes-assembly-${VERSION}.jar \
+    --py-files pyrasterframes/target/scala-2.11/pyrasterframes-python-${VERSION}.zip \
+    app.py
 ```
 
 ## Installing GDAL