Update docs raster-read to discuss geotrellis

Signed-off-by: Jason T. Brown <[email protected]>

Author: vpipkt

pyrasterframes/src/main/python/docs/raster-read.pymd

@@ -8,11 +8,13 @@ from pyrasterframes.rasterfunctions import *
 spark = create_rf_spark_session()
 ```
 
-RasterFrames registers a DataSource named `raster` that enables reading of GeoTIFFs (and other formats when @ref:[GDAL is installed](getting-started.md#installing-gdal)) from arbitrary URIs. In the examples that follow we'll be reading from a Sentinel-2 scene stored in an AWS S3 bucket.
+RasterFrames registers a DataSource named `raster` that enables reading of GeoTIFFs (and other formats when @ref:[GDAL is installed](getting-started.md#installing-gdal)) from arbitrary URIs. The `raster` DataSource operates on either a single raster file location or another DataFrame, called a _catalog_, containing pointers to many raster file locations.
+
+RasterFrames can also read from @ref:[GeoTrellis catalogs and layers](raster-read.md#geotrellis).
 
 ## Single Raster
 
-The simplest form is reading a single raster from a single URI.
+The simplest way to use the `raster` reader is with a single raster from a single URI or file. In the examples that follow we'll be reading from a Sentinel-2 scene stored in an AWS S3 bucket.
 
 ```python read_one_uri
 rf = spark.read.raster('https://s22s-test-geotiffs.s3.amazonaws.com/luray_snp/B02.tif')
@@ -56,19 +58,19 @@ RasterFrames relies on three different IO drivers, selected based on a combinati
 
 | Prefix              | GDAL        | Java I/O | Hadoop   |
 | ------------------- | ----------- | -------- | -------- |
-| `gdal://<vsidrv>//` | +    | -        | -        |
-| `file://`           | +    | + | -        |
-| `http://`           | +    | + | -        |
-| `https://`          | +    | + | -        |
-| `ftp://`            | `/vsicurl/` | + | -        |
-| `hdfs://`           | `/vsihdfs/` |-         | + |
-| `s3://`             |  `/vsis3/`  | + | -        |
-| `s3n://`            |  -          | -        | + |
-| `s3a://`            |  -          | -        | + |
-| `wasb://`           |  `/vsiaz/`  | -        | + |
-| `wasbs://`          |  -          | -        | + |
+| `gdal://<vsidrv>//` | +           | -        | -        |
+| `file://`           | +           | +        | -        |
+| `http://`           | +           | +        | -        |
+| `https://`          | +           | +        | -        |
+| `ftp://`            | `/vsicurl/` | +        | -        |
+| `hdfs://`           | `/vsihdfs/` | -        | + |
+| `s3://`             | `/vsis3/`   | +        | -        |
+| `s3n://`            | -           | -        | + |
+| `s3a://`            | -           | -        | + |
+| `wasb://`           | `/vsiaz/`   | -        | + |
+| `wasbs://`          | -           | -        | + |
 
-Specific [GDAL Virtual File System drivers](https://gdal.org/user/virtual_file_systems.html) can be selected using the `gdal://<vsidrv>//` syntax. For example If you have a `archive.zip` file containing a GeoTiff named `my-file-inside.tif`, you can address it with `gdal://vsizip//path/to/archive.zip/my-file-inside.tif`. See the GDAL documentation for the format of the URIs after the `gdal:/` prefix (which is stripped off before passing the rest of the path to GDAL).
+Specific [GDAL Virtual File System drivers](https://gdal.org/user/virtual_file_systems.html) can be selected using the `gdal://<vsidrv>//` syntax. For example If you have a `archive.zip` file containing a GeoTiff named `my-file-inside.tif`, you can address it with `gdal://vsizip//path/to/archive.zip/my-file-inside.tif`. Another example would be a MRF file in an S3 bucket on AWS: `gdal://vsis3/my-bucket/prefix/to/raster.mrf`. See the GDAL documentation for the format of the URIs after the `gdal:/` scheme. The `gdal:/` scheme is stripped off before passing the rest of the path to GDAL.
 
 
 ## Raster Catalogs
@@ -127,7 +129,7 @@ Observe that the schema of the resulting DataFrame has a projected raster struct
 rf.select('gid', rf_extent('red'), rf_extent('nir'), rf_tile('red'), rf_tile('nir')).show(3, False)
 ```
 
-### Lazy Raster Reads
+## Lazy Raster Reads
 
 By default the raster reads are delayed as long as possible. The DataFrame will contain metadata and pointers to the appropriate portion of the data until
 
@@ -180,6 +182,29 @@ mb2 = spark.read.raster(catalog=spark.createDataFrame(mb_cat),
 mb2.printSchema()
 ```
 
+## GeoTrellis
+
+### GeoTrellis Catalogs
+
+[GeoTrellis][GeoTrellis] is one of the key libraries that RasterFrames builds upon. It provides a Scala language API to working with large raster data with Apache Spark. RasterFrames provides a DataSource that supports both reading and @ref:[writing](raster-write.md#geotrellis-layers) with GeoTrellis.
+
+A GeoTrellis catalog is a set of GeoTrellis layers. We can read a dataframe giving details of the content of a catalog using the following. The scheme is typically `hdfs` or a cloud storage provider like `s3` or `wasb`.
+
+```python, evaluate=False
+gt_cat = spark.read.format('geotrellis-catalog').load('scheme://path-to-gt-catalog')
+```
+
+### GeoTrellis Layers
+
+The catalog will give details on the particular layers available for query. We can read the layer with the same URI to the catalog, the layer name, and the desired zoom level.
+
+```python, evaluate=False
+gt_layer = spark.read.geotrellis(path='scheme://path-to-gt-catalog', layer=layer_name, zoom=zoom_level)
+```
+
+This will return a RasterFrame with additional metadata inherited from the GeoTrellis TileLayerMetadata, such as the SpatialKey. The TileLayerMetadata is also stored as json in the metadata of the tile column.
+
 [CRS]: concepts.md#coordinate-reference-system--crs
 [Extent]: concepts.md#extent
 [Tile]: concepts.md#tile
+[GeoTrellis]: https://geotrellis.readthedocs.io/en/latest/

pyrasterframes/src/main/python/docs/raster-write.pymd

@@ -85,7 +85,7 @@ If there are many tile or projected raster columns in the DataFrame, the GeoTIFF
 
 ## GeoTrellis Layers
 
-[GeoTrellis][GeoTrellis] is one of the key libraries that RasterFrames builds upon. It provides a Scala language API to working with large raster data with Apache Spark. Ingesting raster data into a Layer is one of the key concepts for creating a dataset for processing on Spark. RasterFrames write data from an appropriate DataFrame into a [GeoTrellis Layer](https://geotrellis.readthedocs.io/en/latest/guide/tile-backends.html). RasterFrames provides a `geotrellis` DataSource that supports both reading and writing of GeoTrellis layers.
+[GeoTrellis][GeoTrellis] is one of the key libraries that RasterFrames builds upon. It provides a Scala language API to working with large raster data with Apache Spark. Ingesting raster data into a Layer is one of the key concepts for creating a dataset for processing on Spark. RasterFrames writes data from an appropriate DataFrame into a [GeoTrellis Layer](https://geotrellis.readthedocs.io/en/latest/guide/tile-backends.html). RasterFrames provides a `geotrellis` DataSource that supports both @ref:[reading](raster-read.md#geotrellis) and writing of GeoTrellis layers.
 
 > An example is forthcoming.