Merge pull request #121 from uw-cryo/rasterio144

Try updating to gdal3.12 (rasterio1.5.0)

Author: scottyhq

.devcontainer/devcontainer.json

@@ -26,12 +26,9 @@
       "description": "Maxar Platform API Key",
       "documentationUrl": "https://developers.maxar.com/docs/authentication/guides/api-key"
     },
-    "EARTHDATA_USERNAME": {
-      "description": "NASA Earthdata Username",
-      "documentationUrl": "https://earthaccess.readthedocs.io/en/latest/howto/authenticate/#authenticate-with-earthdata-login"
-    },
-    "EARTHDATA_PASSWORD": {
-      "description": "NASA Earthdata Password"
+    "EARTHDATA_TOKEN": {
+      "description": "NASA Earthdata Token",
+      "documentationUrl": "https://urs.earthdata.nasa.gov/documentation/for_users/user_token"
     }
   }
 }

.github/workflows/ci.yml

@@ -41,8 +41,7 @@ jobs:
       fail-fast: false
       matrix:
         # Test against at least 3 most recent Python versions
-        # NOTE: 3.14 waiting on rasterio 1.4.4 release
-        python-version: ["3.11", "3.12", "3.13"]
+        python-version: ["3.12", "3.13", "3.14"]
         runs-on: [ubuntu-latest, macos-latest]
 
     steps:

.github/workflows/upstream.yml

@@ -23,7 +23,9 @@
    "source": [
     "import coincident\n",
     "import geopandas as gpd\n",
-    "from shapely.geometry import box"
+    "from shapely.geometry import box\n",
+    "import matplotlib.pyplot as plt\n",
+    "# %config InlineBackend.figure_format = 'retina'"
    ]
   },
   {
@@ -33,7 +35,7 @@
    "source": [
     "## Search \n",
     "\n",
-    "We'll search for data in Maryland"
+    "We'll search for data in Maryland in July 2017"
    ]
   },
   {
@@ -49,29 +51,37 @@
     "aoi = aoi.simplify(0.01)\n",
     "m = aoi.explore(color=\"black\")\n",
     "\n",
-    "gf_gliht = coincident.search.search(\n",
+    "gf = coincident.search.search(\n",
     "    dataset=\"gliht\",\n",
     "    intersects=aoi,\n",
-    "    datetime=[\"2017-07-31\"],\n",
+    "    datetime=\"2017-07\",\n",
     ")\n",
-    "print(f\"Found {len(gf_gliht)} G-LiHT scenes\")\n",
-    "gf_gliht.explore(m=m, color=\"magenta\")"
+    "\n",
+    "print(f\"Found {len(gf)} G-LiHT Items\")\n",
+    "print(f\"G-LiHT collections: {gf.collection.unique()}\")\n",
+    "gf.explore(m=m, color=\"magenta\")"
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "id": "4",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "# Subset a particular scene\n",
-    "gf_gliht = gf_gliht.iloc[[0]]\n",
-    "c = gf_gliht.geometry.centroid\n",
-    "mini_aoi = gpd.GeoDataFrame(\n",
-    "    geometry=[box(c.x - 0.0045, c.y - 0.0045, c.x + 0.0045, c.y + 0.0045)],\n",
-    "    crs=\"EPSG:4326\",\n",
-    ")"
+    "## Get Data\n",
+    "\n",
+    "NASA G-LiHT has many different provided gridded datasets. The following collections below are the current datasets supported by `coincident`.\n",
+    "\n",
+    "| Collection                      | Description      |\n",
+    "| -- | -- | \n",
+    "| GLORTHO_001 | orthorectified high-resolution aerial photography |\n",
+    "| GLCHMT_001 | maximum canopy height and canopy variability information |\n",
+    "| GLDSMT_001 | Digital Surface Model, Mean, Aspect, Rugosity, and Slope |\n",
+    "| GLDTMT_001 | bare earth elevation, aspect and slope on the EGM96 Geoid |\n",
+    "| GLLIDARPC_001 | LiDAR Point Cloud data product (LAS format) |\n",
+    "\n",
+    "```{note}\n",
+    "Not all G-LiHT flights will contain every single product listed. For example, a flight may have 'dsm' data but not 'ortho' data.\n",
+    "```"
    ]
   },
   {
@@ -81,37 +91,26 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "m = gf_gliht.explore(color=\"black\")\n",
-    "mini_aoi.explore(color=\"magenta\", m=m)"
+    "# Subset a particular scene\n",
+    "collection = \"GLDTMT_001\"\n",
+    "gf_gliht = gf.query(f'collection == \"{collection}\"')\n",
+    "gf_gliht.id"
    ]
   },
   {
-   "cell_type": "markdown",
+   "cell_type": "code",
+   "execution_count": null,
    "id": "6",
    "metadata": {},
+   "outputs": [],
    "source": [
-    "## Get Data\n",
-    "\n",
-    "NASA G-LiHT has many different provided gridded datasets. The following collections below are the current datasets supported by `coincident`.\n",
-    "\n",
-    "```python\n",
-    "    - 'ortho': Orthorectified aerial imagery\n",
-    "    - 'chm': Canopy height model\n",
-    "    - 'dsm': Digital surface model\n",
-    "    - 'dtm': Digital terrain model\n",
-    "    - 'hyperspectral_ancillary': Ancillary HSI data\n",
-    "    - 'radiance': Hyperspectral aradiance\n",
-    "    - 'reflectance': Hyperspectral surface reflectance\n",
-    "    - 'hyperspectral_vegetation': HSI-derived veg indices\n",
-    "```\n",
-    "\n",
-    "```{note}\n",
-    "Not all G-LiHT flights will contain every single product listed. For example, a flight may have 'dsm' data but not 'ortho' data.\n",
-    "```\n",
-    "\n",
-    "### Stream \n",
-    "\n",
-    "First we'll show how to stream gridded DEMs directly into Xarray"
+    "# Just use the first one\n",
+    "gs_item = gf_gliht.iloc[0]\n",
+    "c = gs_item.geometry.centroid\n",
+    "mini_aoi = gpd.GeoDataFrame(\n",
+    "    geometry=[box(c.x - 0.0045, c.y - 0.0045, c.x + 0.0045, c.y + 0.0045)],\n",
+    "    crs=\"EPSG:4326\",\n",
+    ")"
    ]
   },
   {
@@ -121,14 +120,24 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# we'll need the dataset id from our initial search\n",
-    "gf_gliht.id.item()"
+    "m = gf_gliht.explore(column=\"id\")\n",
+    "mini_aoi.explore(m=m, color=\"red\", style_kwds={\"fill\": False, \"weight\": 3})"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "8",
    "metadata": {},
+   "source": [
+    "### Stream \n",
+    "\n",
+    "First we'll show how to stream a subset of a gridded DSM directly into Xarray"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9",
+   "metadata": {},
    "source": [
     "```{important}\n",
     "Unlike the G-LiHT search, you will need NASA Earthdata credentials (aka EarthData Login (EDL)) to read in and download the gridded datasets from G-LiHT. This requires creating an [EDL Token](https://urs.earthdata.nasa.gov/documentation/for_users/user_token) and making sure you've set the environment variable `EARTHDATA_TOKEN=xxxxx`\n",
@@ -138,66 +147,89 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "9",
+   "id": "10",
    "metadata": {},
    "outputs": [],
    "source": [
     "%%time\n",
-    "ds = coincident.io.xarray.load_gliht_raster(\n",
+    "\n",
+    "da = coincident.io.xarray.load_gliht(\n",
+    "    item=gs_item,\n",
     "    aoi=mini_aoi,\n",
-    "    dataset_id=gf_gliht.id.item(),\n",
-    "    product=\"chm\",\n",
     ")\n",
-    "ds"
+    "da"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "10",
+   "id": "11",
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Convert to in-memory DataArray\n",
-    "da_chm = ds[\"CHM\"].squeeze().load()"
+    "coincident.plot.plot_dem(da)\n",
+    "plt.title(gs_item.id);"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "12",
+   "metadata": {},
+   "source": [
+    "### Download\n",
+    "\n",
+    "#### Download multiple STAC Item Assets\n",
+    "\n",
+    "This will put STAC metadata, extended metadata, a browse/thumbnail image, and a TIF in a specified folder"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "11",
+   "id": "13",
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Plot\n",
-    "da_chm.plot.imshow(cmap=\"Greens\");"
+    "# back to a GeoDataFrame from a Series\n",
+    "gfi = gpd.GeoDataFrame([gs_item])\n",
+    "\n",
+    "# And to a Pystac Item\n",
+    "items = coincident.search.to_pystac_items(gfi)\n",
+    "items[0]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "14",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Download the item assets (browse, metadata, thumbnail)\n",
+    "local_item = await coincident.io.download.download_item(\n",
+    "    items[0],\n",
+    "    path=\"/tmp/gliht/\",\n",
+    ")"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "12",
+   "id": "15",
    "metadata": {},
    "source": [
-    "### Download \n",
-    "\n",
-    "```{note}\n",
-    "Since there are so many gridded products for G-LiHT, you can specify multiple products to download in one call. This download will automatically skip any products that aren't available for the specific flight among the current datasets supported by `coincident`\n",
-    "```"
+    "#### Download a single file"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "13",
+   "id": "16",
    "metadata": {},
    "outputs": [],
    "source": [
-    "coincident.io.download.download_gliht_raster(\n",
-    "    aoi=mini_aoi,\n",
-    "    dataset_id=gf_gliht.id.item(),\n",
-    "    products=[\"chm\", \"dsm\", \"dtm\"],\n",
-    "    output_dir=\"/tmp\",\n",
-    ")"
+    "tif_key = [x for x in gs_item.assets.keys() if x.startswith(\"001\")][0]\n",
+    "href = gs_item.assets[tif_key][\"href\"]\n",
+    "coincident.io.download.download_files([href], \"/tmp/\")"
    ]
   }
  ],

docs/datasets/G-LiHT.ipynb

@@ -63,9 +63,7 @@
    "source": [
     "# Print all the metadata for a particular scene\n",
     "# Note 'stereo_pair_identifiers' and 'id' for ordering full-resolution data\n",
-    "with gpd.pd.option_context(\n",
-    "    \"display.max_rows\", None, \"display.max_columns\", None\n",
-    "):  # more options can be specified also\n",
+    "with gpd.pd.option_context(\"display.max_rows\", None, \"display.max_columns\", None):\n",
     "    print(gf_stereo.iloc[0])"
    ]
   },