Nbclean precommit (#35)

* fix precommit

* test nb

* updates

* add black linelength format to nbs

* a lot of wording and formatting changes

* switch build branch back

Author: e-marshall

.pre-commit-config.yaml

@@ -34,4 +34,10 @@ repos:
           - tags
           - slideshow
           - --
-        exclude: "^.*/data/.*$"
+        exclude: "^.*/data/.*$"
+
+  - repo: https://github.com/tcort/markdown-link-check
+    rev: v3.13.7
+    hooks:
+      - id: markdown-link-check
+        args: [-q]

book/_config.yml

@@ -9,6 +9,7 @@ collapse_inactive_sections: true
 #only want to include files listed in toc
 only_build_toc_files: false
 
+
 # Force re-execution of notebooks on each build.
 # See https://jupyterbook.org/content/execute.html
 execute:
@@ -41,6 +42,8 @@ html:
   home_page_in_navbar: false
   use_issues_button: true
   use_repository_button: true
+  use_multitoc_numbering: false
+
   announcement: >
     🚧 March 2025 - This book is under construction! 🚧
 
@@ -251,9 +254,13 @@ sphinx:
       d2_s1_nb5: "2) Mean over space"
       d3_s1_nb5: "3) Difference"
 
+# Not sure why but uncommenting these causes all of the md 
+# substitution variables and formatting like tabs to not work
+#sphinx:
 #  extra_extensions:
+#    - sphinx_design
 #  - sphinxext.opengraph
 #  config:
-#    ogp_site_url: "https://e-marshall.github.io/itslive/intro.html"
+#    ogp_site_url: "file:///home/emmamarshall/Desktop/phd_research/ch1/cloud_os_geospatial_datacube_workflows/book/_build/html/introduction.html"
 #    ogp_image: "https://e-marshall.github.io/itslive/_static/itslive_explore_img.png"
 #    ogp_description_length: 200

book/_toc.yml

@@ -3,11 +3,13 @@ format: jb-book
 root: introduction
 parts:
   - caption: Introduction
+    #numbered: 2
     chapters:
     - file: intro/getting_started
     - file: intro/learning_objectives
     - file: intro/open_source_setting
   - caption: Background
+    #numbered: 2
     chapters:
     - file: background/context_motivation
     - file: background/data_cubes
@@ -16,6 +18,7 @@ parts:
     - file: intro/software
     - file: background/relevant_concepts
   - caption: "Part 1: ITS_LIVE"
+    #numbered: 2
     chapters:
     - file: itslive/itslive_intro
     - file: itslive/nbs/1_accessing_itslive_s3_data
@@ -24,18 +27,20 @@ parts:
     - file: itslive/nbs/4_exploratory_data_analysis_single
     - file: itslive/nbs/5_exploratory_data_analysis_group
   - caption: Part 2
+    #numbered: 2
     chapters:
     - file: sentinel1/s1_intro
     - file: sentinel1/nbs/1_read_asf_data
     - file: sentinel1/nbs/2_wrangle_metadata
     - file: sentinel1/nbs/3_asf_exploratory_analysis
     - file: sentinel1/nbs/4_read_pc_data
     - file: sentinel1/nbs/5_comparing_s1_rtc_datasets
-  - caption: Summary + Conclusion
+  - caption: Conclusion
+    #numbered: 2
     chapters:
-    - file: pt4/summary
-    - file: pt4/datacubes_revisited
-    - file: pt4/conclusion
+    - file: conclusion/conclusion
+    - file: conclusion/summary
+    - file: conclusion/datacubes_revisited
   - caption: Additional material
     chapters:
     - file: endmatter/appendix

book/background/tutorial_data.md

@@ -32,6 +32,10 @@ Be sure to also check out the ITS_LIVE image pair velocities [documentation](htt
 
 Part 2 focuses on Sentinel-1 Radiometric Terrain Corrected imagery. Sentinel-1 is a dataset of synthetic aperture radar (SAR) imagery collected from sensors located on satellites operated by the Sentinel satellites operated by the European Space Agency (ESA). SAR data is exciting because doesn't require solar illumination like passive optical systems and, at the wavelength where Sentinel-1 imagery is collected, it is minimally impacted by atmospheric water vapor, meaning that Sentinel-1 can acquire clear images of Earth's surface even during cloudy and nighttime conditions. SAR imagery has a wide range of scientific applications including monitoring land surface deformation related to seismic activities, tracking flooding extents following extreme weather events, and mapping deforestation and characterizing biomass. 
 
+:::{tip}
+For an in-depth example of how SAR backscatter data can be used to map flooding extent, check out this [notebook](https://projectpythia.org/eo-datascience-cookbook/notebooks/tutorials/floodmapping.html) in the [Project Pythia Earth Observation Data Science Cookbook](https://projectpythia.org/eo-datascience-cookbook/README.html).
+:::
+
 Because SAR imagery is collected from a side-looking sensor, it can contain distortions related to the viewing geometry of the sensor and the surface topography of the area being imaged. This tutorial focuses on RTC imagery, which is SAR data that has undergone processing to remove the above-mentioned distortions. 
 
 Multiple algorithms perform radiometric terrain correction, and it is important to understand the components of whichever dataset you use and their relative benefits and tradeoffs. This book will demonstrate working with two different (but similar) datasets of Sentinel-1 RTC imagery: one produced by Alaska Satellite Facility and one produced by Microsoft Planetary Coomputer, shown below. Processing of SAR imagery can be very computationally intensive, both of these options leverage cloud-hosted computational resources to make processed SAR imagery available to users, reducing the need for individual users to perform complicated, resource and time-intensive processing. 

book/pt4/datacubes_revisited.md book/conclusion/datacubes_revisited.mdbook/pt4/datacubes_revisited.md renamed to book/conclusion/datacubes_revisited.md

@@ -29,7 +29,7 @@ We saw many examples of situations where it would have been very difficult to ac
 ## Data visualization
 We used a number of visualization tools in these tutorials, each of which are appropriate for different situations and use-cases. These include interactive and static visualizations and plotting tools optimized for n-d array and vector datasets. 
 
-*See (add links to facetgrid plot notebook examples, hvplot examples and geopandas explore examples).
+- [ITS_LIVE, nb3](https://e-marshall.github.io/cloud-open-source-geospatial-datacube-workflows/itslive/nbs/3_combining_raster_vector_data.html#crop-vector-data-to-spatial-extent-of-raster-data), [Adding basemaps to plots of raster and vector data, ITS_LIVE nb4](https://e-marshall.github.io/cloud-open-source-geospatial-datacube-workflows/itslive/nbs/4_exploratory_data_analysis_single.html#load-raster-data-and-visualize-with-vector-data), [Interactive visualization of vector data cubes with Xvec and GeoPandas](https://e-marshall.github.io/cloud-open-source-geospatial-datacube-workflows/itslive/nbs/5_exploratory_data_analysis_group.html#visualize-velocity-data), [Interactive visualization of 2-d data with Xarray and holoviz, Sentinel-1 nb3](https://e-marshall.github.io/cloud-open-source-geospatial-datacube-workflows/sentinel1/nbs/3_asf_exploratory_analysis.html#interactive-visualization-of-layover-shadow-maps), [Using xr.FacetGrid for data inspection and cleaning[(https://e-marshall.github.io/cloud-open-source-geospatial-datacube-workflows/sentinel1/nbs/3_asf_exploratory_analysis.html#d-duplicate-time-steps), [Different ways of plotting 2-d data side-by-side, Sentinel-1 nb3](https://e-marshall.github.io/cloud-open-source-geospatial-datacube-workflows/sentinel1/nbs/3_asf_exploratory_analysis.html#f-data-visualization), [Interactive visualization of time series data with Xarray and holoviz](https://e-marshall.github.io/cloud-open-source-geospatial-datacube-workflows/sentinel1/nbs/3_asf_exploratory_analysis.html#backscatter-time-series), and [here](https://e-marshall.github.io/cloud-open-source-geospatial-datacube-workflows/sentinel1/nbs/5_comparing_s1_rtc_datasets.html#d-visualize-comparisons).
 
 ## Making analysis-ready data cubes
 Several notebooks focused on the tasks of organizing data cubes so that they were appropriate representations of physical observables and attaching relevant metadata to relevant variables and along appropriate dimensions.

book/pt4/nb_summaries.md book/conclusion/nb_summaries.mdbook/pt4/nb_summaries.md renamed to book/conclusion/nb_summaries.md

@@ -0,0 +1,36 @@
+# Wrapping up
+
+It is a popular refrain, and a sentiment many analysts can likey relate to, "that 80% of data analysis is spent on the cleaning and preparing of data" {cite:t}`Wickham_2014_Tidy,Dasu_2003_Exploratory`. This book focuses on the data cleaning and preparation steps of an analytical workflow that ingests satellite remote sensing time series datasets. We draw on the wealth of knowledge and research that attends to this topic in order to produce tutorials that demonstrate and explain these concepts in the context of cloud-optimized, publicly available array data and the software ecosystem built around the Xarray data model in Python. 
+
+In this chapter, you will find summaries of the concepts covered throughout the Jupyter Notebooks included in this book, a return to the introduction's [discussion of data cubes](../background/data_cubes.md) with synthesis and lessons learned from the tutorials, and a short discussion of the broader context of this book and next steps for interested readers. 
+
+## Tutorials Summary[$\tiny \nearrow$](summary.md)
+
+This book features two tutorials, each focuses on a different earth observation dataset and containing five notebooks that cover different steps of a typical workflow such as data access, manpiulation and organizatoin and visualization and exploratory analysis. In this section, you will find a few of the common topics throughout these notebooks and links to where they are addressed in each tutorial. 
+
+## Data cubes revisited [$\tiny \nearrow$](datacubes_revisited.md)
+
+Synthesizing lessons from tutorial examples to enumerate guidance and best-practices for working Xarray geospatial data cubes.
+
+## Broader context [$\tiny \nearrow$]()
+
+It is a popular refrain, and a sentiment many analysts can likey relate to, "that 80% of data analysis is spent on the cleaning and preparing of data" {cite:t}`Wickham_2014_Tidy,Dasu_2003_Exploratory`. This book focuses on the data cleaning and preparation steps of an analytical workflow. We draw on the wealth of knowledge and research that attends to this topic in order to produce tutorials that demonstrate and explain these topics in the context of satellite remote sensing earth observation datasets and the software ecosystem built around the Xarray data model in Python.  
+This book largely focused on the beginning steps of scientific workflows where data is prepared for analysis and manipulated to support different types of analysis. 
+
+### Open source tools and packages
+In this book, we mainly focused on Xarray and several tools within the Xarray ecosystem and that integrate with Xarray to streamline data cube workflows such as:
+- Xvec  
+- cf_xarray   
+- Dask   
+- PySTAC   
+- stackstac   
+- GeoPandas  
+- holoviz
+
+There are many exciting open-source projects and tools related to Xarray data cubes that were not highlighted in this book. A few are:
+- [XRLint](https://github.com/bcdev/xrlint)
+- [LexCube](https://www.lexcube.org/)
+- [xcube](https://xcube.readthedocs.io/en/latest/)
+- [cubo](https://github.com/ESDS-Leipzig/cubo)
+- [Open Data Cube](https://opendatacube.readthedocs.io/en/latest/index.html)
+

book/pt4/summary.md book/conclusion/summary.mdbook/pt4/summary.md renamed to book/conclusion/summary.md

@@ -9,21 +9,32 @@ Throughout this book, we reference a number of resources on different topics. Th
 - [Xarray High-level computational patterns](https://tutorial.xarray.dev/intermediate/01-high-level-computation-patterns.html)
 - [Parallel computing with dask](https://tutorial.xarray.dev/intermediate/xarray_and_dask.html)
 - [Terminology](https://docs.xarray.dev/en/stable/user-guide/terminology.html)
+- [Xarray, Dask - Chunking & Performance](https://docs.xarray.dev/en/stable/user-guide/dask.html#chunking-and-performance)
+- [Regridding with Xarray Cookbook - Project Pythia](https://projectpythia.org/gridding-cookbook/README.html)
 
-## Geospatial data science and data formats
+## Dask resources
+- [Choosing good chunk sizes in Dask](https://blog.dask.org/2021/11/02/choosing-dask-chunk-sizes)
+- [Dask Cookbook - Project Pythia](https://projectpythia.org/dask-cookbook/README.html)
+## Geospatial data science, data formats, and metadata
 - [Introduction to Earth and Environmental Data Science](https://earth-env-data-science.github.io/intro.html)
 - [Geocomputation wtih Python](https://py.geocompx.org/)
 - [Datacubes - openEO](https://openeo.org/documentation/1.0/datacubes.html#what-are-datacubes)
 - [Data Formats - NASA Earth Observation Data Basics](https://www.earthdata.nasa.gov/learn/earth-observation-data-basics/data-formats)
 - [Cloud Computing - NASA Earth Observation Data Basics](https://www.earthdata.nasa.gov/learn/earth-observation-data-basics/cloud-computing)
 - [Vector Data Cubes - Edzer Pebesma](https://r-spatial.org/r/2022/09/12/vdc.html)
-
+- [Climate Forecase Conventions](https://cfconventions.org/)
+- [GDAL Virtual Raster Format](https://gdal.org/en/stable/drivers/raster/vrt.html)
+- [STAC Specification](https://stacspec.org/en)
+- 
 ## Geospatial data and coordinate reference systems
 - [Coordinate Reference Systems - Geocomputaion with Python](https://py.geocompx.org/01-spatial-data#geographic-coordinate-systems)
 - [Reprojecting Geographic Data - Geocomputation with Python](https://py.geocompx.org/06-reproj)
+- [PyPROJ](https://pyproj4.github.io/pyproj/stable/)
+- [cfxarray](https://cf-xarray.readthedocs.io/en/latest/)
 
 ## Remote sensing data types and formats 
 - [Types of SAR Products - NASA Earth Observation Data Basics](https://www.earthdata.nasa.gov/learn/earth-observation-data-basics/types-sar-products)
+- [CEOS Analysis Ready Data](https://ceos.org/ard/index.html)
 
 ## Remote sensing principles
 - [Introduction to Synthetic Aperture Radar (SAR) - Alaska Satellite Facility](https://hyp3-docs.asf.alaska.edu/guides/introduction_to_sar/)
@@ -32,12 +43,18 @@ Throughout this book, we reference a number of resources on different topics. Th
 - [SAR Image Interpretation - NASA Earth Observation Data Basics](https://www.earthdata.nasa.gov/learn/earth-observation-data-basics/sar/image-interpretation)
 - [Microwave Remote Sensing - University of Alaska Fairbanks](https://radar.community.uaf.edu/)
 - [A mathematical tutorial on SAR - Margaret Cheney, NASA EarthData](https://www.earthdata.nasa.gov/s3fs-public/2024-06/sar%20mathematical%20tutorial.pdf)
+- [ASF HyP3 Introduction to SAR](https://hyp3-docs.asf.alaska.edu/guides/introduction_to_sar/
+)
 
-## Data documentation
+## Data  & Processing documentation
 - [ITS_LIVE Documentation](https://its-live.jpl.nasa.gov/#documentation)
 - [ESA Sentinel-1 Documentation](https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar/overview)
 - [ASF Sentinel-1 RTC Product Guide](https://hyp3-docs.asf.alaska.edu/guides/rtc_product_guide/)
+- [Microsoft Planetary Computer - Sentinel-1 RTC data](https://planetarycomputer.microsoft.com/dataset/sentinel-1-rtc
+)
 - [Randolph Glacier Inventory User Guide](http://www.glims.org/rgi_user_guide/welcome.html)
+[ASF HyP3 Tutorials](https://hyp3-docs.asf.alaska.edu/tutorials/)
 
-## Software tips and best practices
-- [Choosing good chunk sizes in Dask](https://blog.dask.org/2021/11/02/choosing-dask-chunk-sizes)
+## More data cube resources
+- [The Data Cube Manifesto](http://www.earthserver.eu/tech/datacube-manifesto/The-Datacube-Manifesto.pdf
+)