Fix: massive update to the structure of the docs section

Author: lwasser

ci-and-testing/intro.md

@@ -1,3 +1,7 @@
-# CI and Testing 
+# CI and Testing - Coming Soon!
+
+
+This section is evolving and should be published by the end of Spring 2023
+
 
 coming soon

documentation/hosting-tools/intro.md

@@ -1,19 +1,20 @@
 # Tools to Build and Host your Documentation 
 
-The mmostsot common tool for building documentation in the Python 
-ecosystem is Sphinx. However, some are using tools like 
-mkdocs for documentation. In the end it is up to you to 
+The most common tool for building documentation in the Python 
+ecosystem currently is Sphinx. However, some maintainers 
+are using tools like [mkdocs](https://www.mkdocs.org/) for documentation. It is up to you to 
 use the platform that you prefer for your documentation!
 
-In this section, we introduce sphinx as a common tool to 
+In this section, we introduce Sphinx as a common tool to 
 build documentation. We also talk about ways to publish your 
 documentation online and Sphinx tools that might help you optimize 
 your documentation website. 
 
 ```{toctree}
 :maxdepth: 2
 
-Sphinx Tools <python-package-documentation-tools.md>
+Sphinx for Docs <sphinx-python-package-documentation-tools.md>
+myST vs Markdown vs rst <myst-markdown-rst-doc-syntax.md>
 Publish Your Docs <publish-documentation-online.md>
 Website Hosting and Optimization <website-hosting-optimizing-your-docs.md>
 ```

documentation/hosting-tools/myst-markdown-rst-doc-syntax.md

@@ -0,0 +1,29 @@
+# Documentation syntax: markdown vs. myST vs. rst syntax to create your docs 
+
+There are three commonly used syntaxes for creating Python documentation:
+1. [markdown](https://www.markdownguide.org/): Markdown is an easy-to-learn text 
+syntax. It is the default syntax use in Jupyter Notebooks. There are tools that you can add to a sphinx website that allow it to render markdown as html. However, using markdown to write documentation has limitations. For instance if you want to add references, 
+colored call out blocks and other custom elements to your documentation, you will 
+need to use either **myST** or **rST**.
+1. [rST (ReStructured Text):](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html). **rST** is the native syntax that sphinx supports. rST was the default syntax used for documentation for many years given the limitations of markdown. However, in recent years myST has risen to the top as a favorite for documentation given the flexibility that it allows.
+1. [myST:](https://myst-parser.readthedocs.io/en/latest/intro.html) myST is a combination of vanilla of `markdown` and `rST` syntax. It is a nice option if you are comfortable writing markdown. `myst` is preferred by many because it offers both the rich functionality 
+of rST combined with a simple-to-write markdown syntax. 
+
+While you can chose to use any of the syntaxes listed above, we suggest using 
+`myST` because:
+
+* It is a simpler syntax and thus easier to learn;
+* The above simplicity will make it easier for more people to contribute to your documentation. 
+* Most of your core python package text files, such as your README.md file, are already in `.md` format
+* `GitHub` and `Jupyter Notebooks` support markdown thus it's more widely used in the scientific ecosystem. 
+
+
+```{tip}
+If you are on the fence about myST vs rst, you might find that **myST** is easier 
+for more people to contribute to.  
+```
+
+<!-- TODO 
+- add some text examples of using rst vs md vs myst? 
+- Better explain what rst / myst offer that markdown can't do
+-->

documentation/hosting-tools/python-package-documentation-tools.md

@@ -0,0 +1,67 @@
+# Using Sphinx to Build Python Package Documentation
+
+<!-- TODO: make this into include files so we can have a summary 
+important points page -->
+<!-- 
+```{important}
+
+## Take Aways: Key Python Package Tools to Use
+
+* Use Sphinx to build your documentation
+* Publish your documentation on ReadTheDocs (or GitHub pages if you are more advanced and also prefer to maintain your website locally)
+* Use `myST` syntax to write your documentation 
+* Use sphinx gallery to write tutorials using .py files that automagically have downloadable .py and jupyter notebook files. Use nbsphinx if you prefer writing tutorials in jupyter notebook format and don't need a grid formatted gallery. *Both of these tools will run your tutorials from beginning to end providing an addition layer of testing to your package!*
+* OPTIONAL: Use [doctest](https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html) to run the examples in your code's docstrings as a way to make sure that your code's functions and methods (the API) are running as you expect them to. 
+``` -->
+
+On this page we discuss using [Sphinx](https://www.sphinx-doc.org/) to build your user-facing 
+package documentation. While Sphinx is currently the most 
+commonly-used tool in the scientific Python ecosystem, you 
+are welcome to explore other tools to build documentation 
+such as [mkdocs](https://www.mkdocs.org/) which is gaining 
+popularity in the Python packaging ecosystem. 
+
+```{tip}
+Examples of documentation websites that we love:
+
+* [GeoPandas](https://geopandas.org/en/stable/)
+    * [View rst to create landing page](https://raw.githubusercontent.com/geopandas/geopandas/main/doc/source/index.rst)
+* [verde](https://www.fatiando.org/verde/latest/)
+    * [View verde landing page code - rst file.](https://github.com/fatiando/verde/blob/main/doc/index.rst)
+* [Here is our documentation if you want to see a myST example of a landing page.](https://github.com/pyOpenSci/python-package-guide/blob/main/index.md)
+```
+
+## Sphinx - a static site generator
+
+Sphinx is a [static-site generator](https://www.cloudflare.com/learning/performance/static-site-generator/). A static site generator is a tool that creates 
+html for a website based upon a set of templates. The html files are then served "Statically" which means that there is no generation or modification of the files on the fly. 
+
+Sphinx is written using Python. 
+
+### Sphinx sites can be customized using extensions and themes 
+
+The functionality of Sphinx can be extended using extensions 
+and themes. A few examples include:
+
+* You can apply documentation themes for quick generation of beautiful documentation.
+* You can [automatically create documentation for your package's functions and classes (that package's API) from docstrings in your code using the autodoc extension](https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html)
+* You can [run and test code examples in your docstrings using the doctest extension](https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html)
+* While sphinx natively supports the `rST` syntax. You can add custom syntax parsers to support easier-to-write syntax using tools such as [myst](https://myst-parser.readthedocs.io/).
+
+### Commonly used sphinx themes 
+
+You are free to use whatever sphinx theme that you prefer. 
+However, the most common sphinx themes used in the Python 
+scientific community include:  
+
+* [pydata-sphinx-theme](https://pydata-sphinx-theme.readthedocs.io/) 
+* [sphinx-book-theme](https://sphinx-book-theme.readthedocs.io/)
+* [furo](https://pradyunsg.me/furo/quickstart/)
+
+
+```{tip}
+This book is created using Sphinx and the `furo` theme.
+```
+<!-- Should this also be it's own page?-->
+
+

documentation/hosting-tools/sphinx-python-package-documentation-tools.md

@@ -5,15 +5,32 @@ add some core sphinx extensions (and theme settings) that will help search
 engines such as Google find your documentation. 
 
 ### Google Analytics
+
+```{important} 
+Google analytics [is not compliant with the European General Data Protection Regulation (GDPR)](https://matomo.org/blog/2022/05/google-analytics-4-gdpr/). While there are many components to this regulation, one of the core elements is that you have to let users know on your site that you are collecting data and they have to consent. WHile it is possible to add infrastructure around Google Analytics to make it close to following GDPR regulations, the community is slowly shifting away from Google using open tools such as [Plausible](https://plausible.io/), [Cloudflare Web Analytics](https://www.cloudflare.com/web-analytics/) and [Matomo](https://matomo.org) for web analytics. 
+
+pyOpenSci is currently looking into free options for open source 
+developers. 
+```
 Some of the [sphinx themes such as the `pydata-sphinx-theme` and 
-sphinx-book-theme have built in support for google analytics](https://pydata-sphinx-theme.readthedocs.io/en/latest/user_guide/analytics.html#google-analytics). However, if the theme that you chose does not offer 
+sphinx-book-theme have built in support for Google Analytics](https://pydata-sphinx-theme.readthedocs.io/en/latest/user_guide/analytics.html#google-analytics). However, if the theme that you chose does not offer 
 Google Analytics support, you can use the [`sphinxcontrib-gtagjs` extension](https://github.com/attakei/sphinxcontrib-gtagjs). 
 This extension will add a Google Analytics site tag to each page of your 
 documentation.  
 
-### [sphinx-sitemap](https://sphinx-sitemap.readthedocs.io/en/latest/index.html)
+### [sphinx-sitemap](https://sphinx-sitemap.readthedocs.io/en/latest/index.html) for search engine optimization 
+
+While we are trying to move away from Google Analytics do 
+to compliance and privacy issues, search engine optimization 
+is still important. Google is the most popular search engine.
+And if your documentation is search optimized, users are more 
+likely to find your package!
 
-If you are interested in optimizing your documentation for search engines such as Google, you want a sitemap.xml file. You can submit this sitemap to Google and it will index your entire site. This over time can make the content on your site more visible to others when they search. 
+If you are interested in optimizing your documentation for 
+search engines such as Google, you want a **sitemap.xml** file. 
+You can submit this sitemap to Google and it will index your 
+entire site. This over time can make the content on your site 
+more visible to others when they search. 
 
 This extension is lightweight.
 
@@ -24,4 +41,6 @@ It [requires that you to add it to your sphinx `conf.py` extension list and site
 OpenGraph is an extension that allows you to add metadata to your documentation 
 content pages. [The OpenGraph protocol allows other websites to provide a 
 useful preview of the content on your page when shared](https://www.freecodecamp.org/news/what-is-open-graph-and-how-can-i-use-it-for-my-website/#what-is-open-graph). This is important 
-for when the pages in your documentation are shared.  
+for when the pages in your documentation are shared on social 
+media sites like Twitter and Mastodon and even for shares on 
+tools like Slack and Discourse.