Fix: edits to package guide from review

Fix: merge conflict fix

Fix: edits to package guide from review

rebase 7

Fix: remove chunk of commented text

Fix: edits to package guide

Fix: edits to package guide

Continued edits to pages

Update package-structure-code/python-package-build-tools.md

Co-authored-by: Randy Döring <[email protected]>

Author: lwasser

conf.py

@@ -128,7 +128,7 @@
 html_theme = 'pydata_sphinx_theme'
 html_static_path = ["_static"]
 html_css_files = ["pyos.css"]
-html_title = "pyOpenSci Package Guide"
+html_title = "pyOpenSci Python Packaging Guide"
 html_logo = "images/logo/logo.png"
 
 # Add any paths that contain custom static files (such as style sheets) here,

index.md

@@ -1,18 +1,20 @@
 # pyOpenSci Python Open Source Package Development Guide
 
 
+
 ```{toctree}
 :hidden:
 :caption: Documentation
 
-Documentation <documentation/index>
-```
+Documentation Overview <documentation/index>
 
+```
 ```{toctree}
 :hidden:
 :caption: Packaging
 
-Packaging  <python-packaging/intro>
+Packaging <package-structure-code/intro>
+
 ```
 
 ```{toctree}
@@ -120,31 +122,6 @@ This guide is now a work in progress. If you have ideas of things you'd like
 to see here, [we invite you to open an issue on GitHub that details any changes or additions that you'd like to see.](https://github.com/pyOpenSci/python-package-guide/issues).
 
 
-```{toctree}
-:hidden:
-:caption: Documentation
-
-Documentation Overview <documentation/index>
-
-```
-
-
-```{toctree}
-:hidden:
-:caption: Package structure & code style
-
-Intro <python-packaging/intro>
-Intro <package-structure-code/intro>
-
-Python package structure <package-structure-code/python-package-structure>
-Package Build Tools <package-structure-code/python-package-build-tools>
-Package Versions <package-structure-code/python-package-versions>
-package-structure-code/overview
-package-structure-code/collaboration
-Code Style & Format <package-structure-code/code-structure-style>
-```
-
-
 
 
 <!--

package-structure-code/complex-python-package-builds.md

@@ -0,0 +1,77 @@
+# Complex Python package builds
+
+
+* link to ralf's blog and book on complex builds
+* keep this page high level so we don't get weight downsides
+* can use the examplePy repo stefan and I are working on that will test various build combinations
+
+
+
+
+* pdm, hatch and poetry all have "ways" of supporting c extensions via pdm-build, hatchling and poetry's build back end.
+* poetry's support for C extensions is not fully developed and documented (yet). * Poetry doesn't offer a way to facilitate "communication" between poetry front end and another back end like meson to build via a build hook. so while some have used it with other back end builds it's not ideal for this application
+* pdm and poetry both rely on setuptools for C extensions. pdm's support claims to be fully developed and documented. poetry claims nothing, and doesn't document it.
+* hatch both offers a plugin type approach to support custom build steps
+PDM (right now) is the only tool that supports other back ends (hatch is working on this - 2 minor releases away)
+At some point a build becomes so complex that you need to use a tool like scikit or meson to support that complexity.
+
+
+
+**Setuptools** is the oldest tool in the above list. While it doesn't have a
+friendly user front end, because "OG" tool that has been used for Python packaging for over a decade, we discuss it here.
+
+**Hatch** and PDM are newer, more modern tool that support customization of any
+part of your packaging steps. These tools also support some C and C++
+extensions.
+
+
+
+<!-- TODO: add - compatible with other build back ends eg pdm can work with hatchling
+
+Eli:
+poetry: supports it, but is undocumented and uses setuptools under the hood, they plan to change how this works and then document it
+pdm-backend: supports it, and documents it -- and also uses setuptools under the hood
+hatchling: permits you to define hooks for you to write your own custom build steps, including to build C++ extensions
+
+-->
+
+
+
+<!-- from eli about pdm
+It would be more accurate to say that PDM supports using PDM and setuptools at the same time, so you run setuptools to produce the C extensions and then PDM receives the compiled extension files (.so, .pyd) and packages it up alongside the pure Python files.
+
+Comment about hatch.
+https://github.com/pyOpenSci/python-package-guide/pull/23#discussion_r1081108118
+
+From ralf: There are no silver bullets here yet, no workflow tool is complete. Both Hatch and PDM are single-author tools, which is another concern. @eli-schwartz's assessment is unfortunately correct here I believe (at a high level at least, not sure about details). Hatch has the worst take on building compiled code by some distance. Unless its author starts developing an understanding of build systems / needs, and implements support for PEP 517 build backend hooks in pyproject.toml, it's pretty much a dead end.
+
+-->
+
+<!--TODO Add examples of builds using each of the tools below?
+
+pdm, hatch and poetry all have "ways" of supporting c extensions via pdm-build, hatchling and poetry's build back end.
+poetry's support for C extensions is not fully developed and documented (yet). Poetry doesn't offer a way to facilitate "communication" between poetry front end and another back end like meson to build via a build hook.
+PDM and hatch both offer a plugin type approach to support custom build steps
+PDM (right now) is the only tool that supports other back ends (hatch is working on this - 2 minor releases away)
+At some point a build becomes so complex that you need to use a tool like scikit or meson to support that complexity.
+
+CORRECTIONS:
+pdm doesn't use plugins. Hatch does.
+pdm and poetry both rely on setuptools for C extensions. pdm's support claims to be fully developed and documented. poetry claims nothing, and doesn't document it.
+
+-->
+
+```{note}
+??
+Poetry supports extensions written in other languages but this functionality is
+currently undocumented.
+
+Tools such as Setuptools, PDM, Hatch and Poetry all have some level of support
+for C and C++ extensions.
+Some Python packaging tools,
+such as **Flit** and the **flit-core** build back-end only support pure-Python
+package builds.
+Some front-end packaging tools, such as PDM, allow you to use other
+build back-ends such as **meson** and **scikit-build**.
+
+```

package-structure-code/intro.md

@@ -24,6 +24,11 @@ package to be reviewed and accepted into our ecosystem.
 
 In all cases, we try to align our suggestions with the most current, accepted
 [PEP's (Python Enhancement Protocols)](https://peps.python.org/pep-0000/) and the [scientific-python community specs](https://scientific-python.org/specs/).
+If you plan to submit a package for review to pyOpenSci and are looking for
+some guidance on package structure, code formats and style, then this section
+is for you.
+
+<!-- TODO: move this either to the top of this section or the landing page?-->
 
 ```{note}
 Have a look at the
@@ -35,14 +40,38 @@ anyone who is just getting started with creating a Python package.
 In general these are basic items that should be in any open software repository.
 ```
 
+## Guidelines for pyOpenSci's packaging recommendations
 
-<!--
+<!-- Might belong on the LANDING page for this entire guide?-->
+
+Python is a flexible programming language that is used across numerous
+disciplines and domains. Python is so flexible that it is one of the few
+languages that can be used to wrap around other languages.
+
+If you are building a pure Python package, then your packaging setup can be
+simple. However, some scientific packages have complex requirements as they may
+need to support extensions or tools written in other languages such as C or C++.
+
+To support the many different uses of Python, there are many ways to create a
+Python package.
+
+The ecosystem is dynamic, and constantly evolving to support
+the numerous needs that developers (and scientists) have using Python.
+
+This dynamic yet flexible environment is what many love about Python.
+
+## What you will learn here
+
+In this section of our Python packaging guide, we try to:
+
+* Provide an overview of the options available to you when packaging your tool
+* Suggest tools and approaches that both meet your needs and also support existing standards.
+* Suggest tools and approaches that will allow you to expand upon a workflow that may begin as a pure Python tool and evolve into a tool that requires addition layers of complexity in the packaging build.
+* Align our suggestions with the most current, accepted
+[PEPs (Python Enhancement Protocols)](https://peps.python.org/pep-0000/) and the [scientific-python community SPECs](https://scientific-python.org/specs/).
+* In an effort to maintain consistency within our community , we also align with existing best practices being implemented by developers of core Scientific Python packages such as Numpy, SciPy and others.
 
-These checks include several items
 
-- **Sufficient Documentation** The package has sufficient documentation available online (README, sphinx docs) to allow us to evaluate package function and scope *without installing the package*. This includes:
-  Get started tutorials or vignettes that help a user understand how to use the package and what it can do for them (often these have a name like "Getting started")
-- **API documentation** - this includes clearly written doc strings with variables defined using a standard docstring format -->
 <!--
 ```{tip}
 ### Python packaging resources that we love
@@ -57,3 +86,16 @@ the `src` directory.
 * [PyPA packaging guide](https://packaging.python.org/en/latest/)
 ```
 -->
+
+
+```{toctree}
+:hidden:
+:caption: Package structure & code style
+
+Intro <self>
+
+Python package structure <python-package-structure>
+What are SDIst & Wheel Files? <python-package-distribution-files-sdist-wheel>
+Package Build Tools <python-package-build-tools>
+Complex Builds <complex-python-package-builds>
+```