Skip to content

Commit b64b59a

Browse files
committed
Fix: many more comments from review 2
1 parent 7ba8292 commit b64b59a

6 files changed

+115
-112
lines changed

index.md

Lines changed: 6 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
# pyOpenSci Python Open Source Package Development Guide
22

3-
43
```{toctree}
54
:hidden:
65
:caption: Documentation
76
8-
Documentation Overview <documentation/index>
7+
Documentation <documentation/index>
98
109
```
10+
1111
```{toctree}
1212
:hidden:
1313
:caption: Packaging
@@ -45,7 +45,6 @@ community make decisions around how to create scientific Python packages. We are
4545
:columns: 8
4646
:class: sd-fs-3
4747

48-
4948
```{button-link} https://www.pyopensci.org/about-peer-review/
5049
:color: primary
5150
:class: sd-rounded-pill float-left
@@ -61,7 +60,6 @@ Learn about our open peer review process
6160
:::
6261
::::
6362

64-
6563
<!-- I think this is the end of the header - below begins the next grid-->
6664

6765
::::{grid} 1 1 2 2
@@ -92,7 +90,6 @@ commonly used in the scientific Python community.
9290
All of the modern tools discussed in this guide will help you build an efficient packaging workflow. This section helps you select the tool that will work best for you.
9391
:::
9492

95-
9693
:::{grid-item-card}
9794
:link: CONTRIBUTING
9895
:link-type: doc
@@ -106,6 +103,7 @@ contribute.
106103
::::
107104

108105
## Who this guidebook is for
106+
109107
We assume that you are here because you are:
110108

111109
1. Looking for guidance on creating a Python package.
@@ -118,9 +116,9 @@ Well, friend, you've come to the right place!
118116

119117
This guidebook contains:
120118

121-
* Explanation for "Good enough" minimum requirements associated with being reviewed by pyOpenSci
122-
* Explanation of better and best practices in case you want to set the bar higher for your package (which we hope you will)!
123-
* A curated list of resources to help you get your package into documented, usable and tested shape.
119+
- Explanation for "Good enough" minimum requirements associated with being reviewed by pyOpenSci
120+
- Explanation of better and best practices in case you want to set the bar higher for your package (which we hope you will)!
121+
- A curated list of resources to help you get your package into documented, usable and tested shape.
124122

125123
## Where this guide is headed
126124

@@ -131,9 +129,6 @@ Good meets the requirements. Going beyond the minimum can make package maintenan
131129
This guide is now a work in progress. If you have ideas of things you'd like
132130
to see here, [we invite you to open an issue on GitHub that details any changes or additions that you'd like to see.](https://github.com/pyOpenSci/python-package-guide/issues).
133131

134-
135-
136-
137132
<!--
138133
COMMENTED OUT TEXT TO BE MOVED
139134

package-structure-code/complex-python-package-builds.md

Lines changed: 19 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -9,15 +9,13 @@ In the future, we want to provide resources for packaging workflows that require
99

1010
You can classify Python package complexity into three general categories. These
1111
categories can in turn help you select the correct package front-end and
12-
back end tools.
12+
back-end tools.
1313

1414
1. **Pure-python packages:** these are packages that only rely on Python to function. Building a pure Python package is simpler. As such, you can chose a tool below that
15-
has the features that you want and be done with your decision!
15+
has the features that you want and be done with your decision!
1616
2. **Python packages with non-Python extensions:** These packages have additional components called extensions written in other languages (such as C or C++). If you have a package with non-python extensions, then you need to select a build back-end tool that allows you to add additional build steps needed to compile your extension code. Further, if you wish to use a front-end tool to support your workflow, you will need to select a tool that
17-
supports additional build setps. In this case, you could use setuptools. However, we suggest that you chose build tool that supports custom build steps such as Hatch with Hatchling or PDM. PDM is an excellent choice as it allows you to also select your build back end of choice. We will discuss this at a high level on the complex builds page.
18-
3.**Python packages that have extensions written in different languages (e.g. fortran and C++) or that have non Python dependencies that are difficult to install (e.g. GDAL)** These packages often have complex build steps (more complex than a package with just a few C extensions for instance). As such, these packages require tools such as [scikit-build](https://scikit-build.readthedocs.io/en/latest/)
19-
or [meson-python](https://mesonbuild.com/Python-module.html) to build. NOTE: you can use meson-python with PDM.
20-
17+
supports additional build setps. In this case, you could use setuptools. However, we suggest that you chose build tool that supports custom build steps such as Hatch with Hatchling or PDM. PDM is an excellent choice as it allows you to also select your build back-end of choice. We will discuss this at a high level on the complex builds page. 3.**Python packages that have extensions written in different languages (e.g. fortran and C++) or that have non Python dependencies that are difficult to install (e.g. GDAL)** These packages often have complex build steps (more complex than a package with just a few C extensions for instance). As such, these packages require tools such as [scikit-build](https://scikit-build.readthedocs.io/en/latest/)
18+
or [meson-python](https://mesonbuild.com/Python-module.html) to build. NOTE: you can use meson-python with PDM.
2119

2220
<!--
2321
On this page, we will focus on using front-end tools to package pure python
@@ -45,17 +43,17 @@ ELI: It would be more accurate to say that PDM supports using PDM and setuptools
4543
Hatch - https://hatch.pypa.io/latest/config/build/#build-hooks uild hooks
4644
4745
Ralf -
48-
Hatch has the worst take on building compiled code by some distance. Unless its author starts developing an understanding of build systems / needs, and implements support for PEP 517 build backend hooks in pyproject.toml, it's pretty much a dead end.
46+
Hatch has the worst take on building compiled code by some distance. Unless its author starts developing an understanding of build systems / needs, and implements support for PEP 517 build back-end hooks in pyproject.toml, it's pretty much a dead end.
4947
****
5048
5149
5250
HEnry: Poetry will move to PEP 621 configuration in version 2.
5351
54-
* pdm, hatch and poetry all have "ways" of supporting c extensions via pdm-backend, hatchling and poetry's build back end.
55-
* poetry's support for C extensions is not fully developed and documented (yet). * Poetry doesn't offer a way to facilitate "communication" between poetry front end and another back end like meson to build via a build hook. so while some have used it with other back end builds it's not ideal for this application
52+
* pdm, hatch and poetry all have "ways" of supporting c extensions via pdm-build, hatchling and poetry's build back-end.
53+
* poetry's support for C extensions is not fully developed and documented (yet). * Poetry doesn't offer a way to facilitate "communication" between poetry front end and another back-end like meson to build via a build hook. so while some have used it with other back-end builds it's not ideal for this application
5654
* pdm and poetry both rely on setuptools for C extensions. pdm's support claims to be fully developed and documented. poetry claims nothing, and doesn't document it.
5755
* hatch both offers a plugin type approach to support custom build steps
58-
PDM (right now) is the only tool that supports other back ends (hatch is working on this - 2 minor releases away)
56+
PDM (right now) is the only tool that supports other back-ends (hatch is working on this - 2 minor releases away)
5957
At some point a build becomes so complex that you need to use a tool like scikit or meson to support that complexity.
6058
6159
@@ -68,42 +66,40 @@ part of your packaging steps. These tools also support some C and C++
6866
extensions.
6967
7068
71-
OFEK - Why use hatchlin vs pdm back end -
69+
OFEK - Why use hatchlin vs pdm back-end -
7270
File inclusion is more configurable and easier by default
7371
There is already a rich ecosystem of plugins and a well-thought-out interface
7472
Consistency since the official Python packaging tutorial uses Hatchling by default
7573
7674
7775
Henry -
78-
The scikit-hep cookie provides 11 backends including flit-core and hatchling, and I've moved packaging to flit-core, and lots of other things to hatchling, and I can say that hatching's defaults are much nicer than flit-core's. Hatching uses .gitignore to decide what to put in the SDist. Flit-core basically tries to keep its hands off of adding defaults, so you have to configure everything manually. To make it even more confusing, if you use flit instead of a standard tool like build, it will switch to using VCS and those ignored files won't be added - meaning it is really easy to have a project that doesn't support build, including various GitHub Actions. Hatchling wins this by a ton.
76+
The scikit-hep cookie provides 11 back-ends including flit-core and hatchling, and I've moved packaging to flit-core, and lots of other things to hatchling, and I can say that hatching's defaults are much nicer than flit-core's. Hatching uses .gitignore to decide what to put in the sdist. Flit-core basically tries to keep its hands off of adding defaults, so you have to configure everything manually. To make it even more confusing, if you use flit instead of a standard tool like build, it will switch to using VCS and those ignored files won't be added - meaning it is really easy to have a project that doesn't support build, including various GitHub Actions. Hatchling wins this by a ton.
7977
80-
<!-- TODO: add - compatible with other build back ends eg pdm can work with hatchling
78+
<!-- TODO: add - compatible with other build back-ends eg pdm can work with hatchling
8179
8280
Eli:
8381
poetry: supports it, but is undocumented and uses setuptools under the hood, they plan to change how this works and then document it
84-
pdm-backend: supports it, and documents it -- and also uses setuptools under the hood
82+
pdm-back-end: supports it, and documents it -- and also uses setuptools under the hood
8583
hatchling: permits you to define hooks for you to write your own custom build steps, including to build C++ extensions
8684
8785
-->
8886

89-
90-
9187
<!-- from eli about pdm
9288
It would be more accurate to say that PDM supports using PDM and setuptools at the same time, so you run setuptools to produce the C extensions and then PDM receives the compiled extension files (.so, .pyd) and packages it up alongside the pure Python files.
9389
9490
Comment about hatch.
9591
https://github.com/pyOpenSci/python-package-guide/pull/23#discussion_r1081108118
9692
97-
From ralf: There are no silver bullets here yet, no workflow tool is complete. Both Hatch and PDM are single-author tools, which is another concern. @eli-schwartz's assessment is unfortunately correct here I believe (at a high level at least, not sure about details). Hatch has the worst take on building compiled code by some distance. Unless its author starts developing an understanding of build systems / needs, and implements support for PEP 517 build backend hooks in pyproject.toml, it's pretty much a dead end.
93+
From ralf: There are no silver bullets here yet, no workflow tool is complete. Both Hatch and PDM are single-author tools, which is another concern. @eli-schwartz's assessment is unfortunately correct here I believe (at a high level at least, not sure about details). Hatch has the worst take on building compiled code by some distance. Unless its author starts developing an understanding of build systems / needs, and implements support for PEP 517 build back-end hooks in pyproject.toml, it's pretty much a dead end.
9894
9995
-->
10096

10197
<!--TODO Add examples of builds using each of the tools below?
10298
103-
pdm, hatch and poetry all have "ways" of supporting c extensions via pdm-build, hatchling and poetry's build back end.
104-
poetry's support for C extensions is not fully developed and documented (yet). Poetry doesn't offer a way to facilitate "communication" between poetry front end and another back end like meson to build via a build hook.
99+
pdm, hatch and poetry all have "ways" of supporting c extensions via pdm-build, hatchling and poetry's build back-end.
100+
poetry's support for C extensions is not fully developed and documented (yet). Poetry doesn't offer a way to facilitate "communication" between poetry front end and another back-end like meson to build via a build hook.
105101
PDM and hatch both offer a plugin type approach to support custom build steps
106-
PDM (right now) is the only tool that supports other back ends (hatch is working on this - 2 minor releases away)
102+
PDM (right now) is the only tool that supports other back-ends (hatch is working on this - 2 minor releases away)
107103
At some point a build becomes so complex that you need to use a tool like scikit or meson to support that complexity.
108104
109105
CORRECTIONS:
@@ -125,10 +121,10 @@ build back-ends such as **meson** and **scikit-build**.
125121
126122
127123
me:
128-
pdm, hatch and poetry all have "ways" of supporting c extensions via pdm-build, hatchling and poetry's build back end.
129-
poetry's support for C extensions is not fully developed and documented (yet). Poetry doesn't offer a way to facilitate "communication" between poetry front end and another back end like meson to build via a build hook.
124+
pdm, hatch and poetry all have "ways" of supporting c extensions via pdm-build, hatchling and poetry's build back-end.
125+
poetry's support for C extensions is not fully developed and documented (yet). Poetry doesn't offer a way to facilitate "communication" between poetry front end and another back-end like meson to build via a build hook.
130126
PDM and hatch both offer a plugin type approach to support custom build steps
131-
PDM (right now) is the only tool that supports other back ends (hatch is working on this - 2 minor releases away)
127+
PDM (right now) is the only tool that supports other back-ends (hatch is working on this - 2 minor releases away)
132128
At some point a build becomes so complex that you need to use a tool like scikit or meson to support that complexity.
133129
@eli-schwartz eli-schwartz 3 weeks ago
134130
PDM and hatch both offer a plugin type approach to support custom build steps
@@ -141,7 +137,6 @@ pdm and poetry both rely on setuptools for C extensions. pdm's support claims to
141137
https://pdm.fming.dev/latest/pyproject/build/#build-platform-specific-wheels
142138
-->
143139

144-
145140
<!-- https://github.com/pyOpenSci/python-package-guide/pull/23#discussion_r1071541329
146141
ELI: A complex build could mean running a python script that processes some data file and produces a pure python module file.
147142

package-structure-code/pyproject-toml-python-package-metadata.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -30,15 +30,16 @@ resources working with complex builds in the future.
3030
```
3131

3232
### Example pyproject.toml for building using PDM
33+
3334
Below is an example build configuration for a Python project. This example
3435
package setup uses:
3536

36-
* **pdm.pep517.api** to build the [package's SDist and wheels](python-package-distribution-files-sdist-wheel)
37+
- **pdm.pep517.api** to build the [package's sdist and wheels](python-package-distribution-files-sdist-wheel)
3738

3839
```
3940
[build-system]
4041
requires = ["pdm-pep517>=1.0.0"]
41-
build-backend = "pdm.pep517.api"
42+
build-back-end = "pdm.pep517.api"
4243
4344
[project]
4445
name = "examplePy"
@@ -56,6 +57,7 @@ dependencies = [
5657
"dependency-package-name-2",
5758
]
5859
```
60+
5961
Notice that dependencies are specified in this file.
6062

6163
### Example pyproject.toml for building using setuptools
@@ -66,19 +68,19 @@ Notice how simple it is to swap out the tools needed to build this package!
6668

6769
In this example package setup you use:
6870

69-
* **setuptools** to build the [package's SDist and wheels](python-package-distribution-files-sdist-wheel)
70-
* **setuptools_scm** to manage package version updates using version control tags
71+
- **setuptools** to build the [package's sdist and wheels](python-package-distribution-files-sdist-wheel)
72+
- **setuptools_scm** to manage package version updates using version control tags
7173

7274
In the example below `[build-system]` is the first table
73-
of values. It has two keys that specify the build front end and backend for a package:
75+
of values. It has two keys that specify the build front end and back-end for a package:
7476

7577
1. `requires =`
76-
1. `build-backend =`
78+
1. `build-back-end =`
7779

7880
```
7981
[build-system]
8082
requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2"]
81-
build-backend = "setuptools.build_meta"
83+
build-back-end = "setuptools.build_meta"
8284
8385
[project]
8486
name = "examplePy"
@@ -97,8 +99,6 @@ dependencies = [
9799
]
98100
```
99101

100-
101-
102102
```{note}
103103
[Click here to read about our packaging build tools including PDM, setuptools, Poetry and Hatch.](/package-structure-code/python-package-build-tools)
104104
```

0 commit comments

Comments
 (0)