Skip to content

Commit 3ca9010

Browse files
committed
Fix: finished draft 1 of this lesson - phew
1 parent 0d65afb commit 3ca9010

File tree

2 files changed

+159
-21
lines changed

2 files changed

+159
-21
lines changed
112 KB
Loading

tutorials/publish-conda-forge.md

Lines changed: 159 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ Conda channels represent various repositories that you can install packages from
6161

6262
## Why publish to conda forge
6363

64-
There are many users, especially in the scientific ecosystem that use conda as their primary package manager / environment tool. Thus, having packages available to these users on the conda-forge channel is useful. In some cases packages on conda-forge can minimize dependency conflicts that can occur when mixing installations using pip and conda. This is particularly important for the spatial ecosystem.
64+
There are many users, especially in the scientific Python ecosystem that use conda as their primary package manager / environment tool. Thus, having packages available to these users on the conda-forge channel is useful. In some cases packages on conda-forge can minimize dependency conflicts that can occur when mixing installations using pip and conda. This is particularly important for the spatial ecosystem.
6565

6666
## How publishing to conda-forge works
6767

@@ -76,11 +76,24 @@ Conda-forge will build your package from the source distribution which you [publ
7676

7777
### Conda-forge publication steps
7878

79-
The high steps to publish to conda-forge are you will walk through below are:
79+
The steps to publish to conda-forge are:
8080

8181
1. Publish your Python package distribution files (sdist & wheel) to PyPI
8282
2. Create a conda-forge recipe, which is a yaml file with instructions on how to build your package on conda-forge, using the grayskull[^grayskull] package.
83-
3. Submit the recipe (yaml file) to the conda-forge staged recipes repository as a pull request for review.
83+
3. Submit the recipe (yaml file) to the conda-forge staged recipes repository as a pull request for review. [Click here for an example submission from pyOpenSci.](https://github.com/conda-forge/staged-recipes/pull/25173)
84+
85+
:::{figure-md} publish-pypi-conda-forge
86+
<img src="../images/pyopensci-python-package-pypi-to-conda-forge.png" alt="." width="700px">
87+
88+
Caption
89+
:::
90+
91+
:::{}
92+
93+
94+
95+
:::
96+
8497

8598
:::{todo}
8699
Here is an example of what a pull request for a conda-forge recipe looks like.
@@ -154,7 +167,7 @@ Executing injected distutils...
154167
Checking >> -- 100% |# |[Elapsed Time: 0:00:00]
155168
Matching license file with database from Grayskull...
156169
Match percentage of the license is 97%. Low match percentage could mean that the license was modified.
157-
License type: BSD-3-Clause
170+
License type: MIT
158171
License file: ['LICENSE']
159172
Build requirements:
160173
<none>
@@ -187,7 +200,7 @@ This will create a meta.yaml file that looks like the example below:
187200

188201
```yaml
189202
{% set name = "pyospackage" %}
190-
{% set version = "0.0.1" %}
203+
{% set version = "0.1.8" %}
191204

192205
package:
193206
name: {{ name|lower }}
@@ -219,7 +232,7 @@ test:
219232
- pip
220233

221234
about:
222-
license: BSD-3-Clause
235+
license: MIT
223236
license_file: LICENSE
224237

225238
extra:
@@ -243,40 +256,165 @@ test:
243256
requires:
244257
- pip
245258
```
259+
246260
### Step 4: Submit a pull request to the staged-recipes repository
247261
248262
Finally, create a pull request in the staged-recipes GitHub repository.
249263
250-
When you do this, a suite of CI actions will run that build and test the build of your package. A conda-forge maintainer will work with you to get your recipe in good shape and merged.
251-
252-
You can follow the [instructions here](https://conda-forge.org/docs/maintainer/adding_pkgs.html) to submit your package
253-
254264
To create your pull request:
255265
256266
1. Fork and clone this repo: https://github.com/conda-forge/staged-recipes
257267
1. Create a branch in your fork rather than submitting from the main branch of your fork.
258-
1. Within your fork's branch, create a new directory with the name of your package, and add your meta.yaml file to it.
268+
1. Within your fork's branch, create a new directory with the name of your package,
269+
1. Add the meta.yaml file that you generated using grayskull to that directory.
259270
260-
`staged-recipes/recipes/<name-of-package>/meta.yaml`
271+
`staged-recipes/recipes/pyospackage/meta.yaml`
261272

262-
1. Submit the pull request
273+
1. Submit a pull request from your fork/branch of the staged-recipes repository.
263274
1. Remember that the conda-forge maintainers are volunteers. Be patient for someone to respond and supportive in your communication with them.
264275

276+
When you do this, a suite of CI actions will run that build and test the build of your package. A conda-forge maintainer will work with you to get your recipe in good shape and merged.
277+
278+
You can follow the [instructions here](https://conda-forge.org/docs/maintainer/adding_pkgs.html) to submit your package
279+
280+
281+
## Conda-forge pull request checklist
282+
283+
When you submit your package to conda forge, the pull request template includes a list of checks that you want to ensure you have covered.
284+
285+
Below we break down each element of that list.
286+
287+
288+
## Conda-forge Staged-recipes Pull Request Checklist
289+
290+
When you submit a pull request for your package's recipe to conda-forge you will notice a checklist of items that you need to ensure you have covered.
291+
292+
A brief explanation of each is below.
293+
294+
::::{admonition} Pull request template checklist tips
295+
:class: note
296+
297+
> -[x] Title of this PR is meaningful: e.g. "Adding my_nifty_package", not "updated meta.yaml".
298+
299+
**Translation:** Make sure that your pull request title is specific. We suggest something like:
300+
`Add: conda-forge recipe for <your package name>`
301+
302+
> -[x] License file is packaged (see [here](https://github.com/conda-forge/staged-recipes/blob/5eddbd7fc9d1502169089da06c3688d9759be978/recipes/example/meta.yaml#L64-L73) for an example).
303+
304+
**Translation:** You should have a LICENSE file included in your package's source distribution. If you have followed the pyOpenSci tutorials then you already have a LICENSE file and are likely using the MIT license. When you run `hatch build`, it will bundle that file into the output [source distribution file (which is the tar.gz file)](python-source-distribution) that conda-forge will use to build your package.
305+
306+
> - [x] Source is from official source.
307+
308+
**Translation:** If your package is on PyPI as you learned in the [previous lesson on publishing your Python package](publish-pypi) then you are in good shape. conda-forge prefers that your distribution is published to a known repository.
309+
310+
> -[x] Package does not vendor other packages. (If a package uses the source of another package, they should be separate packages or the licenses of all packages need to be packaged).
311+
312+
**Translation:** If the code base in your package is your own and it all shares the same LICENSE then you are in good shape. If you have code taken from other packages then you may need to declare that and include licenses for that code if it is different. If you followed these tutorials then you do not have any vendored code.
313+
314+
315+
> -[x] If static libraries are linked in, the license of the static library is packaged.
316+
>
317+
> -[x] Package does not ship static libraries. If static libraries are needed, [follow CFEP-18](https://github.com/conda-forge/cfep/blob/main/cfep-18.md).
318+
319+
320+
321+
:::{todo}
322+
From leah - Can someone validate that this description is accurate or suggest alternative language please? i don't fully understand static library links
323+
:::
324+
325+
**Translation:** The pyOpenSci tutorials did not include any static libraries in a linked or shipped format. If your package is pure Python you likely don't need to worry about this. However, some packages with more complex builds may include links to extensions written in other languages such as C++. If your package does have such extensions you will need to include the LICENSE(s) for those extensions.
326+
327+
> -[ ] Build number is 0.
328+
329+
**Translation:** The build number in your recipe is right below the source location of your package's source distribution. `number: 0` is what you should see in that section of your recipe.
330+
331+
```bash
332+
source:
333+
url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/pyospackage-{{ version }}.tar.gz
334+
sha256: 01e31f5521973710d0d91b15a94491d4f8f8f54566322110098c0f2381dd09ab
335+
336+
build:
337+
noarch: python
338+
script: {{ PYTHON }} -m pip install . -vv --no-deps --no-build-isolation
339+
number: 0
340+
```
341+
342+
> - [x] A tarball (`url`) rather than a repo (e.g. `git_url`) is used in your recipe (see [here](https://conda-forge.org/docs/maintainer/adding_pkgs.html#build-from-tarballs-not-repos) for more details).
343+
344+
**Translation:** Here conda wants you to provide a link to the source distribution on PyPI rather than a link to your GitHub repository distribution. Notice above in the Source section of your recipe there is a `url:` section that provides a PyPI url that ends in tar.gz. That is a link to your source distribution that conda-forge will use.
265345

266346
```yaml
267-
test:
268-
imports:
269-
- pyospackage
270-
commands:
271-
- pip check
272-
requires:
273-
- pip
347+
url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/pyospackage-{{ version }}.tar.gz
274348
```
349+
> - [x] GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there.
350+
351+
**Translation** Once you have submitted your recipe, be sure that all maintainers listed in your recipe respond acknowledging that they are ok with being listed as a maintainer for the conda-forge version of your package.
352+
353+
> - [x] When in trouble, please check our [knowledge base documentation](https://conda-forge.org/docs/maintainer/knowledge_base.html) before pinging a team.
354+
355+
**Translation** The conda team are volunteers who spend their time supporting our community. Please try to troubleshoot on your own first before tagging one of them for help.
356+
357+
This is also why we don't suggest you publish to conda-forge as a practice run.
358+
::::
275359

276-
## Maintaining
360+
Once you have submitted your recipe, you can wait for the CI build to pass. If it's not passing, and you aren't sure why, a conda-forge maintainer can likely help you figure things out.
361+
362+
Once your recipe is built and merged, the conda team will create a new package repository for you similar to [this one for the GemGIS package](https://github.com/conda-forge/gemgis-feedstock).
363+
364+
### <i class="fa-solid fa-wand-magic-sparkles"></i> Congratulations - you have added your package to conda-forge.<i class="fa-solid fa-wand-magic-sparkles"></i>
365+
366+
The last part of this process is maintaining the repository.
367+
We cover that next.
368+
369+
## Maintaining your conda-forge feedstock
370+
371+
Every time you create a new release on PyPI, the conda-forge bots will recognize the release and will rebuild the newly released version of your package. This process may take a day or two to complete so be patient.
372+
373+
Once the conda-forge build it complete, all of the maintainers of your conda-forge feedstock will get a ping on GitHub that a new pull request has been opened.
374+
375+
Review the pull request. If all tests are passing, you can merge it. within the next day the conda-forge release will be available for users to install:
376+
377+
`conda install -c conda-forge yourpackage`
378+
379+
:::{todo}
380+
pr for our tutorial package is up as a draft. right now i'm the only maintainer.
277381

278382
<Create a pr on conda-forge - tag filipe (as it is a test package) and add screenshots here so people understand what maintaining a conda recipe entails >
279383

384+
385+
Questions:
386+
387+
Pandas has this for their recipe -
388+
about:
389+
home: http://pandas.pydata.org
390+
license: BSD-3-Clause
391+
license_file: LICENSE
392+
summary: >-
393+
Powerful data structures for data analysis, time series, and statistics
394+
doc_url: https://pandas.pydata.org/docs/
395+
dev_url: https://github.com/pandas-dev/pandas
396+
397+
their pyprojec.toml is like This:
398+
[project.urls]
399+
homepage = 'https://pandas.pydata.org'
400+
documentation = 'https://pandas.pydata.org/docs/'
401+
repository = 'https://github.com/pandas-dev/pandas'
402+
403+
Should we have users add a homepage or can they add a homepage at a later time??
404+
:::
405+
406+
## <i class="fa-solid fa-hands-bubbles"></i> Wrap up
407+
408+
If you have walked through this entire tutorial series you have now:
409+
410+
1. Understand [what a Python package is ](intro.md)
411+
2. Know how to [make your code installable](1-installable-code.md) into Python environments
412+
3. Know how to create a pyproject.toml file, a README file, and a License and code of conduct.
413+
4. Know how to [publish your package to PyPI](publish-pypi.md) and
414+
5. Know how to publish your package to conda-forge
415+
416+
The above are the basic steps that you need to take to create and publish a Python package. In a future tutorial series we will cover that basics of maintaining your package.
417+
280418
## Footnotes
281419

282420
[^grayskull]: [Grayskull blogpost](https://conda-forge.org/blog/2020/03/05/grayskull/)

0 commit comments

Comments
 (0)