Skip to content

Commit 0d65afb

Browse files
committed
Fix: cross link across pages
1 parent e2047ac commit 0d65afb

File tree

6 files changed

+350
-46
lines changed

6 files changed

+350
-46
lines changed
88.4 KB
Loading

images/python-pypi-conda-channels.png

92.3 KB
Loading

package-structure-code/publish-python-package-pypi-conda.md

Lines changed: 46 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -9,14 +9,22 @@ from a public community repository such as PyPI or a conda channel such as
99
Below you will learn more about the various publishing options for your Python
1010
package.
1111

12-
```{admonition} Take Aways
12+
:::{admonition} Take Aways
1313

1414
* Installing packages in the same environment using both pip and conda can
1515
lead to package conflicts.
1616
* To minimize conflicts for users who may be using conda (or pip) to manage local environments, consider publishing your package to both PyPI and the conda-forge channel of the Anaconda Cloud.
1717

1818
Below you will learn more specifics about the differences between PyPI and conda publishing of your Python package.
19-
```
19+
:::
20+
21+
22+
:::{figure-md} pypi-conda-channels
23+
24+
<img src="../images/publish-python-package-pypi-conda.png" alt="Image showing the progression of creating a Python package, building it and then publishing to PyPI and conda-forge. You take your code and turn it into distribution files (sdist and wheel) that PyPI accepts. then there is an arrow towards the PyPI repository where ou publish both distributions. From PyPI if you create a conda-forge recipe you can then publish to conda-forge. " width="700px">
25+
26+
Once you have published both package distributions (the source distribution and the wheel) to PyPI, you can then publish to conda-forge. Conda forge requires an source distribution on PyPI in order to build your package on conda-forge. You do not need to rebuild your package to publish to conda-forge.
27+
:::
2028

2129
## What is PyPI
2230

@@ -32,6 +40,14 @@ a package is that conda can install any package regardless
3240
of the language(s) that it is written in. Whereas `pip` can
3341
only install Python packages.
3442

43+
:::{button-link} ../tutorials/publish-pypi.html
44+
:color: primary
45+
:class: sd-rounded-pill float-left
46+
47+
Click here for a tutorial on publishing your package to PyPI.
48+
:::
49+
50+
3551
```{tip}
3652
On the package build page, we discussed the [two package distribution
3753
types that you will create when making a Python package](python-package-distribution-files-sdist-wheel): SDist (packaged as a .tar.gz or .zip) and
@@ -40,6 +56,7 @@ be published on PyPI when you use [a standard build tool](python-package-build-t
4056
your package.
4157
```
4258

59+
(about-conda)=
4360
## What is Anaconda Cloud and conda?
4461

4562
conda is an open source package and environment management tool.
@@ -49,20 +66,26 @@ repository](https://repo.anaconda.com/).
4966
Anaconda cloud (anaconda.org) contains public and private repositories for
5067
packages. These repositories are known as channels (discussed below).
5168

52-
The most common public channels are:
69+
:::{admonition} A brief history of conda's evolution
70+
:class: note
5371

54-
- defaults
55-
- conda-forge and
56-
- bioconda
72+
The conda ecosystem evolved years ago to provide support for and
73+
simplify the process of managing software dependencies in scientific
74+
Python projects.
5775

58-
```{tip}
76+
Many of the core scientific Python projects depend upon or wrap around tools and extensions that are written in other languages such as C++. In the early stages of the scientific ecosystem's development, these non python extensions and tools were not well supported on PyPI making publication difficult. In recent years there is more support for complex builds that allow developers to bundle non Python code into a python distribution using the [wheel distribution format](python-wheel).
77+
78+
Conda provides a mechanism to manage these dependencies and ensure that the required packages are installed correctly.
79+
:::
80+
81+
:::{tip}
5982
While conda was originally created to support Python packages, it
6083
is now used across all languages. This cross-language support
6184
makes it easier for some packages to include and have access to
6285
tools written in other languages such as c/c++ (gdal), Julia, or R.
6386
Creating environment that mixes all those packages are usually easier and more
6487
consistent with full fledged package managers like conda.
65-
```
88+
:::
6689

6790
### conda channels
6891

@@ -72,20 +95,21 @@ channels. The conda package manager can install packages from different channels
7295
There are several core public channels that most people use to install
7396
packages using conda including.
7497

75-
- **defaults:** this is a channel managed by the Anaconda, inc. It is the version of the Python packages that you will install if you install the Anaconda Distribution.
76-
- [**conda-forge:**](https://anaconda.org/conda-forge) this is a community driven channel that focuses on scientific packages. This channel is ideal for tools that support geospatial data
77-
- [**bioconda**](https://anaconda.org/bioconda): this channel focuses on biomedical
98+
- **defaults:** this is a channel managed by the Anaconda, inc. It is the version of the Python packages that you will install if you install the Anaconda Distribution. Anaconda decides what packages live on the default channel.
99+
- [**conda-forge:**](https://anaconda.org/conda-forge) this is a community driven channel that focuses on scientific packages. This channel is ideal for tools that support geospatial data. Anyone can publish a package to this channel.
100+
- [**bioconda**](https://anaconda.org/bioconda): this channel focuses on biomedical tools.
78101

79102
**conda-forge** emerged as many of the scientific packages did not
80103
exist in the default Anaconda cloud channel.
81104

82-
:::{figure-md} conda-channels
105+
:::{figure-md} pypi-conda-channels
83106

84-
<img src="../images/conda-channels-geohackweek.jpeg" alt="ADD." width="700px">
107+
<img src="../images/python-pypi-conda-channels.png" alt="Graphic with the title Python package repositories. Below it says Anything hosted on PyPI can be installed using pip install. Packaging hosted on a conda channel can be installed using conda install. Below that there are two rows. the top row says conda channels. next to it are three boxes one with conda-forge, community maintained; bioconda and then default - managed by the anaconda team. Below that there is a row that says PyPI servers. PyPI - anyone can publish to pypi. and test pypi. a testbed server for you to practice. " width="700px">
85108

86-
ADD source: GeoHackWeek
109+
Conda channels represent various repositories that you can install packages from. Because conda-forge is community maintained, anyone can submit a recipe there. PiPY is also a community maintained repository. Anyone can submit a package to PyPI and test PyPI. Unlike conda-forge there are no manual checks of packages submitted to PyPI.
87110
:::
88111

112+
89113
## conda channels, PyPI, conda, pip - Where to publish your package
90114

91115
You might be wondering why there are different package repositories
@@ -147,16 +171,16 @@ Once your package is on PyPI, the process to add your package to conda-forge
147171
is straight forward to do. [You can follow the detailed steps provided
148172
by the conda-forge maintainer team.](https://conda-forge.org/docs/maintainer/adding_pkgs.html#generating-the-recipe).
149173

150-
Generally those steps are:
151174

152-
1. Fork the staged recipes conda-forge GitHub repository
153-
1. Create a new recipe using the `grayskull` tool. You can [learn more about grayskull here](https://conda.github.io/grayskull/).
154-
1. Add the recipe file created by `grayskull` in a folder within the recipes/ directory of the [**conda-forge/staged-recipes**](https://github.com/conda-forge/staged-recipes) repository.
155-
1. Open a pull request with your change. Your package will be tested on Windows, Mac and Linux using the repository CI.
175+
:::{button-link} ../tutorials/publish-conda-forge.html
176+
:color: primary
177+
:class: sd-rounded-pill float-left
178+
179+
Click here for a tutorial on adding your package to conda-forge.
180+
:::
181+
182+
If you want a step by step tutorial, click here.
156183

157-
Once you have submitted a recipe to conda-forge, a maintainer will work
158-
with you to iron out any bugs in your recipe. They will then add your package
159-
to the conda-forge channel.
160184

161185
Once your package is added, you will have a feedstock repository on GitHub with your packages name
162186

package-structure-code/python-package-distribution-files-sdist-wheel.md

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,24 @@
11
# Learn about Building a Python Package
22

3-
:::{figure-md} build_workflow
4-
<img src="../images/python-package-development-process.png" alt="Graphic showing the high level packaging workflow. On the left you see a graphic with code, metadata and tests in it. those items all go into your package. Documentation and data are below that box because they aren't normally published in your packaging wheel distribution. an arrow to the right takes you to a build distribution files box. that box leads you to either publishing to testpypi or the real pypi. from pypi you can then connect to conda forge for an automated build that sends distributions from pypi to conda-forge. " width="700px">
3+
4+
:::{figure-md} pypi-conda-channels
5+
6+
<img src="../images/publish-python-package-pypi-conda.png" alt="Image showing the progression of creating a Python package, building it and then publishing to PyPI and conda-forge. You take your code and turn it into distribution files (sdist and wheel) that PyPI accepts. then there is an arrow towards the PyPI repository where ou publish both distributions. From PyPI if you create a conda-forge recipe you can then publish to conda-forge. " width="700px">
7+
8+
Once you have published both package distributions (the source distribution and the wheel) to PyPI, you can then publish to conda-forge. Conda forge requires an source distribution on PyPI in order to build your package on conda-forge. You do not need to rebuild your package to publish to conda-forge.
9+
:::
510

611
You need to build your Python package in order to publish it to PyPI (or Conda). The build process organizes your code and metadata into a distribution format that can be uploaded to PyPI and subsequently downloaded and installed by users. NOTE: you need to publish a sdist to PyPI in order for conda-forge to properly build your package automatically.
712
:::
813

914
(build-package)=
1015
## What is building a Python package?
1116

12-
To [publish your Python package](build_workflow) and make it easy for anyone to install, you first need to build it.
17+
To [publish your Python package](publish-python-package-pypi-conda) and make it easy for anyone to install, you first need to build it.
1318

1419
But, what does it mean to build a Python package?
1520

16-
[As shown in the figure above](build_workflow), when you build your Python package, you convert the source files into something called a distribution package. A distribution package contains your source code and metadata about the package, in the format required by the Python Package Index, so that it can be installed by tools like pip.
21+
[As shown in the figure above](#pypi-conda-channels), when you build your Python package, you convert the source files into something called a distribution package. A distribution package contains your source code and metadata about the package, in the format required by the Python Package Index, so that it can be installed by tools like pip.
1722

1823
:::{note}
1924
The term package used to mean many different things in Python and other languages. On this page, we adapt the convention of the [Python Packaging Authority](https://www.pypa.io/en/latest/) and refer to the product of the

tutorials/intro.md

Lines changed: 13 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,16 @@
11
# Python packaging 101
22
*A start to finish beginner-friendly tutorial*
33

4+
:::{toctree}
5+
:hidden:
6+
:caption: Python Packaging 101
7+
8+
What is a Python package? <self>
9+
Make your code installable <1-installable-code>
10+
Publish to PyPI <publish-pypi>
11+
Publish to conda-forge <publish-conda-forge>
12+
:::
13+
414
Welcome to the pyOpenSci Python packaging tutorial series. The lessons
515
on the upcoming pages walk you through the core steps needed to
616
create a Python package.
@@ -24,14 +34,7 @@ understanding the steps involved in creating a Python package.
2434
* In the second series, you will learn about infrastructure and documentation needed to support package maintenance.
2535

2636

27-
:::{toctree}
28-
:hidden:
29-
:caption: Python Packaging 101
3037

31-
What is a Python package? <self>
32-
Make your code installable <1-installable-code>
33-
Publish to PyPI <publish-pypi>
34-
:::
3538

3639
:::{admonition} Learning Objectives
3740

@@ -291,23 +294,13 @@ If you want to make your package directly installable without having
291294
to download the code to your computer locally then you need to
292295
publish it in a repository such as **PyPI** or **conda-forge**.
293296

294-
:::{todo}
295-
The links below won't work until those lessons (which are written) are published.
296-
297-
Learn [how to publish your package to PyPI in this tutorial.](6-publish-pypi.md)
298-
:::
299-
297+
Learn [how to publish your package to PyPI in this tutorial.](publish-pypi.md)
300298

301-
:::{todo}
302-
This lesson is also not published - yet but will be reviewed soon.
303-
:::
304299

305300
Then you can create a conda-forge recipe using the [Grayskull](https://github.com/conda/grayskull) tool. You can then submit this recipe to conda-forge.
306301

307-
:::{todo}
308-
remove this todo when this page is published
309-
[You will learn more about the conda-forge publication process here.](7-publish-conda-forge.md)
310-
:::
302+
[You will learn more about the conda-forge publication process here.](publish-conda-forge.md)
303+
311304

312305
:::{figure-md} build-workflow-tutorial
313306
<img src="../images/tutorials/publish-package-pypi-conda.png" alt="Graphic showing the high level packaging workflow. On the left you see a graphic with code, metadata and tests in it. Those items all go into your package. Documentation and data are below that box because they aren't normally published in your packaging wheel distribution. an arrow to the right takes you to a build distribution files box. that box leads you to either publishing to testPyPI or the real PyPI. From PyPI you can then connect to conda forge for an automated build that sends distributions from PyPI to conda-forge." width="700px">

0 commit comments

Comments
 (0)