ENH: Answer the question - what's the difference between pip and conda and why does pip install . -e install a package that's not on pypi?

[lwasser]

This page begins to explain the difference but it doesn't answer the questions posed in the issue title above.

What is the difference between the two
How can pip install -e . install my package if it's not on pypi?

This may be related to the environments discussed in another issue.

cc @chenghlee

We might want to add this content on this page

And also it might be good to consider personas which would help a user find the things that they need to know ...

[lwasser]

the narrative idea of having personas and trying to present those stories with how to do the things that the persona probably needs to do...

[sneakers-the-rat]

i don't know anything about conda, so i would be bad at answering the first one aside from saying "conda manages system dependencies in addition to python dependencies"

I can def answer the second one. It looks like we already do actually: https://www.pyopensci.org/python-package-guide/tutorials/installable-code.html#step-5-install-your-package-locally

python-package-guide/tutorials/installable-code.md

Lines 531 to 551 in f31ba96

	:::{admonition} What does `python -m pip install -e .` do?
	:class: tip
	Let's break down `python -m pip install -e .`
	`python -m pip install -e .` installs your package into the current active
	Python environment in **editable mode** (`-e`). Installing your package in
	editable mode, allows you to work on your code and then test the updates
	interactively in your favorite Python interface. One important caveat of editable mode is that every time you update your code, you may need to restart Python.
	If you wish to install the package regularly (not in editable
	mode) you can use:
	- `python -m pip install . `
	**Using `python -m` when calling `pip`**
	Above, you use`python -m` to call the version of pip installed into your
	current active environment. `python -m` is important to ensure that you are
	calling the version of pip installed in your current environment.
	:::

the only thing we'd need to add there to answer this specific question re: on pypi/not on pypi is just that we are asking it to install "the current directory" when we ask to install ".". We can actually do that from any directory using any other directory, like pip install -e ~/Desktop/some_folder (and it's not specific to -e, pip install . also works). Our package has all the information that is needed to install it no matter where we get it from - pypi is just the thing that allows us to refer to it by name without specifying where it comes from (because pypi is the default place to look). When we instead use a package name, pip will assume we want something from pypi.

I'm not sure if it would be too much detail to add a link to the pip docs which explain this fully, and be like "you can also install stuff from URLs and git repos and stuff" with an example e.g. installing numpy directly from the URL to the .whl file

[ucodery]

Jonny's summary of the second point is excellent. This is exactly what I came here to say: don't make the . some magical target, it is simply a file path and pip works on all file paths (. happens to mean "the directory I am in" on the three most popular OSs). And -e doesn't alter how the user targets a package, only how it gets installed. Pip can even install directly from URLs, and supports a huge variety of protocols, not just http(s).

Also not a typical conda user, but I would say the bigger differences between pip and conda for the end-user developer are the scope of controlled packages (as Jonny said, conda goes beyond Python packages), the default index (PyPI vs anaconda), the expected isolation method (v[irtual]env vs conda) - I don't want to say default as pip is not an environment manager and that is another difference, and for advanced users and authors - the metadata specifications.

I don't know if conda has a concept of an editable install, but I'm sure it has a concept of a local install and in that way it is similar to pip install ..

[lwasser]

Ok - so the page I think that is throwing users off is this one . not everyone who reads the guide will read a tutorial so we can definitely cross link - but i remember now. This page has a very brief overfiew of pypi and then it talks about conda but skips pip.

I think it would be helpful to have an explanation at the top of the two package managers vs

So what is pip vs conda (maybe a table would be nice here)

• Conda can and is most often used to install python
• conda...

Pip is another package manager.

• pips default if you use pip install package will install from pypi
• pip can be also used to install and build a package locally pip install -e . etc...
• pip does not install python but it does come natively with a python installation via python.org, pyenv, uv, hatch etc
• EXTRA: pip can also "build" a package - creating a sdist and wheel

So i think a clear table explaining these two buckets of tools is important to clarify for users. Another thing that came up at pycon was environments and installing python. I think there is just general confusion in this area similar to packaging tools that we could excel at targeting and explaining! I know this isn't all packaging but because it all becomes blurred because for example pip is technically a packaging tool (backend) but also a envt mgr 🤯

i hope this helps clarify this issue a bit! i think it's a great and important one for us to tackle at some point. then we step into UV and pixi from there... we can do it. 🚀

[ucodery]

Pip is not an environment manager; it is a packaging tool, but it is not a build backend and should not be treated as a traditional build front end (though I guess it is technically).

pip can be also used to install and build a package locally

Just say "to install a package"

EXTRA: pip can also "build" a package - creating a sdist and wheel

Please don't put this in any guide or tutorial. Pip should not be a part of any regular build/release workflow of a package maintainer. It has been abused as such for many years, but this has largely harmed the packaging ecosystem, and is not a recommended practice.