feat: Improve Neovim detection of pynvim

Problem: Detection of the pynvim module is currently done by finding the
first Python interpreter in the `PATH` and checking if it can import
pynvim.  This has several effects:

- Activation of an unrelated Python virtual environment will break
  automatic detection, unless pynvim is also installed in that
  environment.

- Installing pynvim to the expected location is difficult. User
  installation into the system-wide or user-wide Python site area is now
  deprecated.  On Ubuntu 24.04 with Python 3.12, for example, the
  command `pip install --user pynvim` now fails with the error message
  `error: externally-managed-environment`.

- Users may create a dedicated virtual environment in which to install
  pynvim, but Neovim won't detect it; instead, they must either activate
  it before launching Neovim (which interferes with the user of other
  virtual environments) or else hard-code the variable
  `g:python3_host_prog` in their `init.vim` to the path of the correct
  Python interpreter.  Neither option is desirable.

Solution: Expose pynvim's Python interpreter on the `PATH` under the
name `pynvim-python`.  In the typical flow:

- User installs either uv or pipx.

- User installs pynvim via:

      uv tool install --upgrade pynvim
      # Or:
      pipx install --upgrade pynvim

With corresponding changes in Neovim, the above is all that's needed for
Neovim to detect the installed location of pynvim, even if an unrelated
Python virtual environments is activated.  It uses standard Python
tooling to automate the necessary creation of a Python virtual
environment for pyenv and the publication of `pynvim-python` to a
directory on `PATH`.

See #593 for additional
discussion of this idea.

Author: drmikehenry

README.md

@@ -12,17 +12,35 @@ Install
 
 Supports python 3.7 or later.
 
-    pip3 install pynvim
+- Installation option #1: install using uv (recommended):
 
-You can install the package without being root by adding the `--user` flag.
-Anytime you upgrade Neovim, make sure to upgrade pynvim as well:
+  - Install uv (https://docs.astral.sh/uv/).
 
-    pip3 install --upgrade pynvim
+  - Install pynvim (the `--upgrade` switch ensures installation of the latest
+    version):
 
-Alternatively, you can install the development version by cloning this
-repository and executing the following at the top level:
+        uv tool install --upgrade pynvim
 
-    pip3 install .
+  - Anytime you upgrade Neovim, make sure to upgrade pynvim as well by
+    re-running the above command.
+
+- Installation option #2: install using pipx:
+
+  - Install pipx (https://pipx.pypa.io/stable/).
+
+  - Install pynvim (the `--upgrade` switch ensures installation of the latest
+    version):
+
+        pipx install --upgrade pynvim
+
+  - Anytime you upgrade Neovim, make sure to upgrade pynvim as well by
+    re-running the above command.
+
+- Other installation options:
+
+  - See [pynvim installation
+    documentation](https://pynvim.readthedocs.io/en/latest/installation.html)
+    for additional installation options and information.
 
 Python Plugin API
 -----------------

docs/installation.rst

@@ -3,17 +3,63 @@ Installation
 
 The Neovim Python client supports Python 3.7 or later.
 
-Using pip
----------
+Using uv or pipx
+----------------
 
-You can install the package without being root by adding the ``--user`` flag::
+For automatic detection by Neovim, pynvim should be installed in a dedicated
+Python virtual environment and the ``pynvim-python`` executable should be placed
+on the ``PATH``.  The recommended approach for this is to use a tool like uv
+(https://docs.astral.sh/uv/) or pipx (https://pipx.pypa.io/stable/); the
+``--upgrade`` switch ensures installation of the latest version:
 
-    pip3 install --user pynvim
+- Install using uv (recommended)::
 
-If you follow Neovim HEAD, make sure to upgrade ``pynvim`` when you upgrade
-Neovim::
+    uv tool install --upgrade pynvim
 
-    pip3 install --upgrade pynvim
+- Install using pipx::
+
+    pipx install --upgrade pynvim
+
+**NOTE** For Neovim before v0.12.0, set the variable ``python3_host_prog`` in
+``init.vim`` to point to ``pynvim-python``::
+
+    let g:python3_host_prog = 'pynvim-python'
+
+Using manually created Python virtual environment
+-------------------------------------------------
+
+Alternatively, you may manually create a Python virtual environment
+(https://docs.python.org/3.13/library/venv.html)::
+
+    python3 -m venv pynvim-venv
+
+Then install pynvim into the virtual environment; the
+``--upgrade`` switch ensures installation of the latest version::
+
+- For Unix::
+
+    pynvim-venv/bin/python -m pip install --upgrade pynvim
+
+- For Windows::
+
+    pynvim-venv\Scripts\python -m pip install --upgrade pynvim
+
+Then copy the ``pynvim-python`` executable somewhere on the ``PATH``:
+
+- For Unix::
+
+    # Assuming `~/.local/bin` is on `PATH`:
+    cp pynvim-venv/bin/pynvim-python ~/.local/bin/pynvim-python
+
+- For Windows::
+
+    REM Assuming `C:\apps` is on `PATH`:
+    copy pynvim-venv\Scripts\pynvim-python.exe C:\apps\pynvim-python.exe
+
+**NOTE** For Neovim before v0.12.0, set the variable ``python3_host_prog`` in
+``init.vim`` to point to ``pynvim-python``::
+
+    let g:python3_host_prog = 'pynvim-python'
 
 Install from source
 -------------------
@@ -23,6 +69,70 @@ Clone the repository somewhere on your disk and enter to the repository::
     git clone https://github.com/neovim/pynvim.git
     cd pynvim
 
-Now you can install it on your system::
+Now you can install it following the instructions above, using ``.`` instead of
+``pynvim``; the ``--upgrade`` switch ensures installation of the latest version:
+
+- Install from source using uv::
+
+    uv tool install --upgrade .
+
+- Install from source using pipx::
+
+    pipx install --upgrade .
+
+- Install from source using manually created Python virtual environment:
+
+  - Create ``pynvim-venv`` as above.
+
+  - Install:
+
+    - For Unix::
+
+        pynvim-venv/bin/python -m pip install --upgrade .
+
+    - For Windows::
+
+        pynvim-venv\Scripts\python -m pip install --upgrade .
+
+  - Copy ``pynvim-python`` executable as above.
+
+**NOTE** For Neovim before v0.12.0, set the variable ``python3_host_prog`` in
+``init.vim`` to point to ``pynvim-python``::
+
+    let g:python3_host_prog = 'pynvim-python'
+
+Upgrade pynvim when upgrading Neovim
+------------------------------------
+
+Make sure to upgrade ``pynvim`` when you upgrade Neovim.  Follow the previous
+instructions; the ``--upgrade`` switch will ensure installation of the latest
+version.
+
+Explicitly choosing pynvim virtual environment
+----------------------------------------------
+
+As an alternative to exposing ``pynvim-python`` on ``PATH``, you may configure
+Neovim to use a specific Python interpreter that has pynvim installed; this may
+be useful when working on pynvim itself.
+
+After installing into a virtual environment named ``pynvim-venv``, add the
+following into Neovim's ``init.vim`` file:
+
+- For Unix::
+
+      let g:python3_host_prog = '/path/to/pynvim-venv/bin/python'
+
+- For Windows::
+
+      let g:python3_host_prog = 'c:\path\to\pynvim-venv\bin\python.exe'
+
+Installing outside of a virtual environment is deprecated
+---------------------------------------------------------
+
+Installing into the per-user Python site package area is a deprecated practice
+with recent Python versions.  For example, the following command fails on Ubuntu
+24.04 with the error message ``error: externally-managed-environment``::
+
+    pip install --user pynvim
 
-    pip3 install .
+Instead, always install into a virtual environment.

pynvim/python.py

@@ -0,0 +1,6 @@
+import subprocess
+import sys
+
+
+def main() -> None:
+    subprocess.run([sys.executable] + sys.argv[1:])

setup.py

@@ -58,4 +58,9 @@
       setup_requires=setup_requires,
       tests_require=tests_require,
       extras_require=extras_require,
+      entry_points={
+              'console_scripts': [
+                  'pynvim-python=pynvim.python:main',
+              ],
+          },
       )