PEX documentation and Makefile is unclear on supported Python versions

[Cagrosso]

I am unable to build a PEX file as described in the documentation.

I am attempting to build the PEX file in a docker container for my version of Python (3.6). According to the Makefile, that version of Python is the default $PYVER. I come across multiple errors from the default $SPACY_EXTRAS and final packaging that point to Python 3.6 being incompatible.

My docker-compose.yml

version: '3.3'

services:
  build:
    image: python:3.6-bullseye
    volumes: 
      - ./spaCy:/spacy
    working_dir: /spacy
    command: tail -f /dev/null

The docker file exists in a directory with the SpaCy repo cloned into a folder spaCy ex:

- docker-compose.yml
- spaCy
| --- ... SpaCy repo root

After launching through a docker compose up -d I have to install the rust compiler following their instructions here.

Once the rust compiler is installed, run make in the working directory. This will result in the $SPACY_EXTRAS failing to build properly. For example, when attempting to build sudachipy, it fails with error: the configured Python interpreter version (3.6) is lower than PyO3's minimum supported version (3.7)

Even dropping the extras that I don't need still results in a failed build at the 'packaging' stage saying that Python3.6 is not supported.

If there is a different version of SpaCy that supports python3.6, or a different method of packaging, I would greatly appreciate being pointed in the correct direction.

Which page or section is this issue related to?

documentation source

[adrianeboyd]

The idea with the Makefile is that you can set PYVER and SPACY_EXTRAS to the versions/packages that you need by setting env variables (or by editing the Makefile).

A word of warning about pex on linux with torch>=1.12: the default pypi torch 1.12+ (with nvidia libraries) does not appear to work at all in the pex file and in any case, the final pex file with spacy requirements will go beyond pex's hard 2GB filesize limit.

For our own internal use (again, linux-only), we recently switched from pex to python-appimage. The setup is a little clunky because it assumes you need a desktop icon, etc., but otherwise I thought it was easy to get working. Put any models as requirements with github URLs into your requirements.txt.

[adrianeboyd]

Let me move this to the discussion board...