All Requirements Being Shown as Extra Requirements

[seankfh]

In the past this package worked as expected for me, but now when I use it all requirements seem to be displayed as missing requirements. Was there a breaking change or am I making an obvious mistake?

~/projects/test$ ls
env  requirements.txt  src

~/projects/test$ cat requirements.txt
numpy==1.24.2

~/projects/test$ cat src/main.py
import numpy as np
a = np.arange(15).reshape(3, 5)
print(a)

~/projects/test$ pip-extra-reqs src
Extra requirements:
numpy in requirements.txt

[adamtheturtle]

Is numpy installed in the Python environment you are running this in?

[seankfh]

Yes. This is the simplest example I could come up with for the sake of discussion, but for me the package behaved similarly in two other projects with a different directory structure and dependencies.

~/projects/test$ pip freeze
numpy==1.24.2
packaging==23.0
pip-check-reqs==2.4.4
pkg_resources==0.0.0

[adamtheturtle]

Thank you. I'm not seeing this - however my reproduction does not include the env file / directory you have as I do not know the contents.

What version of pip-check-reqs are you using? (please make sure it is the latest version).

[seankfh]

Interesting. Thanks for trying to help! I don't know what is happening.

You can see from the pip freeze above it is 2.4.4.

The env was created with python3 -m venv env and contains the standard contents:

~/projects/test$ ls env
bin  include  lib  lib64  pyvenv.cfg  share

[adamtheturtle]

Ok - I can reproduce if I create the env.
Can you try moving env somewhere else - that seems to work for me.

[seankfh]

Hm I'm not sure what you mean somewhere else. I tried moving it into src and had the same results:

~/projects/test$ python3 -m venv src/env
~/projects/test$ source src/env/bin/activate
~/projects/test$ pip install -r requirements.txt 
~/projects/test$ pip-extra-reqs src
Extra requirements:
numpy in requirements.txt

You mean the root / outer projects directory? I need each sub-directory to have its own virtual environment. I guess that is possible to do but a different workflow than I am used to.

[seankfh]

Okay yes I can see that working:

~/projects$ python3 -m venv test-env
~/projects$ cd test
~/projects/test$ source ../test-env/bin/activate
~/projects/test$ cat requirements.txt 
numpy==1.24.2
pytest==7.2.1
~/projects/test$ pip install -r requirements.txt
~/projects/test$ pip-extra-reqs src
Extra requirements:
pytest in requirements.txt

[MrMino]

Debug logs show something interesting:

...
installed package: numpy (at /tmp/env/lib/python3.9/site-packages)
used module: numpy (from file /tmp/env/lib/python3.9/site-packages/numpy, assuming stdlib or local)

The branching that does the "assuming stdlib or local" is this:

if info.filename in installed_files:

Where info.filename is

/tmp/env/lib/python3.9/site-packages/numpy

What we are adding to installed_files (and this is the buggy part) is this:

'/tmp/env/lib/python3.9/site-packages/env/lib/python3.9/site-packages/numpy': 'numpy',

[borutzki]

Came across something similar. paramiko was marked as an extra requirements, whereas it's used in code.

Similarly, although I can see this print:
installed package: paramiko (at c:\dev\virtualenvs\rexio_multiuser\lib\site-packages)

I can also see this one:
used module: paramiko (from file C:\dev\virtualenvs\rexio_multiuser\Lib\site-packages\paramiko, assuming stdlib or local)

and hence this error:

Extra requirements:
paramiko in requirements\base.txt

Please note that lib vs. Lib difference between prints.

pip-extra-reqs v 2.4.4 on Python 3.11.3.

[adamtheturtle]

Great @borutzki ! I think that your note of the case is pertinent.

Corroborating evidence:

• NumPy is not all lowercase in PyPI
• I've seen this with Werkzeug (also not lowercase)

I think somewhere we need case normalisation

[adamtheturtle]

I expect that this may be the same issue as #12