BUG: ModuleNotFoundError: No module named '_glmnet'

[amirhessam88]

Hello,
I have forked the repo to work on a new feature. Based on the contributing documents, I started to see if I can run pytest on tests files, which I could not. Apparently, _glmnet module is missing. I tried to load the modules, and I failed as well. I am wondering if you can help me. I get the following error.

      1 import pkg_resources
      2 
----> 3 from .logistic import LogitNet
      4 from .linear import ElasticNet
      5 

~/Documents/GitHub/GLM-Net/glmnet/logistic.py in <module>
     12 
     13 from .errors import _check_error_flag
---> 14 from _glmnet import lognet, splognet, lsolns
     15 from glmnet.util import (_fix_lambda_path,
     16                          _check_user_lambda,

ModuleNotFoundError: No module named '_glmnet'

[jacksonllee]

Hello, thank you for filing this issue! Based on the stack trace shown, my guess is that the underlying glmnet fortran library wasn't compiled, which would explain why _glmnet module wasn't found. I see that our contributing docs need updating: We'll update the page to mention installing python-glmnet (which includes compiling the fortran library) after cloning.

If an example is helpful for how an environment can be set up for dev work and running tests, here is the configuration for this repo's continuous integration; this example is for Linux.

[amirhessam88]

Gotcha. I try to follow the config guidelines. I let you know soon.
Thanks
-Amir

[thibault-wch]

I also encountered the same problem,whether the software installed on the Windows can directly provide copy import

[eduard93]

Got the same error on:

import glmnet

first after

pip install glmnet

and later with:

python setup.py install

Workaround. With process monitor I determined the locations where import searched for the dll:

Time of Day	Process Name	PID	Operation	Path	Result	Detail
12:58:36,7560094	python.exe	17688	FASTIO_NETWORK_QUERY_OPEN	%PYTHONHOME%\Lib\site-packages\libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll	FAST IO DISALLOWED
12:58:36,7562356	python.exe	17688	IRP_MJ_CREATE	%PYTHONHOME%\Lib\site-packages\libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll	NAME NOT FOUND	Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a
12:58:36,7564435	python.exe	17688	FASTIO_NETWORK_QUERY_OPEN	%PYTHONHOME%\libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll	FAST IO DISALLOWED
12:58:36,7566073	python.exe	17688	IRP_MJ_CREATE	%PYTHONHOME%\libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll	NAME NOT FOUND	Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a
12:58:36,7568173	python.exe	17688	FASTIO_NETWORK_QUERY_OPEN	%PYTHONHOME%\Lib\site-packages\scipy.libs\libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll	FAST IO DISALLOWED
12:58:36,7569767	python.exe	17688	IRP_MJ_CREATE	%PYTHONHOME%\Lib\site-packages\scipy.libs\libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll	NAME NOT FOUND	Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a
12:58:36,7571792	python.exe	17688	FASTIO_NETWORK_QUERY_OPEN	%PYTHONHOME%\Lib\site-packages\numpy.libs\libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll	FAST IO DISALLOWED
12:58:36,7573352	python.exe	17688	IRP_MJ_CREATE	%PYTHONHOME%\Lib\site-packages\numpy.libs\libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll	NAME NOT FOUND	Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a
12:58:36,7575515	python.exe	17688	FASTIO_NETWORK_QUERY_OPEN	C:\Windows\System32\libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll	FAST IO DISALLOWED
12:58:36,7577210	python.exe	17688	IRP_MJ_CREATE	C:\Windows\System32\libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll	NAME NOT FOUND	Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a

So essentially it searched for the glmnet fortran lib in these folders:

%PYTHONHOME%\Lib\site-packages\
%PYTHONHOME%\
%PYTHONHOME%\Lib\site-packages\scipy\.libs\
%PYTHONHOME%\Lib\site-packages\numpy\.libs\
C:\Windows\System32\

Note that %PYTHONHOME%\Lib\site-packages\_glmnet\.libs was not there (and it was empty anyway) but whatever.

In my <repo>\build\lib.win-amd64-3.8\_glmnet\.libs I have found libglmnet5.US5ZYWTFU2OBR7NGPF2SI664TNQQMRA3.gfortran-win_amd64.dll, copied it to %PYTHONHOME%\Lib\site-packages\. After that import glmnet started working.

[aFewThings]

With this solution of @eduard93, I finally succeed in building the glmnet in window os.

Maybe this is fundamentally unrelated to this thread, but I think there are people like me who are not familiar with the compiling process. so I will briefly summarize the installation for windows users.

When you compile using fortran, the os architecture version of 32-bit or 64-bit is important.

If you are 64-bit windows user, I recommend the following procedure.

1. Download and install tdm64-gcc-***.exe from tdm-gcc website

• During the installation, you should select MinGW-w64/TDM64 (32-bit and 64-bit). And also check fortran checkbox as below.

When it installed completely, you should confirm that your terminal recognize gfortran -v command as below.

C:\... >gfortran -v
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=C:/TDM-GCC-64/bin/../libexec/gcc/x86_64-w64-mingw32/10.3.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../../../src/gcc-git-10.3.0/configure --build=x86_64-w64-mingw32 --enable-targets=all --enable-languages=ada,c,c++,fortran,jit,lto,objc,obj-c++ --enable-libgomp --enable-lto --enable-graphite --enable-cxx-flags=-DWINPTHREAD_STATIC --disable-build-with-cxx --disable-build-poststage1-with-cxx --enable-libstdcxx-debug --enable-threads=posix --enable-version-specific-runtime-libs --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts=yes --disable-libstdcxx-pch --enable-libstdcxx-threads --enable-libstdcxx-time=yes --enable-mingw-wildcard --with-gnu-ld --disable-werror --enable-nls --disable-win32-registry --enable-large-address-aware --disable-rpath --disable-symvers --prefix=/mingw64tdm --with-local-prefix=/mingw64tdm --with-pkgversion=tdm64-1 --with-bugurl=https://github.com/jmeubank/tdm-gcc/issues
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.3.0 (tdm64-1)

2. Install glmnet library

• it will install glmnet, but I also met same issue with this thread (ModuleNotFoundError).

3. Follow the solution of @eduard93