-
Notifications
You must be signed in to change notification settings - Fork 115
Installation FAQ (pip, setup.py)
This page covers questions about installing GalSim with pip or setup.py. For questions about installing with SCons, go here.
Sorry this page is so spare currently. This is a new installation process, so we're not sure yet what the "frequently-asked" questions will be. For now, see the installation instructions here
-
Dependencies
-
Installing with pip
-
Installing with setup.py
-
Installing with conda
-
Importing galsim after installation
When running pip install galsim, you may get the message:
****
Installation requires setuptools version >= 38.
Please upgrade or install with pip install -U setuptools
****
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/tmp/pip-build-a04km4cq/galsim/setup.py", line 30, in <module>
from setuptools.command.build_clib import build_clib
ImportError: No module named 'setuptools.command.build_clib'
The suggestion to run pip install -U setuptools is normally effective to get this working.
Sometimes you may run into an SSL certificate verify error. E.g. the following output:
Collecting galsim
Downloading https://files.pythonhosted.org/packages/f5/c9/eae469e73476b208701a7cb21a22daec61ec16cfb121e1abb328a9344e9f/GalSim-2.0.0rc1.tar.gz (36.4MB)
100% |████████████████████████████████| 36.4MB 33kB/s
Complete output from command python setup.py egg_info:
Download error on https://pypi.org/simple/eigency/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600) -- Some packages may not be found!
Couldn't find index page for 'eigency' (maybe misspelled?)
Download error on https://pypi.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600) -- Some packages may not be found!
No local packages or working download links found for eigency>=1.77
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/tmp/pip-build-7cpz0c4h/galsim/setup.py", line 864, in <module>
zip_safe=False,
File "/sw/lib/python3.4/site-packages/setuptools/__init__.py", line 128, in setup
_install_setup_requires(attrs)
File "/sw/lib/python3.4/site-packages/setuptools/__init__.py", line 123, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/sw/lib/python3.4/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs
replace_conflicting=True,
File "/sw/lib/python3.4/site-packages/pkg_resources/__init__.py", line 770, in resolve
replace_conflicting=replace_conflicting
File "/sw/lib/python3.4/site-packages/pkg_resources/__init__.py", line 1053, in best_match
return self.obtain(req, installer)
File "/sw/lib/python3.4/site-packages/pkg_resources/__init__.py", line 1065, in obtain
return installer(requirement)
File "/sw/lib/python3.4/site-packages/setuptools/dist.py", line 581, in fetch_build_egg
return cmd.easy_install(req)
File "/sw/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 667, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('eigency>=1.77')
Using setuptools version 39.2.0
Python version = 3.4.3 (default, Sep 26 2015, 00:09:24)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
Adding eigency to build_dep
GalSim version is 2.0.0rc1
The result is that pip cannot automatically install the dependencies for you. So the solution is to just install each one by hand. In the above case, just pip install -U eigency. Then, in my case, that also failed to find its dependency, setuptools_scm. So I had to do pip install -U setuptools_scm. Etc.
There may be a different solution to fix the underlying problem with the SSL certificate, but manual installation of the dependencies worked, so I didn't investigate further.
Bryan Gillis reported the following error when trying to build GalSim: (The error actually happened when compiling the dependency, LSSTDESC/Coord, but the error is unrelated to the module being compiled.)
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -Iinclude -I/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-2.0/usr/include/python3.6m -c src/Angle.cpp -o build/temp.linux-x86_64-3.6/src/Angle.o
g++ -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-3.6/src/Angle.o -L/usr/lib64 -lpython3.6m -o build/lib.linux-x86_64-3.6/coord/_coord.cpython-36m-x86_64-linux-gnu.so
/usr/bin/ld: cannot find -lpython3.6m
collect2: error: ld returned 1 exit status
error: command 'g++' failed with exit status 1
The problem was that the library libpython3.6m.so was not in his LIBRARY_PATH. The directory with this library needs to be in both LIBRARY_PATH and LD_LIBRARY_PATH. The former is where the linker looks at build time, and the latter is where the system looks at run time. So you need to have both of these system variables set properly.
The solution is to make sure both of the following are set properly, usually in your .bash_profile. In his case, this looked like:
export LIBRARY_PATH=$LIBARY_PATH:/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-2.0/usr/lib64/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-2.0/usr/lib64/
Obviously, you should replace the directory name abouve with wherever your system has the libpython*.so library file.
If you build with sudo and then later build without sudo, this can happen. The easiest resolution is probably to sudo rm -r ~/.ccache to blow away the current cache (with sudo) and start over.
a) Are you doing import galsim in the GalSim source directory?
If so, you might see something like the following:
>>> import galsim
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/Mike/GalSim/galsim/__init__.py", line 96, in <module>
from .position import Position, PositionI, PositionD
File "/Users/Mike/GalSim/galsim/position.py", line 22, in <module>
from . import _galsim
ImportError: cannot import name '_galsim'
Python looks in the current directory first when importing something. So in this case, it sees the galsim directory and tries to import that rather than from the installed location. But the build process doesn't build the _galsim.so library there. (It's in a subdirectory of the build directory.) So you get this error.
The solution is to change to a different directory and running Python from there.
b) It may be finding the wrong installation of galsim.
Sometimes you can end up with multiple GalSim versions installed on your machine, and python can get confused about which one is the right one to use. Especially if you have switched among the pip, SCons, and setup.py installation methods.
Normally you can check which version of something in imported by checking e.g. galsim.__file__. But if it
cannot import, this method doesn't work. Instead, you need to do the following:
>>> import imp
>>> imp.find_module('galsim')
(None, '/Users/Mike/anaconda3/lib/python3.6/site-packages/galsim', ('', '', 5))
That second item in the tuple is the location that it is trying to import from when you do import galsim. If that doesn't match the location that you think galsim is installed, then try deleting that installation and trying again.
One easy way to make sure you only have one version of galsim installed is to do pip uninstall galsim repeatedly until it says there is no version of galsim installed. Then reinstall it once.
a) This is another possible error if you have multiple versions of galsim installed and it is finding the wrong one at run time.
E.g. Niall MacCrann reported the following error:
>> import galsim
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/global/project/projectdirs/des/maccrann/py3/lib/python3.6/site-packages/galsim/__init__.py", line 94, in <module>
from .position import PositionI, PositionD
File "/global/project/projectdirs/des/maccrann/py3/lib/python3.6/site-packages/galsim/position.py", line 22, in <module>
from . import _galsim
ImportError: /global/project/projectdirs/des/maccrann/py3/lib/python3.6/site-packages/galsim/_galsim.so: undefined symbol: _ZN6galsim7Silicon10accumulateIfEEdRKNS_11PhotonArrayENS_14UniformDeviateENS_9ImageViewIT_EENS_8PositionIiEE
He found that deleting the files in this directory and doing a clean install fixed it.
Sometimes, especially when switching from a SCons-based installation to a pip or setup.py installation, not all of the previous version's files are correctly removed or updated, so manual deleting may be required. Further updates should work fine if you keep using the same installation method.
b) Another possibility is that the _galsim.so file is correct, but it is linked to the wrong libgalsim.so library.
E.g. the following error:
>>> import galsim
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/Mike/anaconda3/lib/python3.6/site-packages/galsim/__init__.py", line 96, in <module>
from .position import Position, PositionI, PositionD
File "/Users/Mike/anaconda3/lib/python3.6/site-packages/galsim/position.py", line 22, in <module>
from . import _galsim
ImportError: dlopen(/Users/Mike/anaconda3/lib/python3.6/site-packages/galsim/_galsim.cpython-36m-darwin.so, 2): Symbol not found: __ZN6galsim11invertImageISt7complexIdEEEvNS_9ImageViewIT_EE
Referenced from: /Users/Mike/anaconda3/lib/python3.6/site-packages/galsim/_galsim.cpython-36m-darwin.so
Expected in: flat namespace
in /Users/Mike/anaconda3/lib/python3.6/site-packages/galsim/_galsim.cpython-36m-darwin.so
>>>
This probably means that there is some libgalsim.so file in your LIBRARY_PATH, which was found at compile time, probably from an older SCons installation procedure. If setup.py finds that at link time, it can link to that older library, rather than the libgalsim.a library that it just compiled.
The way to check is to use either ldd (on Unix) or otool -L (on Mac) to check what external libraries _galsim.so is linking to:
$ otool -L /Users/Mike/anaconda3/lib/python3.6/site-packages/galsim/_galsim.cpython-36m-darwin.so
/Users/Mike/anaconda3/lib/python3.6/site-packages/galsim/_galsim.cpython-36m-darwin.so:
/sw/lib/libgalsim.2.0.dylib (compatibility version 2.0.0, current version 2.0.0)
/sw/lib/libfftw3.3.dylib (compatibility version 7.0.0, current version 7.2.0)
@rpath/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.19.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
/Users/Mike/anaconda3/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
The above is an example of an erroneous build. The libgalsim line should not be there, since we now statically link all the C++ code into the _galsim.so library.
The solution is to delete the file in the above location (/Users/Mike/lib/libgalsim.2.0.dylib in the above case), remove the GalSim/build directory, and rerun python setup.py install.
If you used pip to install, it might still have linked to this wrong library. To force pip to reinstall, you can do
pip uninstall galsim [possibly multiple times to make sure all versions are removed]
pip install galsim
That should then rebuild and hopefully link properly.
In the above example, after I removed the file /sw/lib/libgalsim.2.0.dylib, the output of otool is:
$ otool -L build/*/galsim/_galsim.cpython-36m-darwin.so
build/lib.macosx-10.7-x86_64-3.6/galsim/_galsim.cpython-36m-darwin.so:
/sw/lib/libfftw3.3.dylib (compatibility version 7.0.0, current version 7.2.0)
@rpath/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.19.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
/Users/Mike/anaconda3/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
Obviously, there are many possibilities here, but so far the main reason seems to be similar to the above ImportError problem. There may be some incompatibility between the _galsim library and the back-end libgalsim library. It may be that import galsim doesn't notice the incompatibility, but when the code tries to do something that uses the back-end C++ code, there is an incompatibility and you get a segmentation fault.
The solution is the same. Delete the old, now invalid libgalsim library, and rebuild GalSim.