-
Notifications
You must be signed in to change notification settings - Fork 114
Installation FAQ (pip, setup.py)
This page covers questions about installing GalSim with pip or setup.py. For questions about installing with SCons, go here.
Sorry this page is so spare currently. This is a new installation process, so we're not sure yet what the "frequently-asked" questions will be. For now, see the installation instructions here
-
Dependencies
-
Installing with pip
-
Installing with setup.py
-
Installing with conda
-
Importing galsim after installation
When running pip install galsim, you may get the message:
****
Installation requires setuptools version >= 38.
Please upgrade or install with pip install -U setuptools
****
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/tmp/pip-build-a04km4cq/galsim/setup.py", line 30, in <module>
from setuptools.command.build_clib import build_clib
ImportError: No module named 'setuptools.command.build_clib'
The suggestion to run pip install -U setuptools is normally effective to get this working.
Sometimes you may run into an SSL certificate verify error. E.g. the following output:
Collecting galsim
Downloading https://files.pythonhosted.org/packages/f5/c9/eae469e73476b208701a7cb21a22daec61ec16cfb121e1abb328a9344e9f/GalSim-2.0.0rc1.tar.gz (36.4MB)
100% |████████████████████████████████| 36.4MB 33kB/s
Complete output from command python setup.py egg_info:
Download error on https://pypi.org/simple/eigency/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600) -- Some packages may not be found!
Couldn't find index page for 'eigency' (maybe misspelled?)
Download error on https://pypi.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600) -- Some packages may not be found!
No local packages or working download links found for eigency>=1.77
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/tmp/pip-build-7cpz0c4h/galsim/setup.py", line 864, in <module>
zip_safe=False,
File "/sw/lib/python3.4/site-packages/setuptools/__init__.py", line 128, in setup
_install_setup_requires(attrs)
File "/sw/lib/python3.4/site-packages/setuptools/__init__.py", line 123, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/sw/lib/python3.4/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs
replace_conflicting=True,
File "/sw/lib/python3.4/site-packages/pkg_resources/__init__.py", line 770, in resolve
replace_conflicting=replace_conflicting
File "/sw/lib/python3.4/site-packages/pkg_resources/__init__.py", line 1053, in best_match
return self.obtain(req, installer)
File "/sw/lib/python3.4/site-packages/pkg_resources/__init__.py", line 1065, in obtain
return installer(requirement)
File "/sw/lib/python3.4/site-packages/setuptools/dist.py", line 581, in fetch_build_egg
return cmd.easy_install(req)
File "/sw/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 667, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('eigency>=1.77')
Using setuptools version 39.2.0
Python version = 3.4.3 (default, Sep 26 2015, 00:09:24)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
Adding eigency to build_dep
GalSim version is 2.0.0rc1
The result is that pip cannot automatically install the dependencies for you. So the solution is to just install each one by hand. In the above case, just pip install -U eigency. Then, in my case, that also failed to find its dependency, setuptools_scm. So I had to do pip install -U setuptools_scm. Etc.
There may be a different solution to fix the underlying problem with the SSL certificate, but manual installation of the dependencies worked, so I didn't investigate further.
Bryan Gillis reported the following error when trying to build GalSim: (The error actually happened when compiling the dependency, LSSTDESC/Coord, but the error is unrelated to the module being compiled.)
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -Iinclude -I/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-2.0/usr/include/python3.6m -c src/Angle.cpp -o build/temp.linux-x86_64-3.6/src/Angle.o
g++ -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-3.6/src/Angle.o -L/usr/lib64 -lpython3.6m -o build/lib.linux-x86_64-3.6/coord/_coord.cpython-36m-x86_64-linux-gnu.so
/usr/bin/ld: cannot find -lpython3.6m
collect2: error: ld returned 1 exit status
error: command 'g++' failed with exit status 1
The problem was that the library libpython3.6m.so was not in his LIBRARY_PATH. The directory with this library needs to be in both LIBRARY_PATH and LD_LIBRARY_PATH. The former is where the linker looks at build time, and the latter is where the system looks at run time. So you need to have both of these system variables set properly.
The solution is to make sure both of the following are set properly, usually in your .bash_profile. In his case, this looked like:
export LIBRARY_PATH=$LIBARY_PATH:/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-2.0/usr/lib64/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/cvmfs/euclid-dev.in2p3.fr/CentOS7/EDEN-2.0/usr/lib64/
Obviously, you should replace the directory name abouve with wherever your system has the libpython*.so library file.
If you build with sudo and then later build without sudo, this can happen. The easiest resolution is probably to sudo rm -r ~/.ccache to blow away the current cache (with sudo) and start over.
a) This error can happen if you try to import galsim in the GalSim repository directory. Python tries to import the galsim directory that it finds there, rather than the (correct) installed one. Try changing to a different directory and running Python from there.
b) Sometimes you can end up with multiple GalSim versions installed on your machine, and python can get confused about which one is the right one to use. Especially if you have switched among the pip, SCons, and setup.py installation methods. Try deleting any old versions of GalSim on your system and trying again.
Note: If you aren't sure which GalSim installation python is trying to import, you can use the imp module. imp.find_module('galsim') returns a tuple whose second element is the directory location of galsim that import galsim will try to import.
a) This probably means that the _galsim.so file that is installed is not the current version, and a clean installation will likely fix the problem.
E.g. Niall MacCrann reported the following error:
>> import galsim
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/global/project/projectdirs/des/maccrann/py3/lib/python3.6/site-packages/galsim/__init__.py", line 94, in <module>
from .position import PositionI, PositionD
File "/global/project/projectdirs/des/maccrann/py3/lib/python3.6/site-packages/galsim/position.py", line 22, in <module>
from . import _galsim
ImportError: /global/project/projectdirs/des/maccrann/py3/lib/python3.6/site-packages/galsim/_galsim.so: undefined symbol: _ZN6galsim7Silicon10accumulateIfEEdRKNS_11PhotonArrayENS_14UniformDeviateENS_9ImageViewIT_EENS_8PositionIiEE
He found that deleting the files in this directory and doing a clean install fixed it.
Sometimes, especially when switching from a SCons-based installation to a pip or setup.py installation, not all of the previous version's files are correctly removed or updated, so manual deleting may be required. Further updates should work fine if you keep using the same installation method.
ImportError: dlopen([...]/site-packages/galsim/_galsim.cpython-36m-darwin.so, 2): Symbol not found: __ZN6galsim11invertImageISt7complexIdEEEvNS_9ImageViewIT_EE
This is a slight modification of the above error. Here, you probably have the right _galsim.so file, but it linked to the wrong libgalsim.so (or libgalsim.dylib on Mac).
Most likely, you still have a libgalsim library installed somewhere in your LIBRARY_PATH, probably from an old SCons installation, and the linker found that before the newly built libgalsim library.
If you installed directly with setup.py, rather than letting pip do it for you, you can check what _galsim.so is linking to using ldd (on Unix) or otool -L (on Mac):
$ otool -L build/*/galsim/_galsim.cpython-36m-darwin.so
build/lib.macosx-10.7-x86_64-3.6/galsim/_galsim.cpython-36m-darwin.so:
/sw/lib/libgalsim.2.0.dylib (compatibility version 2.0.0, current version 2.0.0)
/sw/lib/libfftw3.3.dylib (compatibility version 7.0.0, current version 7.2.0)
@rpath/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.19.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
/Users/Mike/anaconda3/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
The above is an example of an erroneous build. The libgalsim line should not be there, since we now statically link all the C++ code into the _galsim.so library.
The solution is to delete the file in the above location (/Users/Mike/lib/libgalsim.2.0.dylib in the above case), remove the GalSim/build directory, and rerun python setup.py install.
If you used pip to install, it might still have linked to this wrong library. To force pip to reinstall, you can do
pip uninstall galsim [possibly multiple times to make sure all versions are removed]
pip install galsim
That should then rebuild and hopefully link properly.
In the above example, after I removed the file /sw/lib/libgalsim.2.0.dylib, the output of otool is:
$ otool -L build/*/galsim/_galsim.cpython-36m-darwin.so
build/lib.macosx-10.7-x86_64-3.6/galsim/_galsim.cpython-36m-darwin.so:
/sw/lib/libfftw3.3.dylib (compatibility version 7.0.0, current version 7.2.0)
@rpath/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.19.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
/Users/Mike/anaconda3/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
Obviously, there are many possibilities here, but so far the main reason seems to be similar to the above ImportError problem. There may be some incompatibility between the _galsim library and the back-end libgalsim library. It may be that import galsim doesn't notice the incompatibility, but when the code tries to do something that uses the back-end C++ code, there is an incompatibility and you get a segmentation fault.
The solution is the same. Delete the old, now invalid libgalsim library, and rebuild GalSim.