pdm cannot install gensim on Apple M1

[miss85246]

I tried install gensim on Mac mini M1 use pdm. I got this error:

  File "/var/folders/z2/t43z9qs932s2l40kdy9gft1w0000gn/T/pdm-build-env-kncd39dv-shared/lib/python3.9/site-packages/setuptools/_distutils/command/sdist.py", line 311, in _add_defaults_ext
    build_ext = self.get_finalized_command('build_ext')
  File "/var/folders/z2/t43z9qs932s2l40kdy9gft1w0000gn/T/pdm-build-env-kncd39dv-shared/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 299, in get_finalized_command
    cmd_obj.ensure_finalized()
  File "/var/folders/z2/t43z9qs932s2l40kdy9gft1w0000gn/T/pdm-build-env-kncd39dv-shared/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 107, in ensure_finalized
    self.finalize_options()
  File "setup.py", line 109, in finalize_options
    __builtins__.__NUMPY_SETUP__ = False
AttributeError: 'dict' object has no attribute '__NUMPY_SETUP__'
Error occurs: 
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/pdm/installers/synchronizers.py", line 185, in install_candidate
    self.manager.install(can)
  File "/opt/homebrew/lib/python3.9/site-packages/pdm/installers/manager.py", line 39, in install
    installer(prepared.build(), self.environment, prepared.direct_url())
  File "/opt/homebrew/lib/python3.9/site-packages/pdm/models/candidates.py", line 332, in build
    self.wheel = builder.build(build_dir, metadata_directory=self._metadata_dir)
  File "/opt/homebrew/lib/python3.9/site-packages/pdm/builders/wheel.py", line 28, in build
    filename = self._hook.build_wheel(out_dir, config_settings, metadata_directory)
  File "/opt/homebrew/lib/python3.9/site-packages/pep517/wrappers.py", line 208, in build_wheel
    return self._call_hook('build_wheel', {
  File "/opt/homebrew/lib/python3.9/site-packages/pep517/wrappers.py", line 322, in _call_hook
    self._subprocess_runner(
  File "/opt/homebrew/lib/python3.9/site-packages/pdm/builders/base.py", line 238, in subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
  File "/opt/homebrew/lib/python3.9/site-packages/pdm/builders/base.py", line 86, in log_subprocessor
    raise BuildError(
pdm.exceptions.BuildError: Call command ['/opt/homebrew/opt/[email protected]/bin/python3.9', '/opt/homebrew/lib/python3.9/site-packages/pep517/in_process/_in_process.py', 'build_wheel', '/var/folders/z2/t43z9qs932s2l40kdy9gft1w0000gn/T/tmpmylkqjez'] return non-zero status(1).
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting setuptools<=51.0.0
  Using cached https://mirrors.aliyun.com/pypi/packages/3d/f2/1489d3b6c72d68bf79cd0fba6b6c7497df4ebf7d40970e2d7eceb8d0ea9c/setuptools-51.0.0-py3-none-any.whl (785 kB)
Building wheels for collected packages: numpy
  Building wheel for numpy (pyproject.toml): started
  Building wheel for numpy (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Building wheel for numpy (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [897 lines of output]
      Running from numpy source directory.
      /private/var/folders/z2/t43z9qs932s2l40kdy9gft1w0000gn/T/pip-install-7buewoyz/numpy_1b6df4f4db7443f184cff22c4797a03a/tools/cythonize.py:73: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        required_version = LooseVersion('0.29.21')
      /private/var/folders/z2/t43z9qs932s2l40kdy9gft1w0000gn/T/pip-install-7buewoyz/numpy_1b6df4f4db7443f184cff22c4797a03a/tools/cythonize.py:75: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        if LooseVersion(cython_version) < required_version:
      numpy/random/_bounded_integers.pxd.in has not changed
      numpy/random/_philox.pyx has not changed
      numpy/random/_bounded_integers.pyx.in has not changed
      numpy/random/_sfc64.pyx has not changed
      numpy/random/_mt19937.pyx has not changed
      numpy/random/bit_generator.pyx has not changed
      Processing numpy/random/_bounded_integers.pyx
      numpy/random/mtrand.pyx has not changed
      numpy/random/_generator.pyx has not changed
      numpy/random/_pcg64.pyx has not changed
      numpy/random/_common.pyx has not changed
      Cythonizing sources
      blas_opt_info:
      blas_mkl_info:
      customize UnixCCompiler
        libraries mkl_rt not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
      
      blis_info:
        libraries blis not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
      
      openblas_info:
        libraries openblas not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
      
      atlas_3_10_blas_threads_info:
      Setting PTATLAS=ATLAS
        libraries tatlas not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
      
      atlas_3_10_blas_info:
        libraries satlas not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
      
      atlas_blas_threads_info:
      Setting PTATLAS=ATLAS
        libraries ptf77blas,ptcblas,atlas not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
      
      atlas_blas_info:
        libraries f77blas,cblas,atlas not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
      
      accelerate_info:
        libraries accelerate not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
      Library accelerate was not found. Ignoring
        libraries veclib not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
      Library veclib was not found. Ignoring
        FOUND:
          extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers']
          extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
          define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
      
        FOUND:
          extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers']
          extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
          define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
      
      non-existing path in 'numpy/distutils': 'site.cfg'
      lapack_opt_info:
      lapack_mkl_info:
        libraries mkl_rt not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
      
      openblas_lapack_info:
        libraries openblas not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
      
      openblas_clapack_info:
        libraries openblas,lapack not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
      
      flame_info:
        libraries flame not found in ['/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE

I try install gensim with pip without any mistake. Is there any idea?

[frostming]

Because the setup script of gensim relies on an implementation detail that does not work with PEP 517 build backend:

• Under __main__ script, __builtins__ is a module,
• Otherwise, __builtins__ is a dict, setting attributes on a dict leads to an error.

pip, however, doesn't enforce PEP 517 and will fall back to running setup.py if not available, so it works.

If you change the above line to:

import builtins
builtins.__NUMPY_SETUP__ = False

Everything works fine.

[miss85246]

but when I install it on my Intel macbook with pdm, It's don't raise any error. why?

[frostming]

Look at the released file list, it provides a wheel for mac intel, but no for mac M1, installing from a wheel doesn't need to execute the setup.py script(wheel is a file format that contains built products, so no build is needed).