ERROR: Could not build wheels for deepmd-kit, which is required to install pyproject.toml-based projects #1958

wangych6 · 2022-09-30T07:39:24Z

wangych6
Sep 30, 2022

envs：Centos/tf2.9 in docker/python3.8/ (also try in tf2.8-docker, tf2.0-docker all of these meet the problem).

When I install from source code (pip install .) ，I meet this problem, and I try a lot (such as update pip, install wheel ), but can't solve this question.

njzjz · 2022-09-30T07:42:18Z

njzjz
Sep 30, 2022
Maintainer

Since you are using docker, could you provide the docker image for debug?

8 replies

njzjz Feb 15, 2023
Maintainer

Can you open a new thread to provide detailed descriptions and the entire error messages? If you want to find the root cause, only some simple descriptions are not enough.

Zhang-Zhiyuan-zzy Feb 15, 2023

Yes, the entire error messages, following:
$ pip install .

Processing /home/pub/sw/tmp/deepmd-src
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting dargs>=0.2.6
Using cached dargs-0.3.4-py3-none-any.whl (18 kB)
Requirement already satisfied: h5py in /home/pub/sw/CCDC/Python_API_2022/miniconda/lib/python3.7/site-packages (from deepmd-kit==2.2.0b0) (2.10.0)
Requirement already satisfied: packaging in /home/pub/sw/CCDC/Python_API_2022/miniconda/lib/python3.7/site-packages (from deepmd-kit==2.2.0b0) (21.3)
Requirement already satisfied: importlib-metadata>=1.4 in /home/pub/sw/CCDC/Python_API_2022/miniconda/lib/python3.7/site-packages (from deepmd-kit==2.2.0b0) (4.11.3)
Collecting wcmatch
Using cached wcmatch-8.4.1-py3-none-any.whl (39 kB)
Collecting pyyaml
Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
Requirement already satisfied: numpy in /home/pub/sw/CCDC/Python_API_2022/miniconda/lib/python3.7/site-packages (from deepmd-kit==2.2.0b0) (1.21.6)
Requirement already satisfied: scipy in /home/pub/sw/deepmd/lib/python3.7/site-packages (from deepmd-kit==2.2.0b0) (1.4.1)
Collecting python-hostlist>=1.21
Using cached python_hostlist-1.23.0-py3-none-any.whl
Requirement already satisfied: zipp>=0.5 in /home/pub/sw/CCDC/Python_API_2022/miniconda/lib/python3.7/site-packages (from importlib-metadata>=1.4->deepmd-kit==2.2.0b0) (3.8.0)
Requirement already satisfied: typing-extensions>=3.6.4 in /home/pub/sw/CCDC/Python_API_2022/miniconda/lib/python3.7/site-packages (from importlib-metadata>=1.4->deepmd-kit==2.2.0b0) (4.2.0)
Requirement already satisfied: six in /home/pub/sw/CCDC/Python_API_2022/miniconda/lib/python3.7/site-packages (from h5py->deepmd-kit==2.2.0b0) (1.16.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/pub/sw/CCDC/Python_API_2022/miniconda/lib/python3.7/site-packages (from packaging->deepmd-kit==2.2.0b0) (3.0.8)
Collecting bracex>=2.1.1
Using cached bracex-2.3.post1-py3-none-any.whl (12 kB)
Building wheels for collected packages: deepmd-kit
Building wheel for deepmd-kit (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for deepmd-kit (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [59 lines of output]

  --------------------------------------------------------------------------------
  -- Trying 'Ninja' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 9.3.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- The CXX compiler identification is GNU 9.3.0
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /home/pub/sw/tmp/deepmd-src/_cmake_test_compile/build
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'Ninja' generator - success
  --------------------------------------------------------------------------------
  
  Configuring Project
    Working directory:
      /home/pub/sw/tmp/deepmd-src/_skbuild/linux-x86_64-3.7/cmake-build
    Command:
      /tmp/pip-build-env-v3ykuhpf/overlay/lib/python3.7/site-packages/cmake/data/bin/cmake /home/pub/sw/tmp/deepmd-src/source -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/home/pub/sw/tmp/deepmd-src/_skbuild/linux-x86_64-3.7/cmake-install -DPYTHON_VERSION_STRING:STRING=3.7.9 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-v3ykuhpf/overlay/lib/python3.7/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/home/pub/sw/deepmd/bin/python3.7 -DPYTHON_INCLUDE_DIR:PATH=/home/pub/sw/CCDC/Python_API_2022/miniconda/include/python3.7m -DPYTHON_LIBRARY:PATH=/home/pub/sw/CCDC/Python_API_2022/miniconda/lib/libpython3.7m.so -DPython_EXECUTABLE:PATH=/home/pub/sw/deepmd/bin/python3.7 -DPython_ROOT_DIR:PATH=/home/pub/sw/deepmd -DPython_INCLUDE_DIR:PATH=/home/pub/sw/CCDC/Python_API_2022/miniconda/include/python3.7m -DPython_FIND_REGISTRY:STRING=NEVER -DPython3_EXECUTABLE:PATH=/home/pub/sw/deepmd/bin/python3.7 -DPython3_ROOT_DIR:PATH=/home/pub/sw/deepmd -DPython3_INCLUDE_DIR:PATH=/home/pub/sw/CCDC/Python_API_2022/miniconda/include/python3.7m -DPython3_FIND_REGISTRY:STRING=NEVER -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-v3ykuhpf/overlay/lib/python3.7/site-packages/ninja/data/bin/ninja -DTENSORFLOW_ROOT:PATH=/home/pub/sw/deepmd/lib/python3.7/site-packages/tensorflow -DBUILD_PY_IF:BOOL=TRUE -DBUILD_CPP_IF:BOOL=FALSE -DCMAKE_BUILD_TYPE:STRING=Release
  
  CMake Error: The current CMakeCache.txt directory /home/pub/sw/tmp/deepmd-src/_skbuild/linux-x86_64-3.7/cmake-build/CMakeCache.txt is different than the directory /home/pub/sw/tmp/deepmd-kit/_skbuild/linux-x86_64-3.7/cmake-build where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
  CMake Error: The source "/home/pub/sw/tmp/deepmd-src/source/CMakeLists.txt" does not match the source "/home/pub/sw/tmp/deepmd-kit/source/CMakeLists.txt" used to generate cache.  Re-run cmake with a different source directory.
  Traceback (most recent call last):
    File "/tmp/pip-build-env-v3ykuhpf/overlay/lib/python3.7/site-packages/skbuild/setuptools_wrap.py", line 646, in setup
      languages=cmake_languages,
    File "/tmp/pip-build-env-v3ykuhpf/overlay/lib/python3.7/site-packages/skbuild/cmaker.py", line 338, in configure
      "An error occurred while configuring with CMake.\n"
  
  An error occurred while configuring with CMake.
    Command:
      /tmp/pip-build-env-v3ykuhpf/overlay/lib/python3.7/site-packages/cmake/data/bin/cmake /home/pub/sw/tmp/deepmd-src/source -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/home/pub/sw/tmp/deepmd-src/_skbuild/linux-x86_64-3.7/cmake-install -DPYTHON_VERSION_STRING:STRING=3.7.9 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-v3ykuhpf/overlay/lib/python3.7/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/home/pub/sw/deepmd/bin/python3.7 -DPYTHON_INCLUDE_DIR:PATH=/home/pub/sw/CCDC/Python_API_2022/miniconda/include/python3.7m -DPYTHON_LIBRARY:PATH=/home/pub/sw/CCDC/Python_API_2022/miniconda/lib/libpython3.7m.so -DPython_EXECUTABLE:PATH=/home/pub/sw/deepmd/bin/python3.7 -DPython_ROOT_DIR:PATH=/home/pub/sw/deepmd -DPython_INCLUDE_DIR:PATH=/home/pub/sw/CCDC/Python_API_2022/miniconda/include/python3.7m -DPython_FIND_REGISTRY:STRING=NEVER -DPython3_EXECUTABLE:PATH=/home/pub/sw/deepmd/bin/python3.7 -DPython3_ROOT_DIR:PATH=/home/pub/sw/deepmd -DPython3_INCLUDE_DIR:PATH=/home/pub/sw/CCDC/Python_API_2022/miniconda/include/python3.7m -DPython3_FIND_REGISTRY:STRING=NEVER -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-v3ykuhpf/overlay/lib/python3.7/site-packages/ninja/data/bin/ninja -DTENSORFLOW_ROOT:PATH=/home/pub/sw/deepmd/lib/python3.7/site-packages/tensorflow -DBUILD_PY_IF:BOOL=TRUE -DBUILD_CPP_IF:BOOL=FALSE -DCMAKE_BUILD_TYPE:STRING=Release
    Source directory:
      /home/pub/sw/tmp/deepmd-src/source
    Working directory:
      /home/pub/sw/tmp/deepmd-src/_skbuild/linux-x86_64-3.7/cmake-build
  Please see CMake's output for more information.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for deepmd-kit
Failed to build deepmd-kit
ERROR: Could not build wheels for deepmd-kit, which is required to install pyproject.toml-based projects

njzjz Feb 15, 2023
Maintainer

Per your error message, it should be helpful to clean the cache.

CMake Error: The current CMakeCache.txt directory /home/pub/sw/tmp/deepmd-src/_skbuild/linux-x86_64-3.7/cmake-build/CMakeCache.txt is different than the directory /home/pub/sw/tmp/deepmd-kit/_skbuild/linux-x86_64-3.7/cmake-build where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
  CMake Error: The source "/home/pub/sw/tmp/deepmd-src/source/CMakeLists.txt" does not match the source "/home/pub/sw/tmp/deepmd-kit/source/CMakeLists.txt" used to generate cache.  Re-run cmake with a different source directory.

Zhang-Zhiyuan-zzy Mar 2, 2023

如果您认识中文的话能否帮忙解决一下这个错误：
-似乎是需要手动设置OP_CXX_ABI的值，我尝试了好几个值比如 1，2，3, cxx11, cxx17。但是得到完全相同的错误。
-我之所以想要从源编译是因为我想使用--finetune 这个命令参数。但是我目前只找到2.2.0的源编译版本。

A new error has arisen that appears to suggest I need to manually set OP_CXX_ABI. I have attempted to solve the issue by trying several values: 1, 2, 3, cxx11, cxx17. However, the problem persists and I receive the following message (I want to fine-tune my models, but it is currently only allowed in v2.2.0):

Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy in /home/pub/sw/deep-kit/lib/python3.10/site-packages (from deepmd-kit==2.2.0b0) (1.22.3)
Requirement already satisfied: wcmatch in /home/pub/sw/deep-kit/lib/python3.10/site-packages (from deepmd-kit==2.2.0b0) (8.2)
Requirement already satisfied: scipy in /home/pub/sw/deep-kit/lib/python3.10/site-packages (from deepmd-kit==2.2.0b0) (1.7.3)
Requirement already satisfied: dargs>=0.2.6 in /home/pub/sw/deep-kit/lib/python3.10/site-packages (from deepmd-kit==2.2.0b0) (0.2.6)
Requirement already satisfied: h5py in /home/pub/sw/deep-kit/lib/python3.10/site-packages (from deepmd-kit==2.2.0b0) (3.7.0)
Requirement already satisfied: python-hostlist>=1.21 in /home/pub/sw/deep-kit/lib/python3.10/site-packages (from deepmd-kit==2.2.0b0) (1.21)
Requirement already satisfied: pyyaml in /home/pub/sw/deep-kit/lib/python3.10/site-packages/PyYAML-6.0-py3.10-linux-x86_64.egg (from deepmd-kit==2.2.0b0) (6.0)
Requirement already satisfied: packaging in /home/pub/sw/deep-kit/lib/python3.10/site-packages (from deepmd-kit==2.2.0b0) (21.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/pub/sw/deep-kit/lib/python3.10/site-packages (from packaging->deepmd-kit==2.2.0b0) (3.0.9)
Requirement already satisfied: bracex>=2.1.1 in /home/pub/sw/deep-kit/lib/python3.10/site-packages (from wcmatch->deepmd-kit==2.2.0b0) (2.1.1)
Building wheels for collected packages: deepmd-kit
Building wheel for deepmd-kit (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for deepmd-kit (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [85 lines of output]

  --------------------------------------------------------------------------------
  -- Trying 'Ninja' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 9.3.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- The CXX compiler identification is GNU 9.3.0
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /home/pub/sw/tmp/deepmd-kit/_cmake_test_compile/build
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'Ninja' generator - success
  --------------------------------------------------------------------------------
  
  Configuring Project
    Working directory:
      /home/pub/sw/tmp/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-build
    Command:
      /tmp/pip-build-env-c7aojm7y/overlay/lib/python3.10/site-packages/cmake/data/bin/cmake /home/pub/sw/tmp/deepmd-kit/source -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/home/pub/sw/tmp/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.4 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-c7aojm7y/overlay/lib/python3.10/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/home/pub/sw/deepmd10/bin/python -DPYTHON_INCLUDE_DIR:PATH=/home/pub/sw/deep-kit/include/python3.10 -DPYTHON_LIBRARY:PATH=/home/pub/sw/deep-kit/lib/libpython3.10.so -DPython_EXECUTABLE:PATH=/home/pub/sw/deepmd10/bin/python -DPython_ROOT_DIR:PATH=/home/pub/sw/deepmd10 -DPython_INCLUDE_DIR:PATH=/home/pub/sw/deep-kit/include/python3.10 -DPython_FIND_REGISTRY:STRING=NEVER -DPython3_EXECUTABLE:PATH=/home/pub/sw/deepmd10/bin/python -DPython3_ROOT_DIR:PATH=/home/pub/sw/deepmd10 -DPython3_INCLUDE_DIR:PATH=/home/pub/sw/deep-kit/include/python3.10 -DPython3_FIND_REGISTRY:STRING=NEVER -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-c7aojm7y/overlay/lib/python3.10/site-packages/ninja/data/bin/ninja -DTENSORFLOW_ROOT:PATH=/home/pub/sw/deep-kit/lib/python3.10/site-packages/tensorflow -DBUILD_PY_IF:BOOL=TRUE -DBUILD_CPP_IF:BOOL=FALSE -DCMAKE_BUILD_TYPE:STRING=Release
  
  -- Supported model version: 1.1
  -- Will not build nv GPU support
  -- Will not build AMD GPU support
  -- Disabled cpp interface build, looking for tensorflow_framework
  CMake Warning (dev) at cmake/Findtensorflow.cmake:190 (file):
    You have used file(GET_RUNTIME_DEPENDENCIES) in project mode.  This is
    probably not what you intended to do.  Instead, please consider using it in
    an install(CODE) or install(SCRIPT) command.  For example:
  
      install(CODE [[
        file(GET_RUNTIME_DEPENDENCIES
          # ...
          )
        ]])
  Call Stack (most recent call first):
    CMakeLists.txt:123 (find_package)
  This warning is for project developers.  Use -Wno-dev to suppress it.
  
  -- Found TensorFlow: /home/pub/sw/deep-kit/lib/python3.10/site-packages/tensorflow/include;/tmp/pip-build-env-czgo4457/normal/lib/python3.10/site-packages/tensorflow/include, , /home/pub/sw/deep-kit/lib/python3.10/site-packages/tensorflow/libtensorflow_framework.so.2  in /home/pub/sw/deep-kit/lib/python3.10/site-packages/tensorflow;/home/pub/sw/deep-kit/lib/python3.10/site-packages/tensorflow/../tensorflow_core (found version "2.9.0")
  CMake Error at cmake/Findtensorflow.cmake:304 (message):
    Failed to detect OP_CXX_ABI, please set it manually
  Call Stack (most recent call first):
    CMakeLists.txt:123 (find_package)
  
  
  -- Configuring incomplete, errors occurred!
  See also "/home/pub/sw/tmp/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-build/CMakeFiles/CMakeOutput.log".
  See also "/home/pub/sw/tmp/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-build/CMakeFiles/CMakeError.log".
  Traceback (most recent call last):
    File "/tmp/pip-build-env-c7aojm7y/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 634, in setup
      env = cmkr.configure(
    File "/tmp/pip-build-env-c7aojm7y/overlay/lib/python3.10/site-packages/skbuild/cmaker.py", line 332, in configure
      raise SKBuildError(
  
  An error occurred while configuring with CMake.
    Command:
      /tmp/pip-build-env-c7aojm7y/overlay/lib/python3.10/site-packages/cmake/data/bin/cmake /home/pub/sw/tmp/deepmd-kit/source -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/home/pub/sw/tmp/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.4 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-c7aojm7y/overlay/lib/python3.10/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/home/pub/sw/deepmd10/bin/python -DPYTHON_INCLUDE_DIR:PATH=/home/pub/sw/deep-kit/include/python3.10 -DPYTHON_LIBRARY:PATH=/home/pub/sw/deep-kit/lib/libpython3.10.so -DPython_EXECUTABLE:PATH=/home/pub/sw/deepmd10/bin/python -DPython_ROOT_DIR:PATH=/home/pub/sw/deepmd10 -DPython_INCLUDE_DIR:PATH=/home/pub/sw/deep-kit/include/python3.10 -DPython_FIND_REGISTRY:STRING=NEVER -DPython3_EXECUTABLE:PATH=/home/pub/sw/deepmd10/bin/python -DPython3_ROOT_DIR:PATH=/home/pub/sw/deepmd10 -DPython3_INCLUDE_DIR:PATH=/home/pub/sw/deep-kit/include/python3.10 -DPython3_FIND_REGISTRY:STRING=NEVER -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-c7aojm7y/overlay/lib/python3.10/site-packages/ninja/data/bin/ninja -DTENSORFLOW_ROOT:PATH=/home/pub/sw/deep-kit/lib/python3.10/site-packages/tensorflow -DBUILD_PY_IF:BOOL=TRUE -DBUILD_CPP_IF:BOOL=FALSE -DCMAKE_BUILD_TYPE:STRING=Release
    Source directory:
      /home/pub/sw/tmp/deepmd-kit/source
    Working directory:
      /home/pub/sw/tmp/deepmd-kit/_skbuild/linux-x86_64-3.10/cmake-build
  Please see CMake's output for more information.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for deepmd-kit
Failed to build deepmd-kit
ERROR: Could not build wheels for deepmd-kit, which is required to install pyproject.toml-based projects

njzjz Mar 3, 2023
Maintainer

See #2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ERROR: Could not build wheels for deepmd-kit, which is required to install pyproject.toml-based projects #1958

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment 8 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

ERROR: Could not build wheels for deepmd-kit, which is required to install pyproject.toml-based projects #1958

Uh oh!

wangych6 Sep 30, 2022

Replies: 1 comment · 8 replies

Uh oh!

njzjz Sep 30, 2022 Maintainer

Uh oh!

njzjz Feb 15, 2023 Maintainer

Uh oh!

Zhang-Zhiyuan-zzy Feb 15, 2023

Yes, the entire error messages, following: $ pip install .

Uh oh!

njzjz Feb 15, 2023 Maintainer

Uh oh!

Zhang-Zhiyuan-zzy Mar 2, 2023

Uh oh!

njzjz Mar 3, 2023 Maintainer

wangych6
Sep 30, 2022

Replies: 1 comment 8 replies

njzjz
Sep 30, 2022
Maintainer

njzjz Feb 15, 2023
Maintainer

Yes, the entire error messages, following:
$ pip install .

njzjz Feb 15, 2023
Maintainer

njzjz Mar 3, 2023
Maintainer