Unable to install llama cpp python

[vivekshinde27]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

$ CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install --upgrade --force-reinstall llama-cpp-python --no-cache-dir
Collecting llama-cpp-python
  Downloading llama_cpp_python-0.2.57.tar.gz (36.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 36.9/36.9 MB 9.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting typing-extensions>=4.5.0 (from llama-cpp-python)
  Downloading typing_extensions-4.10.0-py3-none-any.whl.metadata (3.0 kB)
Collecting numpy>=1.20.0 (from llama-cpp-python)
  Downloading numpy-1.26.4-cp39-cp39-win_amd64.whl.metadata (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.0/61.0 kB ? eta 0:00:00
Collecting diskcache>=5.6.1 (from llama-cpp-python)
  Downloading diskcache-5.6.3-py3-none-any.whl.metadata (20 kB)
Collecting jinja2>=2.11.3 (from llama-cpp-python)
  Downloading Jinja2-3.1.3-py3-none-any.whl.metadata (3.3 kB)
Collecting MarkupSafe>=2.0 (from jinja2>=2.11.3->llama-cpp-python)
  Downloading MarkupSafe-2.1.5-cp39-cp39-win_amd64.whl.metadata (3.1 kB)
Downloading diskcache-5.6.3-py3-none-any.whl (45 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.5/45.5 kB ? eta 0:00:00
Downloading Jinja2-3.1.3-py3-none-any.whl (133 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.2/133.2 kB ? eta 0:00:00
Downloading numpy-1.26.4-cp39-cp39-win_amd64.whl (15.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.8/15.8 MB 23.3 MB/s eta 0:00:00
Downloading typing_extensions-4.10.0-py3-none-any.whl (33 kB)
Downloading MarkupSafe-2.1.5-cp39-cp39-win_amd64.whl (17 kB)
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [40 lines of output]
      *** scikit-build-core 0.8.2 using CMake 3.29.0 (wheel)
      *** Configuring CMake...
      2024-03-28 09:14:36,144 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None
      loading initial cache file C:\Users\AI\AppData\Local\Temp\tmpzh8s7v_k\build\CMakeInit.txt
      -- Building for: Visual Studio 17 2022
      -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045.
      -- The C compiler identification is MSVC 19.39.33522.0
      -- The CXX compiler identification is MSVC 19.39.33522.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found Git: C:/Program Files/Git/mingw64/bin/git.exe (found version "2.37.3.windows.1")
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
      -- Looking for pthread_create in pthreads
      -- Looking for pthread_create in pthreads - not found
      -- Looking for pthread_create in pthread
      -- Looking for pthread_create in pthread - not found
      -- Found Threads: TRUE
      -- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/include (found version "11.8.89")
      -- cuBLAS found
      CMake Error at C:/Users/AI/AppData/Local/Temp/pip-build-env-pefrjhcc/normal/Lib/site-packages/cmake/data/share/cmake-3.29/Modules/CMakeDetermineCompilerId.cmake:563 (message):
        No CUDA toolset found.
      Call Stack (most recent call first):
        C:/Users/AI/AppData/Local/Temp/pip-build-env-pefrjhcc/normal/Lib/site-packages/cmake/data/share/cmake-3.29/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
        C:/Users/AI/AppData/Local/Temp/pip-build-env-pefrjhcc/normal/Lib/site-packages/cmake/data/share/cmake-3.29/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test)
        C:/Users/AI/AppData/Local/Temp/pip-build-env-pefrjhcc/normal/Lib/site-packages/cmake/data/share/cmake-3.29/Modules/CMakeDetermineCUDACompiler.cmake:131 (CMAKE_DETERMINE_COMPILER_ID)
        vendor/llama.cpp/CMakeLists.txt:368 (enable_language)
     
     
      -- Configuring incomplete, errors occurred!
     
      *** CMake configuration failed
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

Description

I am trying to install llama cpp python as per the guideline mentioned in the langchain documentation but I am getting above errors. Please help me to resolve this issue.

System Info

SKYNET01+AI@SKYNET01 MINGW64 /c/ml-poc
$ pip freeze | grep langchain
langchain==0.1.11
langchain-community==0.0.25
langchain-core==0.1.29
langchain-text-splitters==0.0.1

[dosubot[bot]]

The issue you're encountering with the installation of llama-cpp-python seems to stem from the CUDA toolkit not being found. This is indicated by the error message No CUDA toolset found. To address this, first ensure that the CUDA toolkit is installed on your system. You can check this by running nvcc --version in your terminal, which should return the version of CUDA installed.

If CUDA is not installed, download it from the official NVIDIA website, making sure the version is compatible with both your GPU and the llama-cpp-python package. The specific CUDA version required by llama-cpp-python is not mentioned, so you might need to refer to the package's documentation for this detail.

After confirming that CUDA is correctly installed and configured, attempt reinstalling the llama-cpp-python package. If the problem persists, providing the exact version of the llama-cpp-python package you're trying to install could be helpful, as this detail was not included in your initial query.

It's also worth noting that a similar issue has been discussed in the LangChain repository: Unable to install llama cpp python, giving some wheel error. Monitoring this discussion may offer additional insights or solutions.

Should you need further assistance or have more questions, don't hesitate to reach out.

Sources

• Unable to install llama cpp python, giving some wheel error
• docs/docs/integrations/llms/llamacpp.ipynb
• libs/langchain/langchain/llms/llamacpp.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[ukicomputers]

@dosu-bot what do you think about this

cmake -B build -DBUILD_SHARED_LIBS=OFF -DCMAKE_PREFIX_PATH="~/llamacpp-build/lib/cmake/Llama"                                                                                                           
-- The C compiler identification is GNU 13.2.0
-- The CXX compiler identification is GNU 13.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:5 (find_package):
  Could not find a configuration file for package "Llama" that is compatible
  with requested version "0.0.1".

  The following configuration files were considered but not accepted:

    /usr/local/lib/cmake/llama/llama-config.cmake, version: unknown



-- Configuring incomplete, errors occurred!