Skip to content

libcublas.so.*[0-9] not found in the system path on CPU-only machine #19

@0x62

Description

@0x62

I'm trying to make use of this package on a fly.io machine. The image is built with Docker:

# Using the python:3.11-slim image
FROM python:3.11-slim

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1 PIPENV_VENV_IN_PROJECT=1 CUDA_VISIBLE_DEVICES=""

# Update and install dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends gcc libffi-dev libssl-dev && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Install pip and pipenv
RUN pip install --upgrade pip && \
    pip install pipenv

# Copy Pipfile and Pipfile.lock
COPY Pipfile Pipfile.lock ./

# Install dependencies using pipenv
RUN pipenv install --deploy --ignore-pipfile

# Copy the rest of the application files
COPY . .

ENTRYPOINT ["pipenv", "run"]

The image builds successfully, but when run it I get an error from Torch:

   INFO Preparing to run: `pipenv run python worker.py` as root
   INFO [fly api proxy] listening at /.fly/api
  2023/09/13 17:25:40 listening on [fdaa:1:291c:a7b:173:c2e5:763f:2]:22 (DNS: [fdaa::3]:53)
  Loading .env environment variables...
  Traceback (most recent call last):
    File "/app/.venv/lib/python3.11/site-packages/torch/__init__.py", line 168, in _load_global_deps
      ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
    File "/usr/local/lib/python3.11/ctypes/__init__.py", line 376, in __init__
      self._handle = _dlopen(self._name, mode)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  OSError: libcurand.so.10: cannot open shared object file: No such file or directory
  During handling of the above exception, another exception occurred:
  Traceback (most recent call last):
    File "/app/worker.py", line 5, in <module>
      from workers.classification import process as classify
    File "/app/workers/classification.py", line 2, in <module>
      import tweetnlp
    File "/app/.venv/lib/python3.11/site-packages/tweetnlp/__init__.py", line 2, in <module>
      from .loader import load_model, load_dataset
    File "/app/.venv/lib/python3.11/site-packages/tweetnlp/loader.py", line 3, in <module>
      from .text_classification.model import Sentiment, Offensive, Irony, Hate, Emotion, Emoji,\
    File "/app/.venv/lib/python3.11/site-packages/tweetnlp/text_classification/model.py", line 7, in <module>
      import torch
    File "/app/.venv/lib/python3.11/site-packages/torch/__init__.py", line 228, in <module>
      _load_global_deps()
    File "/app/.venv/lib/python3.11/site-packages/torch/__init__.py", line 189, in _load_global_deps
      _preload_cuda_deps(lib_folder, lib_name)
    File "/app/.venv/lib/python3.11/site-packages/torch/__init__.py", line 154, in _preload_cuda_deps
      raise ValueError(f"{lib_name} not found in the system path {sys.path}")
  ValueError: libcublas.so.*[0-9] not found in the system path ['/app', '/usr/local/lib/python311.zip', '/usr/local/lib/python3.11', '/usr/local/lib/python3.11/lib-dynload', '/app/.venv/lib/python3.11/site-packages']

Looking at #18, it appears to support CPU only, and the machine won't have any GPU's available so I'm not sure why it's not following the CPU init path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions