Failed to load CUDA backend in JNI environment #3538
Description
Description
I use the source code of the latest version of whisper.cpp(v1.8.2)
as a submodule in my project. And I compile it with CUDA enabled on GitHub Actions and copy the libraries from CUDA Toolkit. All the libraries are:
graph TD;
A[whisper-jni.dll] --> B[whisper.dll];
A --> C[ggml.dll];
B --> C;
C --> D[ggml-base.dll];
C --> E[ggml-cpu.dll];
C --> F[ggml-cuda.dll];
E --> D;
F --> D;
F --> G[cublas64_12.dll];
F --> H[cublasLt64_12.dll];
I found that the CUDA backend is not loaded while loading the library. And the program will crash after logging whisper_init_with_params_no_state: dtw = 0. It is clear that the program crashes at whisper.cpp#L3713 ggml_backend_dev_count(),
in whisper_init_with_params_no_state() because the CUDA backend is not loaded so the native function will access a null pointer.
I try to download the pre-compiled too whisper-cublas-12.4.0-bin-x64.zip and
replace the dynamic link library (whisper.dll, ggml.dll, ggml-base.dll, ggml-cpu.dll, ggml-cuda.dll) with the which I compiled, and run the CLI tool successfully.
However, the JNI program still crashes at the same place. It can be inferred that all the dependent DLLs are correctly compiled and linked.
When I use the CLI tool, the log shows that the CUDA backend is loaded successfully.
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 3060 Laptop GPU, compute capability 8.6, VMM: yes
But I did not find any log about the CUDA backend in the JNI program. I can confirm that I call ggml_backend_load_all() in JNI_OnLoad()
before loading whisper model, but I did not find any log about the CUDA backend in the JNI program. It means that the CUDA backend is not loaded successfully.
Environment
-
OS: Windows 11
-
Java Version: 21
-
CUDA Version: 12.4
-
GPU: RTX 3060
-
whisper.cpp version: v1.8.2