v0.3.0 Android LLM app demo crashing on Pixel 8

[grisaitis]

🐛 Describe the bug

I'm finding that the v0.3.0 Android demo app crashes after I select the model artifact and tokenizer:

I think the reason is that it's trying to load /lib/arm64/libexecutorch_llama_jni.so, but /lib/arm64/ doesn't exist.

screen-20240920-183311.2.mp4

This is the logcat output:

---------------------------- PROCESS STARTED (3581) for package com.example.executorchllamademo ----------------------------
2024-09-20 18:33:01.983  3581-3581  Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 303326708; UID 10441; state: DISABLED
2024-09-20 18:33:02.010  3581-3581  Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 237531167; UID 10441; state: DISABLED
2024-09-20 18:33:02.881  3581-3588  utorchllamademo         com.example.executorchllamademo      I  Compiler allocated 4206KB to compile void android.widget.TextView.<init>(android.content.Context, android.util.AttributeSet, int, int)
2024-09-20 18:33:03.309  3581-3581  WindowOnBackDispatcher  com.example.executorchllamademo      W  sendCancelIfRunning: isInProgress=falsecallback=android.view.ViewRootImpl$$ExternalSyntheticLambda11@1d75f7a
2024-09-20 18:33:03.443  3581-3588  utorchllamademo         com.example.executorchllamademo      I  Compiler allocated 4666KB to compile void android.view.ViewRootImpl.performTraversals()
2024-09-20 18:33:04.209  3581-3581  WindowOnBackDispatcher  com.example.executorchllamademo      W  sendCancelIfRunning: isInProgress=falsecallback=android.view.ViewRootImpl$$ExternalSyntheticLambda11@280bf3d
2024-09-20 18:33:04.216  3581-3638  nativeloader            com.example.executorchllamademo      D  Load /data/app/~~BdsTx_Ph1EP8fHrLa-eCOQ==/com.example.executorchllamademo-ThN8KYgtvLAgbAABZH7YKQ==/lib/arm64/libexecutorch_llama_jni.so using ns clns-6 from class loader (caller=/data/app/~~BdsTx_Ph1EP8fHrLa-eCOQ==/com.example.executorchllamademo-ThN8KYgtvLAgbAABZH7YKQ==/base.apk!classes4.dex): dlopen failed: cannot locate symbol "_ZNSt6__ndk113basic_filebufIcNS_11char_traitsIcEEED1Ev" referenced by "/data/app/~~BdsTx_Ph1EP8fHrLa-eCOQ==/com.example.executorchllamademo-ThN8KYgtvLAgbAABZH7YKQ==/lib/arm64/libexecutorch_llama_jni.so"...
2024-09-20 18:33:04.218  3581-3638  AndroidRuntime          com.example.executorchllamademo      E  FATAL EXCEPTION: Thread-2
                                                                                                    Process: com.example.executorchllamademo, PID: 3581
                                                                                                    java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_ZNSt6__ndk113basic_filebufIcNS_11char_traitsIcEEED1Ev" referenced by "/data/app/~~BdsTx_Ph1EP8fHrLa-eCOQ==/com.example.executorchllamademo-ThN8KYgtvLAgbAABZH7YKQ==/lib/arm64/libexecutorch_llama_jni.so"...
                                                                                                    	at java.lang.Runtime.loadLibrary0(Runtime.java:1081)
                                                                                                    	at java.lang.Runtime.loadLibrary0(Runtime.java:1003)
                                                                                                    	at java.lang.System.loadLibrary(System.java:1765)
                                                                                                    	at com.facebook.soloader.nativeloader.SystemDelegate.loadLibrary(SystemDelegate.java:24)
                                                                                                    	at com.facebook.soloader.nativeloader.NativeLoader.loadLibrary(NativeLoader.java:52)
                                                                                                    	at com.facebook.soloader.nativeloader.NativeLoader.loadLibrary(NativeLoader.java:30)
                                                                                                    	at org.pytorch.executorch.LlamaModule.<clinit>(LlamaModule.java:21)
                                                                                                    	at com.example.executorchllamademo.MainActivity.setLocalModel(MainActivity.java:79)
                                                                                                    	at com.example.executorchllamademo.MainActivity.access$700(MainActivity.java:26)
                                                                                                    	at com.example.executorchllamademo.MainActivity$1.run(MainActivity.java:151)
                                                                                                    	at java.lang.Thread.run(Thread.java:1012)
2024-09-20 18:33:04.636  3581-3581  VRI[MainActivity]       com.example.executorchllamademo      D  visibilityChanged oldVisibility=true newVisibility=false
2024-09-20 18:33:04.723  3581-3581  WindowOnBackDispatcher  com.example.executorchllamademo      W  sendCancelIfRunning: isInProgress=falsecallback=android.view.ViewRootImpl$$ExternalSyntheticLambda11@35c6c15
2024-09-20 18:33:06.050  3581-3638  Process                 com.example.executorchllamademo      I  Sending signal. PID: 3581 SIG: 9
---------------------------- PROCESS ENDED (3581) for package com.example.executorchllamademo ----------------------------

I'm building the app with Llama 2 7B.

Any idea why this is happening?

Versions

PyTorch version: 2.4.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 14.4.1 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.3.9.4)
CMake version: version 3.30.3
Libc version: N/A

Python version: 3.10.0 (default, Mar  3 2022, 03:54:28) [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-14.4.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1

Versions of relevant libraries:
[pip3] executorch==0.3.0a0+7d77d78
[pip3] numpy==2.1.1
[pip3] torch==2.4.0
[pip3] torchao==0.1
[pip3] torchaudio==2.4.0
[pip3] torchsr==1.0.4
[pip3] torchvision==0.19.0
[conda] executorch                0.3.0a0+7d77d78          pypi_0    pypi
[conda] numpy                     2.1.1                    pypi_0    pypi
[conda] torch                     2.4.0                    pypi_0    pypi
[conda] torchao                   0.1                      pypi_0    pypi
[conda] torchaudio                2.4.0                    pypi_0    pypi
[conda] torchsr                   1.0.4                    pypi_0    pypi
[conda] torchvision               0.19.0                   pypi_0    pypi

[guangy10]

cc: @kirklandsign

[kirklandsign]

Thank you for the report! This is a known issue and I suspect it is related to NDK version. May I ask which NDK version you are using? I used 25 and worked for me.

Also you could try with main branch.

[grisaitis]

thanks @kirklandsign! I was using NDK 27 - that might be why.

I built with v0.4.0-rc2 and NDK 25, and it is able to load the model.

However, it crashes on generation. I'll create a separate issue for that, as it's a different kind of failure, and probably a different root cause.

Closing out as you solved the problem I brought up; the app now doesn't crash when i try to load the model + tokenizer artifacts 🙂

[grisaitis]

PS to clarify - i'm able to chat with llama 2 7B on the v0.3.0 demo app! works wonderfully now. NDK 25 did the trick.

[kirklandsign]

Hi @grisaitis BTW after #6163 we can use NDK 27 as well