Merge pull request #2405 from HenryDen/ViT_update

Update the vision llm page and add content of UI demo

Author: pareenaverma

content/learning-paths/mobile-graphics-and-gaming/vision-llm-inference-on-android-with-kleidiai-and-mnn/2-generate-apk.md

@@ -0,0 +1,53 @@
+---
+title: Benchmark the Vision Transformer performance with KleidiAI
+weight: 4
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Clone Vision Language Models repo
+
+In this section, you will run the Qwen model in action using a demo application using a Android Package Kit (APK).
+
+This repository is set up to enable building the app as an Android Studio project. 
+
+Run the following commands to clone the repository and checkout the source tree:
+
+```bash
+git clone https://gitlab.arm.com/kleidi/kleidi-examples/vision-language-models
+```
+
+## Build the App Using Android Studio
+
+You can use Android Studio to build the app and create an APK.
+
+### Open project and build
+
+Open Android Studio. 
+
+Go to **File > Open**. 
+
+Navigate to the vision-language-models directories, and click `Open`.
+
+This triggers a build of the project, and you should see output similar to the following on completion:
+
+```output
+BUILD SUCCESSFUL in 1m 42s
+```
+
+### Generate and Run the APK
+
+Navigate to **Build > Generate App Bundles or APKs**. Select **Generate APKs**.
+
+The build will be executed, and then the app will be copied and installed on the Android device.
+
+After opening the app, you will see the splash screen:
+
+![Loading screenshot](Loading_page.png)
+
+Finally, you can use the UI to chat with the app. Try uploading an image and ask a question on it.
+
+![Loading screenshot](chat2.png)
+
+The final step is to examine how KleidiAI can improve the performance of the model. Continue to the next section to find out.

content/learning-paths/mobile-graphics-and-gaming/vision-llm-inference-on-android-with-kleidiai-and-mnn/2-benchmark.md renamed to content/learning-paths/mobile-graphics-and-gaming/vision-llm-inference-on-android-with-kleidiai-and-mnn/3-benchmark.md

@@ -1,6 +1,6 @@
 ---
 title: Build the MNN Command-line ViT Demo
-weight: 4
+weight: 5
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
@@ -29,7 +29,7 @@ Run the following commands to clone the MNN repository and checkout the source t
 cd $HOME
 git clone https://github.com/alibaba/MNN.git
 cd MNN
-git checkout a739ea5870a4a45680f0e36ba9662ca39f2f4eec
+git checkout fa3b2161a9b38ac1e7dc46bb20259bd5eb240031
 ```
 
 Create a build directory and run the build script. 
@@ -40,10 +40,9 @@ The first time that you do this, build the binaries with the `-DMNN_KLEIDIAI` fl
 cd $HOME/MNN/project/android
 mkdir build_64 && cd build_64
 
-../build_64.sh "-DMNN_LOW_MEMORY=true -DLLM_SUPPORT_VISION=true -DMNN_KLEIDIAI=FALSE  \
-  -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true -DMNN_BUILD_LLM=true \
-  -DMNN_SUPPORT_TRANSFORMER_FUSE=true -DMNN_ARM82=true -DMNN_OPENCL=true \
-  -DMNN_USE_LOGCAT=true -DMNN_IMGCODECS=true -DMNN_BUILD_OPENCV=true"
+../build_64.sh "-DMNN_BUILD_LLM=true -DMNN_BUILD_LLM_OMNI=ON -DLLM_SUPPORT_VISION=true \
+-DMNN_BUILD_OPENCV=true -DMNN_IMGCODECS=true -DMNN_LOW_MEMORY=true \
+-DMNN_CPU_WEIGHT_DEQUANT_GEMM=true -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true"
 ```
 {{% notice Note %}}
 If your NDK toolchain isn't set up correctly, you might run into issues with the above script. Make a note of where the NDK was installed - this will be a directory named after the version you downloaded earlier. Try exporting the following environment variables before re-running `build_64.sh`:
@@ -102,14 +101,19 @@ prefill speed = 192.28 tok/s
 
 ## Enable KleidiAI and Re-run Inference
 
-The next step is to re-generate the binaries with KleidiAI activated. This is done by updating the flag `-DMNN_KLEIDIAI` to `TRUE`. 
+The next step is to re-generate the binaries with KleidiAI activated. This is done by inserting a hint into the code. 
+
+From the `MNN` directory, run:
+```bash
+sed -i '/void Llm::setRuntimeHint(std::shared_ptr<Express::Executor::RuntimeManager> &rtg) {/a\
+    rtg->setHint(MNN::Interpreter::CPU_ENABLE_KLEIDIAI, 1);' transformers/llm/engine/src/llm.cpp
+```
 
 From the `build_64` directory, run:
 ```bash
-../build_64.sh "-DMNN_LOW_MEMORY=true -DLLM_SUPPORT_VISION=true -DMNN_KLEIDIAI=TRUE \
--DMNN_CPU_WEIGHT_DEQUANT_GEMM=true -DMNN_BUILD_LLM=true \
--DMNN_SUPPORT_TRANSFORMER_FUSE=true -DMNN_ARM82=true -DMNN_OPENCL=true \
--DMNN_USE_LOGCAT=true -DMNN_IMGCODECS=true -DMNN_BUILD_OPENCV=true"
+../build_64.sh "-DMNN_BUILD_LLM=true -DMNN_BUILD_LLM_OMNI=ON -DLLM_SUPPORT_VISION=true \
+-DMNN_BUILD_OPENCV=true -DMNN_IMGCODECS=true -DMNN_LOW_MEMORY=true \
+-DMNN_CPU_WEIGHT_DEQUANT_GEMM=true -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true"
 ```
 ## Update Files on the Device