Updated phi-3 mobile android example (#420)

* updated phi-3 mobile android example

* add model downloading step from url

* remove generated files

* fix the update

* update

* minor update

* Update mobile/examples/phi-3/android/.gitignore

Co-authored-by: Scott McKay <[email protected]>

* Update mobile/examples/phi-3/android/.gitignore

Co-authored-by: Scott McKay <[email protected]>

* Update mobile/examples/phi-3/android/README.md

Co-authored-by: Scott McKay <[email protected]>

* Update mobile/examples/phi-3/android/app/src/main/cpp/native-lib.cpp

Co-authored-by: Scott McKay <[email protected]>

* Update mobile/examples/phi-3/android/app/src/main/java/ai/onnxruntime/genai/demo/GenAIWrapper.java

Co-authored-by: Scott McKay <[email protected]>

* Update mobile/examples/phi-3/android/app/src/main/java/ai/onnxruntime/genai/demo/GenAIWrapper.java

Co-authored-by: Scott McKay <[email protected]>

* Update mobile/examples/phi-3/android/app/src/main/java/ai/onnxruntime/genai/demo/GenAIWrapper.java

Co-authored-by: Scott McKay <[email protected]>

* apply pr suggestions

* minor update

* apply pr suggestions

* update and fix

---------

Co-authored-by: Scott McKay <[email protected]>
Co-authored-by: rachguo <[email protected]>
Co-authored-by: Emma <[email protected]>

Author: 4 people

mobile/examples/phi-3/android/.gitignore

@@ -0,0 +1,10 @@
+*.iml
+.gradle
+.idea
+.DS_Store
+build
+/captures
+.externalNativeBuild
+.cxx
+local.properties
+

mobile/examples/phi-3/android/README.md

@@ -0,0 +1,59 @@
+# ONNX Runtime Mobile Phi-3 Android sample application with ORT GenAI library support
+
+## Overview
+
+This is a basic [Phi-3](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx) example application for [ONNX Runtime](https://github.com/microsoft/onnxruntime) on Android with [Ort-GenAI](https://github.com/microsoft/onnxruntime-genai) support for running generative AI models. This app demonstrates the usage of phi-3 model in a simple question answering chatbot mode.
+
+### Model
+The model used here is from the source: [HuggingFace](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx#phi-3-mini-4k-instruct-onnx-models)
+and optimized for int4 CPU and Mobile usage.
+
+### Requirements
+- Android Studio Giraffe | 2022.3.1 or later (installed on Mac/Windows/Linux)
+- Android SDK 29+
+- Android NDK r22+
+- An Android device or an Android Emulator
+
+## Build And Run
+
+### Step 1: Clone the ONNX runtime mobile examples source code
+
+Clone this repository to get the sample application. 
+
+`[email protected]:microsoft/onnxruntime-inference-examples.git`
+
+### [Optional] Step 2: Prepare the model
+
+The current set up supports downloading Phi-3-mini model directly from Huggingface repo to the android device folder. However, it takes time since the model data is >2.5G.
+
+You can also follow this link to download **Phi-3-mini**: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/tree/main/cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4
+and manually copy to the android device file directory following the below instructions:
+
+#### Steps for manual copying models to android device directory:
+From Android Studio:
+  - create (if necessary) and run your emulator/device
+    - make sure it has at least 8GB of internal storage
+  - debug/run the app so it's deployed to the device and creates it's `files` directory
+    - expected to be `/data/data/ai.onnxruntime.genai.demo/files`
+      - this is the path returned by `getFilesDir()`
+  - Open Device Explorer in Android Studio
+  - Navigate to `/data/data/ai.onnxruntime.genai.demo/files`
+    - adjust as needed if the value returned by getFilesDir() differs for your emulator or device
+  - copy the whole [phi-3](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/tree/main/cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4) model folder to the `files` directory
+
+### Step 3: Connect Android Device and Run the app
+  Connect your Android Device to your computer or select the Android Emulator in Android Studio Device manager.
+
+  Then select `Run -> Run app` and this will prompt the app to be built and installed on your device or emulator.
+
+  Now you can try giving some sample prompt questions and test the chatbot android app by clicking the ">" action button.
+
+#
+Here are some sample example screenshots of the app.
+
+<img width=20% src="images/Local_LLM_1.jpg" alt="App Screenshot 1" />
+
+<img width=20% src="images/Local_LLM_2.jpg" alt="App Screenshot 2" />
+
+<img width=20% src="images/Local_LLM_3.jpg" alt="App Screenshot 3" />
+

mobile/examples/phi-3/android/app/.gitignore

@@ -0,0 +1 @@
+/build

mobile/examples/phi-3/android/app/build.gradle.kts

@@ -0,0 +1,63 @@
+plugins {
+    id("com.android.application")
+}
+
+android {
+    namespace = "ai.onnxruntime.genai.demo"
+    compileSdk = 33
+
+    defaultConfig {
+        applicationId = "ai.onnxruntime.genai.demo"
+        minSdk = 27
+        targetSdk = 33
+        versionCode = 1
+        versionName = "1.0"
+
+        testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
+        externalNativeBuild {
+            cmake {
+                cppFlags += "-std=c++17"
+            }
+        }
+
+        ndk {
+            //noinspection ChromeOsAbiSupport
+            abiFilters += listOf("arm64-v8a", "x86_64")
+        }
+    }
+
+    buildTypes {
+        release {
+            isMinifyEnabled = false
+            proguardFiles(
+                getDefaultProguardFile("proguard-android-optimize.txt"),
+                "proguard-rules.pro"
+            )
+        }
+    }
+
+    compileOptions {
+        sourceCompatibility = JavaVersion.VERSION_1_8
+        targetCompatibility = JavaVersion.VERSION_1_8
+    }
+
+    externalNativeBuild {
+        cmake {
+            path = file("src/main/cpp/CMakeLists.txt")
+            version = "3.22.1"
+        }
+    }
+    buildFeatures {
+        viewBinding = true
+    }
+}
+
+dependencies {
+
+    implementation("androidx.appcompat:appcompat:1.6.1")
+    implementation("com.google.android.material:material:1.9.0")
+    implementation("androidx.constraintlayout:constraintlayout:2.1.4")
+    testImplementation("junit:junit:4.13.2")
+    androidTestImplementation("androidx.test.ext:junit:1.1.5")
+    androidTestImplementation("androidx.test.espresso:espresso-core:3.5.1")
+}

mobile/examples/phi-3/android/app/proguard-rules.pro

@@ -0,0 +1,21 @@
+# Add project specific ProGuard rules here.
+# You can control the set of applied configuration files using the
+# proguardFiles setting in build.gradle.
+#
+# For more details, see
+#   http://developer.android.com/guide/developing/tools/proguard.html
+
+# If your project uses WebView with JS, uncomment the following
+# and specify the fully qualified class name to the JavaScript interface
+# class:
+#-keepclassmembers class fqcn.of.javascript.interface.for.webview {
+#   public *;
+#}
+
+# Uncomment this to preserve the line number information for
+# debugging stack traces.
+#-keepattributes SourceFile,LineNumberTable
+
+# If you keep the line number information, uncomment this to
+# hide the original source file name.
+#-renamesourcefileattribute SourceFile

mobile/examples/phi-3/android/app/src/androidTest/java/ai/onnxruntime/genai/demo/ExampleInstrumentedTest.java

@@ -0,0 +1,26 @@
+package ai.onnxruntime.genai.demo;
+
+import android.content.Context;
+
+import androidx.test.platform.app.InstrumentationRegistry;
+import androidx.test.ext.junit.runners.AndroidJUnit4;
+
+import org.junit.Test;
+import org.junit.runner.RunWith;
+
+import static org.junit.Assert.*;
+
+/**
+ * Instrumented test, which will execute on an Android device.
+ *
+ * @see <a href="http://d.android.com/tools/testing">Testing documentation</a>
+ */
+@RunWith(AndroidJUnit4.class)
+public class ExampleInstrumentedTest {
+    @Test
+    public void useAppContext() {
+        // Context of the app under test.
+        Context appContext = InstrumentationRegistry.getInstrumentation().getTargetContext();
+        assertEquals("ai.onnxruntime.genai.demo", appContext.getPackageName());
+    }
+}

mobile/examples/phi-3/android/app/src/main/AndroidManifest.xml

@@ -0,0 +1,29 @@
+<?xml version="1.0" encoding="utf-8"?>
+<manifest xmlns:android="http://schemas.android.com/apk/res/android"
+    xmlns:tools="http://schemas.android.com/tools">
+
+    <uses-permission android:name="android.permission.INTERNET" />
+    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
+
+    <application
+        android:allowBackup="true"
+        android:dataExtractionRules="@xml/data_extraction_rules"
+        android:fullBackupContent="@xml/backup_rules"
+        android:icon="@mipmap/ic_launcher"
+        android:label="@string/app_name"
+        android:roundIcon="@mipmap/ic_launcher_round"
+        android:supportsRtl="true"
+        android:theme="@style/Theme.ORTGenAIDemo"
+        tools:targetApi="31">
+        <activity
+            android:name=".MainActivity"
+            android:exported="true">
+            <intent-filter>
+                <action android:name="android.intent.action.MAIN" />
+
+                <category android:name="android.intent.category.LAUNCHER" />
+            </intent-filter>
+        </activity>
+    </application>
+
+</manifest>

mobile/examples/phi-3/android/app/src/main/cpp/CMakeLists.txt

@@ -0,0 +1,43 @@
+# For more information about using CMake with Android Studio, read the
+# documentation: https://d.android.com/studio/projects/add-native-code.html.
+# For more examples on how to use CMake, see https://github.com/android/ndk-samples.
+
+# Sets the minimum CMake version required for this project.
+cmake_minimum_required(VERSION 3.22.1)
+
+# Declares the project name. The project name can be accessed via ${ PROJECT_NAME},
+# Since this is the top level CMakeLists.txt, the project name is also accessible
+# with ${CMAKE_PROJECT_NAME} (both CMake variables are in-sync within the top level
+# build script scope).
+project("genai")
+
+#set(APP_LIB_DIR ${PROJECT_SOURCE_DIR}/../../../libs)
+
+# Creates and names a library, sets it as either STATIC
+# or SHARED, and provides the relative paths to its source code.
+# You can define multiple libraries, and CMake builds them for you.
+# Gradle automatically packages shared libraries with your APK.
+#
+# In this top level CMakeLists.txt, ${CMAKE_PROJECT_NAME} is used to define
+# the target library name; in the sub-module's CMakeLists.txt, ${PROJECT_NAME}
+# is preferred for the same purpose.
+#
+# In order to load a library into your app from Java/Kotlin, you must call
+# System.loadLibrary() and pass the name of the library defined here;
+# for GameActivity/NativeActivity derived applications, the same library name must be
+# used in the AndroidManifest.xml file.
+add_library(${CMAKE_PROJECT_NAME} SHARED
+        # List C/C++ source files with relative paths to this CMakeLists.txt.
+        native-lib.cpp)
+
+target_link_directories(${CMAKE_PROJECT_NAME} PRIVATE ${PROJECT_SOURCE_DIR}/../jniLibs/${CMAKE_ANDROID_ARCH_ABI})
+
+# Specifies libraries CMake should link to your target library. You
+# can link libraries from various origins, such as libraries defined in this
+# build script, prebuilt third-party libraries, or Android system libraries.
+target_link_libraries(${CMAKE_PROJECT_NAME}
+        # List libraries link to the target library
+        onnxruntime-genai
+        onnxruntime
+        android
+        log)

mobile/examples/phi-3/android/app/src/main/cpp/README.md

@@ -0,0 +1,5 @@
+**Note:**
+
+
+`ort_genai_c.h` file is copied from ORT GenAI package so it matches GenAI libraries.
+We copy it here because Android Studio code completion doesn't work nicely with the C++ code if the file is not in this directory.