Skip to content

Commit a8f43fe

Browse files
committed
shameless plug
1 parent 642868d commit a8f43fe

File tree

1 file changed

+4
-0
lines changed
  • recipes/inference/mobile_inference/android_inference

1 file changed

+4
-0
lines changed

recipes/inference/mobile_inference/android_inference/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,16 @@
11
# Running Llama3 8B Instruct on Android with MLC-LLM
22

3+
Author: Thierry Moreau - [email protected]
4+
35
# Overview
46
In this tutorial we'll learn how to deploy Llama3 8B Instruct on an Android-based phone using MLC-LLM.
57

68
Machine Learning Compilation for Large Language Models (MLC LLM) is a high-performance universal deployment solution that allows native deployment of any large language models with native APIs with compiler acceleration. The mission of this project is to enable everyone to develop, optimize and deploy AI models natively on everyone's devices with ML compilation techniques.
79

810
You can read more about MLC-LLM at the following [link](https://github.com/mlc-ai/mlc-llm).
911

12+
MLC-LLM is also what powers the Llama3 inference APIs provided by [OctoAI](https://octo.ai/). You can use OctoAI for your Llama3 cloud-based inference needs by trying out the examples under the [following path](../../../llama_api_providers/OctoAI_API_examples/).
13+
1014
This tutorial was tested with the following setup:
1115
* MacBook Pro 16 inch from 2021 with Apple M1 Max and 32GB of RAM running Sonoma 14.3.1
1216
* OnePlus 12 Android Smartphone with a Snapdragon 8Gen3 SoC and 12GB or RAM, running OxygenOS 14.0

0 commit comments

Comments
 (0)