You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> **⚠️ Important:** Complete platform-specific setup before using the plugin.
124
+
125
+
1. **Download Model and optionally LoRA Weights:** Obtain a pre-trained Gemma model (recommended: 2b or 2b-it) [from Kaggle](https://www.kaggle.com/models/google/gemma/frameworks/tfLite/)
126
+
* For **multimodal support**, download [Gemma 3 Nano models](https://huggingface.co/google/gemma-3n-E2B-it-litert-preview) or [Gemma 3 Nano in LitertLM format](https://huggingface.co/google/gemma-3n-E2B-it-litert-lm) that support vision input
127
+
* Optionally, [fine-tune a model for your specific use case]( https://www.kaggle.com/code/juanmerinobermejo/llm-pr-fine-tuning-with-gemma-2b?scriptVersionId=169776634)
128
+
* If you have LoRA weights, you can use them to customize the model's behavior without retraining the entire model.
129
+
* [There is an article that described all approaches](https://medium.com/@denisov.shureg/fine-tuning-gemma-with-lora-for-on-device-inference-android-ios-web-with-separate-lora-weights-f05d1db30d86)
130
+
2. **Platform specific setup:**
131
+
132
+
**iOS**
133
+
134
+
* **Set minimum iOS version** in `Podfile`:
135
+
```ruby
136
+
platform :ios, '16.0' # Required for MediaPipe GenAI
137
+
```
138
+
139
+
* **Enable file sharing** in `Info.plist`:
140
+
```plist
141
+
<key>UIFileSharingEnabled</key>
142
+
<true/>
143
+
```
144
+
145
+
* **Add network access description** in `Info.plist` (for development):
146
+
```plist
147
+
<key>NSLocalNetworkUsageDescription</key>
148
+
<string>This app requires local network access for model inference services.</string>
149
+
```
150
+
151
+
* **Enable performance optimization** in `Info.plist` (optional):
152
+
```plist
153
+
<key>CADisableMinimumFrameDurationOnPhone</key>
154
+
<true/>
155
+
```
156
+
157
+
* **Add memory entitlements** in `Runner.entitlements` (for large models):
158
+
```xml
159
+
<?xml version="1.0" encoding="UTF-8"?>
160
+
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
* If you want to use a GPU to work with the model, you need to add OpenGL support in the manifest.xml. If you plan to use only the CPU, you can skip this step.
202
+
203
+
Add to 'AndroidManifest.xml' above tag `</application>`
* **For release builds with ProGuard/R8 enabled**, the plugin automatically includes necessary ProGuard rules. If you encounter issues with `UnsatisfiedLinkError` or missing classes in release builds, ensure your `proguard-rules.pro` includes:
214
+
215
+
```proguard
216
+
# MediaPipe
217
+
-keep class com.google.mediapipe.** { *; }
218
+
-dontwarn com.google.mediapipe.**
219
+
220
+
# Protocol Buffers
221
+
-keep class com.google.protobuf.** { *; }
222
+
-dontwarn com.google.protobuf.**
223
+
224
+
# RAG functionality
225
+
-keep class com.google.ai.edge.localagents.** { *; }
226
+
-dontwarn com.google.ai.edge.localagents.**
227
+
```
228
+
229
+
**Web**
230
+
231
+
* **Authentication:** For gated models (Gemma 3 Nano, Gemma 3 1B/270M), you need to configure HuggingFace token. See [HuggingFace Authentication](#huggingface-authentication) section.
232
+
* Web currently works only GPU backend models, CPU backend models are not supported by MediaPipe yet
233
+
* **Multimodal support** (images) is fully supported on web platform
234
+
* **Model formats**: Use `.litertlm` files for optimal web compatibility (recommended for multimodal models)
235
+
236
+
* Add dependencies to `index.html` file in web folder
237
+
```html
238
+
<script type="module">
239
+
import { FilesetResolver, LlmInference } from 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-genai@0.10.25';
240
+
window.FilesetResolver = FilesetResolver;
241
+
window.LlmInference = LlmInference;
242
+
</script>
243
+
```
244
+
121
245
## Quick Start
122
246
247
+
> **⚠️ Important:** Complete [platform setup](#setup) before running this code.
**Important:** On web, FileSource only works with URLs or asset paths, not local file system paths.
553
679
554
-
## Setup
555
-
556
-
1. **Download Model and optionally LoRA Weights:** Obtain a pre-trained Gemma model (recommended: 2b or 2b-it) [from Kaggle](https://www.kaggle.com/models/google/gemma/frameworks/tfLite/)
557
-
* For **multimodal support**, download [Gemma 3 Nano models](https://huggingface.co/google/gemma-3n-E2B-it-litert-preview) or [Gemma 3 Nano in LitertLM format](https://huggingface.co/google/gemma-3n-E2B-it-litert-lm) that support vision input
558
-
* Optionally, [fine-tune a model for your specific use case]( https://www.kaggle.com/code/juanmerinobermejo/llm-pr-fine-tuning-with-gemma-2b?scriptVersionId=169776634)
559
-
* If you have LoRA weights, you can use them to customize the model's behavior without retraining the entire model.
560
-
* [There is an article that described all approaches](https://medium.com/@denisov.shureg/fine-tuning-gemma-with-lora-for-on-device-inference-android-ios-web-with-separate-lora-weights-f05d1db30d86)
561
-
2. **Platform specific setup:**
562
-
563
-
**iOS**
564
-
565
-
* **Set minimum iOS version** in `Podfile`:
566
-
```ruby
567
-
platform :ios, '16.0' # Required for MediaPipe GenAI
568
-
```
569
-
570
-
* **Enable file sharing** in `Info.plist`:
571
-
```plist
572
-
<key>UIFileSharingEnabled</key>
573
-
<true/>
574
-
```
575
-
576
-
* **Add network access description** in `Info.plist` (for development):
577
-
```plist
578
-
<key>NSLocalNetworkUsageDescription</key>
579
-
<string>This app requires local network access for model inference services.</string>
580
-
```
581
-
582
-
* **Enable performance optimization** in `Info.plist` (optional):
583
-
```plist
584
-
<key>CADisableMinimumFrameDurationOnPhone</key>
585
-
<true/>
586
-
```
587
-
588
-
* **Add memory entitlements** in `Runner.entitlements` (for large models):
589
-
```xml
590
-
<?xml version="1.0" encoding="UTF-8"?>
591
-
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
* **Change the linking type** of pods to static in `Podfile`:
605
-
```ruby
606
-
use_frameworks! :linkage => :static
607
-
```
608
-
609
-
**Android**
610
-
611
-
* If you want to use a GPU to work with the model, you need to add OpenGL support in the manifest.xml. If you plan to use only the CPU, you can skip this step.
612
-
613
-
Add to 'AndroidManifest.xml' above tag `</application>`
* **For release builds with ProGuard/R8 enabled**, the plugin automatically includes necessary ProGuard rules. If you encounter issues with `UnsatisfiedLinkError` or missing classes in release builds, ensure your `proguard-rules.pro` includes:
624
-
625
-
```proguard
626
-
# MediaPipe
627
-
-keep class com.google.mediapipe.** { *; }
628
-
-dontwarn com.google.mediapipe.**
629
-
630
-
# Protocol Buffers
631
-
-keep class com.google.protobuf.** { *; }
632
-
-dontwarn com.google.protobuf.**
633
-
634
-
# RAG functionality
635
-
-keep class com.google.ai.edge.localagents.** { *; }
636
-
-dontwarn com.google.ai.edge.localagents.**
637
-
```
638
-
639
-
**Web**
640
-
641
-
* **Authentication:** For gated models (Gemma 3 Nano, Gemma 3 1B/270M), you need to configure HuggingFace token. See [HuggingFace Authentication](#huggingface-authentication) section.
642
-
* Web currently works only GPU backend models, CPU backend models are not supported by MediaPipe yet
643
-
* **Multimodal support** (images) is fully supported on web platform
644
-
* **Model formats**: Use `.litertlm` files for optimal web compatibility (recommended for multimodal models)
645
-
646
-
* Add dependencies to `index.html` file in web folder
647
-
```html
648
-
<script type="module">
649
-
import { FilesetResolver, LlmInference } from 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-genai@0.10.25';
650
-
window.FilesetResolver = FilesetResolver;
651
-
window.LlmInference = LlmInference;
652
-
</script>
653
-
```
654
-
655
680
## Migration from Legacy to Modern API 🔄
656
681
657
682
If you're upgrading from the Legacy API, here are common migration patterns:
0 commit comments