Skip to content

genai-prompt

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

genai-prompt

README.md

Capacitor ML Kit GenAI Prompt Plugin

Unofficial Capacitor plugin for the ML Kit GenAI Prompt API.1

Warning

The ML Kit GenAI APIs are currently in beta. The API surface and behavior may change in future releases.

Use Cases

The GenAI Prompt plugin is typically used to send custom natural-language requests to Gemini Nano on-device, for example:

  • Content generation: Generate short texts such as poems, captions or replies from a custom prompt.
  • Text transformation: Rephrase, classify or extract information from user-provided text.
  • Image understanding: Ask questions about an image by combining an image with a text prompt.

Requirements

This plugin uses Gemini Nano, which runs entirely on-device. It is only available on Android and has the following device requirements:

  • The device must support Gemini Nano (see supported devices).
  • The device must run Android API level 26 or higher.
  • The device must not have an unlocked bootloader.

Use the checkFeatureStatus(...) method to check whether the feature is available on the device.

Important

The Prompt API performs open-ended generation with an on-device model. The output is not moderated. You are responsible for reviewing the model output for your use case (see Responsible AI).

Compatibility

Plugin Version Capacitor Version Status
8.x.x >=8.x.x Active support

Installation

You can use our AI-Assisted Setup to install the plugin. Add the Capawesome Skills to your AI tool using the following command:

npx skills add capawesome-team/skills --skill capacitor-plugins

Then use the following prompt:

Use the `capacitor-plugins` skill from `capawesome-team/skills` to install the `@capacitor-mlkit/genai-prompt` plugin in my project.

If you prefer Manual Setup, install the plugin by running the following commands and follow the platform-specific instructions below:

npm install @capacitor-mlkit/genai-prompt
npx cap sync

Android

API level

This plugin requires a minimum API level of 26. Make sure to set the minSdkVersion in your variables.gradle file to at least 26.

Variables

This plugin will use the following project variables (defined in your app’s variables.gradle file):

  • $mlkitGenaiPromptVersion version of com.google.mlkit:genai-prompt (default: 1.0.0-beta2)

Usage

The following example shows how to check the feature status, download the feature and generate content from a prompt.

The prompt must be under 4,000 tokens (or about 3,000 English words). Use cases that require a long output (more than 256 tokens) should be avoided. The Prompt API is currently only validated for English and Korean.

Check the feature status

Before using the prompt feature, check whether it is available on the device:

import { FeatureStatus, GenAiPrompt } from '@capacitor-mlkit/genai-prompt';

const checkFeatureStatus = async () => {
  const { featureStatus } = await GenAiPrompt.checkFeatureStatus();
  return featureStatus === FeatureStatus.Available;
};

Download the feature

If the feature status is DOWNLOADABLE, download the feature before using it. Add a listener for the downloadProgress event to get notified about the download progress:

import { GenAiPrompt } from '@capacitor-mlkit/genai-prompt';

const downloadFeature = async () => {
  await GenAiPrompt.addListener('downloadProgress', event => {
    console.log('Total bytes downloaded:', event.totalBytesDownloaded);
  });
  await GenAiPrompt.downloadFeature();
};

Generate content

Pass a prompt to generateContent(...) to generate content. Add a listener for the inferenceProgress event to receive partial results as they are generated:

import { GenAiPrompt } from '@capacitor-mlkit/genai-prompt';

const generateContent = async () => {
  await GenAiPrompt.addListener('inferenceProgress', event => {
    console.log('Partial result:', event.text);
  });
  const { text } = await GenAiPrompt.generateContent({
    prompt: 'Write a short poem about the sea.',
  });
  return text;
};

You can also include an image in the prompt for a multimodal (image and text) request and tune the generation with optional parameters:

import { GenAiPrompt } from '@capacitor-mlkit/genai-prompt';

const generateContentFromImage = async () => {
  const { text } = await GenAiPrompt.generateContent({
    prompt: 'What is shown in this image?',
    imagePath: 'file:///path/to/image.jpg',
    temperature: 0.2,
    topK: 16,
    maxOutputTokens: 256,
  });
  return text;
};

API

checkFeatureStatus()

checkFeatureStatus() => Promise<CheckFeatureStatusResult>

Check the current availability status of the prompt feature.

If the status is DOWNLOADABLE, you can download the feature using downloadFeature(...).

Only available on Android.

Returns: Promise<CheckFeatureStatusResult>

Since: 8.2.0

downloadFeature()

downloadFeature() => Promise<void>

Download the prompt feature.

The downloadProgress event listener will notify you about the download progress. The returned promise resolves when the download is complete or the feature is already available.

Only available on Android.

Since: 8.2.0

generateContent(...)

generateContent(options: GenerateContentOptions) => Promise<GenerateContentResult>

Generate content from a custom text-only or multimodal (image and text) prompt.

The inferenceProgress event listener will notify you about partial results as they are generated. The returned promise resolves with the full result.

The output is open-ended generation from an on-device model and is not moderated. The prompt is currently only validated for English and Korean.

Only available on Android.

Param Type
options GenerateContentOptions

Returns: Promise<GenerateContentResult>

Since: 8.2.0

addListener('downloadProgress', ...)

addListener(eventName: 'downloadProgress', listenerFunc: (event: DownloadProgressEvent) => void) => Promise<PluginListenerHandle>

Called while the prompt feature is being downloaded.

Only available on Android.

Param Type
eventName 'downloadProgress'
listenerFunc (event: DownloadProgressEvent) => void

Returns: Promise<PluginListenerHandle>

Since: 8.2.0

addListener('inferenceProgress', ...)

addListener(eventName: 'inferenceProgress', listenerFunc: (event: InferenceProgressEvent) => void) => Promise<PluginListenerHandle>

Called when a new partial result is available during inference.

Only available on Android.

Param Type
eventName 'inferenceProgress'
listenerFunc (event: InferenceProgressEvent) => void

Returns: Promise<PluginListenerHandle>

Since: 8.2.0

removeAllListeners()

removeAllListeners() => Promise<void>

Remove all listeners for this plugin.

Since: 8.2.0

Interfaces

CheckFeatureStatusResult

Prop Type Description Since
featureStatus FeatureStatus The current availability status of the prompt feature. 8.2.0

GenerateContentResult

Prop Type Description Since
text string The generated text. 8.2.0

GenerateContentOptions

Prop Type Description Since
imagePath string The local file path of an image to include in the prompt for a multimodal (image and text) request. 8.2.0
maxOutputTokens number The maximum number of tokens that can be generated in the response. Use cases that require a long output (more than 256 tokens) should be avoided. If not set, the default value of the model is used. 8.2.0
prompt string The prompt to generate content from. The input must be under 4,000 tokens (or about 3,000 English words). 8.2.0
seed number The random seed used during inference. Using the same seed with the same options produces deterministic results. If not set, the default value of the model is used. 8.2.0
temperature number Controls the randomness of the output. Lower values produce more deterministic output, higher values produce more creative output. If not set, the default value of the model is used. 8.2.0
topK number The number of highest-probability tokens that are considered when sampling the next token. If not set, the default value of the model is used. 8.2.0

PluginListenerHandle

Prop Type
remove () => Promise<void>

DownloadProgressEvent

Prop Type Description Since
totalBytesDownloaded number The total number of bytes downloaded so far. 8.2.0

InferenceProgressEvent

Prop Type Description Since
text string The new text of the partial result. 8.2.0

Enums

FeatureStatus

Members Value Description Since
Available 'AVAILABLE' The feature is available to use. 8.2.0
Downloadable 'DOWNLOADABLE' The feature can be downloaded using downloadFeature(...). 8.2.0
Downloading 'DOWNLOADING' The feature is currently being downloaded. 8.2.0
Unavailable 'UNAVAILABLE' The feature is not available on this device. 8.2.0

FAQ

What are the requirements to use this plugin?

The content generation is performed on-device by Gemini Nano. This requires a device that supports Gemini Nano, Android API level 26 or higher and a locked bootloader (see Requirements). Use the checkFeatureStatus(...) method to check whether the feature is available on the device.

Why is this plugin only available on Android?

The ML Kit GenAI APIs are based on Gemini Nano and AICore, which are currently only available on Android. On iOS and Web, all methods reject with an appropriate error.

How large can the prompt be?

The prompt must be under 4,000 tokens (or about 3,000 English words). Use cases that require a long output (more than 256 tokens) should be avoided.

Is the output moderated?

No, the Prompt API performs open-ended generation with an on-device model and the output is not moderated. You are responsible for reviewing the model output for your use case.

How do I get the output as a stream?

Add a listener for the inferenceProgress event before calling generateContent(...). The listener is called with each partial result as it is generated. The promise returned by generateContent(...) resolves with the full result.

Can I use this plugin with Ionic, React, Vue or Angular?

Yes, the plugin is framework-agnostic. It works in any Capacitor app regardless of the web framework, including Ionic with Angular, React, or Vue, as well as plain JavaScript projects.

Related Plugins

Terms & Privacy

This plugin uses the Google ML Kit:

Newsletter

Stay up to date with the latest news and updates about the Capawesome, Capacitor, and Ionic ecosystem by subscribing to our Capawesome Newsletter.

Changelog

See CHANGELOG.md.

License

See LICENSE.

Footnotes

  1. This project is not affiliated with, endorsed by, sponsored by, or approved by Google LLC or any of their affiliates or subsidiaries. ↩