Skip to content

Commit d90e7d6

Browse files
authored
Merge pull request #2416 from pareenaverma/content_review
Updated to multimodal VA LP
2 parents a8fdea3 + 1e9a32a commit d90e7d6

File tree

3 files changed

+13
-13
lines changed

3 files changed

+13
-13
lines changed

content/learning-paths/mobile-graphics-and-gaming/voice-assistant/2-overview.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -39,15 +39,15 @@ The voice assistant pipeline imports and builds a separate module to provide thi
3939
https://gitlab.arm.com/kleidi/kleidi-examples/speech-to-text
4040
```
4141

42-
and build for various platforms to independently benchmark STT functionality:
42+
You can build the pipeline for various platforms and independently benchmark the STT functionality:
4343

4444
|Platform|Details|
4545
|---|---|
4646
|Linux|x86_64 - KleidiAI is disabled by default, aarch64 - KleidiAI is enabled by default.|
4747
|Android|Cross-compile for an Android device, ensure the Android NDK path is set and correct toolchain file is provided. KleidiAI enabled by default.|
48-
|MacOS|Native or cross-compilation for a Mac device. KleidiAI and SME kernels can be used if available on device.|
48+
|macOS|Native or cross-compilation for a Mac device. KleidiAI and SME kernels can be used if available on device.|
4949

50-
Currently, this module uses [whisper.cpp](https://github.com/ggml-org/whisper.cpp) and wraps the backend library by a thin C++ layer. The module also provides JNI bindings for developers targetting Android based applications.
50+
Currently, this module uses [whisper.cpp](https://github.com/ggml-org/whisper.cpp) and wraps the backend library with a thin C++ layer. The module also provides JNI bindings for developers targeting Android based applications.
5151

5252
{{% notice %}}
5353
You can get more information on how to build and use this module [here](https://gitlab.arm.com/kleidi/kleidi-examples/speech-to-text/-/blob/main/README.md?ref_type=heads)
@@ -67,15 +67,15 @@ The voice assistant pipeline imports and builds a separate module to provide thi
6767
https://gitlab.arm.com/kleidi/kleidi-examples/large-language-models
6868
```
6969

70-
and build for various platforms to independently benchmark LLM functionality:
70+
You can build this pipeline for various platforms and independently benchmark the LLM functionality:
7171

7272
|Platform|Details|
7373
|---|---|
7474
|Linux|x86_64 - KleidiAI is disabled by default, aarch64 - KleidiAI is enabled by default.|
7575
|Android|Cross-compile for an Android device, ensure the Android NDK path is set and correct toolchain file is provided. KleidiAI enabled by default.|
76-
|MacOS|Native or cross-compilation for a Mac device. KleidiAI and SME kernels can be used if available on device.|
76+
|macOS|Native or cross-compilation for a Mac device. KleidiAI and SME kernels can be used if available on device.|
7777

78-
Currently, this module provides a thin C++ layer as well as JNI bindings for developers targetting Android based applications, supported backends are:
78+
Currently, this module provides a thin C++ layer as well as JNI bindings for developers targeting Android based applications, supported backends are:
7979
|Framework|Dependency|Input modalities supported|Output modalities supported|Neural Network|
8080
|---|---|---|---|---|
8181
|llama.cpp|https://github.com/ggml-org/llama.cpp|`image`, `text`|`text`|phi-2,Qwen2-VL-2B-Instruct|
@@ -94,4 +94,4 @@ This part of the application pipeline uses the Android Text-to-Speech API along
9494

9595
In synchronous mode, speech playback begins only after the full LLM response is received. By default, the application operates in asynchronous mode, where speech synthesis starts as soon as a full or partial sentence is ready. Remaining tokens are buffered and processed by the Android Text-to-Speech engine to ensure uninterrupted playback.
9696

97-
You are now familiar with the building blocks of this application and can build these independently for various platforms. You can now build the multi-modal Voice Assistant example which runs on Android OS in the next step.
97+
You are now familiar with the building blocks of this application and can build these independently for various platforms. You can now build the multimodal Voice Assistant example which runs on Android OS in the next step.

content/learning-paths/mobile-graphics-and-gaming/voice-assistant/5-kleidiai.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,5 @@ To disable KleidiAI during build:
3131

3232
KleidiAI simplifies development by abstracting away low-level optimization: developers can write high-level code while the KleidiAI library selects the most efficient implementation at runtime based on the target hardware. This is possible thanks to its deeply optimized micro-kernels tailored for Arm architectures.
3333

34-
As newer versions of the architecture become available, KleidiAI becomes even more powerful: simply updating the library allows applications like the multi-modal Voice Assistant to take advantage of the latest architectural improvements - such as SME2 without requiring any code changes. This means better performance on newer devices with no additional effort from developers.
34+
As newer versions of the architecture become available, KleidiAI becomes even more powerful: simply updating the library allows applications like the multimodal Voice Assistant to take advantage of the latest architectural improvements such as SME2, without requiring any code changes. This means better performance on newer devices with no additional effort from developers.
3535

content/learning-paths/mobile-graphics-and-gaming/voice-assistant/_index.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
---
2-
title: Accelerate multi-modal Voice Assistant performance with KleidiAI and SME2
2+
title: Accelerate multimodal Voice Assistant performance with KleidiAI and SME2
33

44
minutes_to_complete: 30
55

6-
who_is_this_for: This is an introductory topic for developers who want to see a pipeline of a multi-modal Voice Assistant application and accelerate the performance on Android devices using KleidiAI and SME2.
6+
who_is_this_for: This is an introductory topic for developers who want to implement a multimodal pipeline for a Voice Assistant application and accelerate the performance on Android devices using KleidiAI and SME2.
77

88
learning_objectives:
9-
- Learn about the multi-modal Voice Assistant pipeline and different components used.
9+
- Learn about the multimodal Voice Assistant pipeline and different components used.
1010
- Learn about the functionality of ML components used and how these can be built and benchmarked on various platforms.
11-
- Compile and run a multi-modal Voice Assistant example based on Android OS.
12-
- Optimize performance of multi-modal Voice Assistant using KleidiAI and SME2.
11+
- Compile and run a multimodal Voice Assistant example based on Android OS.
12+
- Optimize performance of multimodal Voice Assistant using KleidiAI and SME2.
1313

1414
prerequisites:
1515
- An Android phone that supports the i8mm Arm architecture feature (8-bit integer matrix multiplication). This Learning Path was tested on a Google Pixel 8 Pro.

0 commit comments

Comments
 (0)