Skip to content

Commit 0001046

Browse files
authored
Merge pull request #167 from arm-university/main
Proj and spotlight updates
2 parents 65e7b5a + 530c433 commit 0001046

File tree

4 files changed

+235
-223
lines changed

4 files changed

+235
-223
lines changed

Projects/Projects/Edge-AI-On-Mobile.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -43,17 +43,20 @@ Other devices also support SME2, including both Apple and Android - [see the ful
4343

4444
### Project Summary
4545

46-
Select a **mobile edge AI application** that benefits from large matrix operations, multi-modal fusion, or transformer-based processing enabled by SME2. Build and optimize a proof-of-concept application on a vivo X300 phone or other device supporting SME2.
46+
Select a **mobile edge AI application** that benefits from large matrix operations, multi-modal fusion, or transformer-based processing accelerated by SME2, with real-time speech-to-speech translation, profanity filtering or filler-word removal, and on-device Small Language Models (SLMs) as key example use cases.
4747

4848
Example project areas:
49-
- Real-time video semantic segmentation (e.g., background removal + AR compositing)
50-
- Live object detection + natural-language description (text summary of what the camera sees)
51-
- Multi-sensor fusion (camera + IMU + microphone) for gesture + voice recognition
52-
- On-device lightweight LLM or encoder-only transformer processing for mobile assistants
49+
- Real-time speech-to-speech translation
50+
- On-device SLM for contextual understanding, rewriting, or assistant tasks
51+
- Profanity detection and filtering (masking, rewriting, audio bleeping)
52+
- Filler-word removal / speech clean-up (e.g., removing “um”, “uh”, false starts)
53+
- Real-time video semantic segmentation (e.g., AR captions + background removal)
54+
- Live object detection with natural-language description
55+
- Multi-sensor fusion (camera + IMU + microphone) for gesture + voice interaction
5356

5457
Identify a model architecture that maps to wide matrix operations (e.g., ViT, MLP-Mixer, multi-branch CNN with large FC layers). Utilise a mobile-friendly framework (e.g., ExecuTorch, LiteRT, ONNX Runtime, MediaPipe) to leverage SME2 optimizations. Optimize quantization, memory layout, and verify that the large matrix multiplications get scheduled efficiently on the SME2-enabled CPU. Build a mobile app (Android or iOS) that executes the model and utilises it for a compelling use-case.
5558

56-
Utilise the resources and learning paths below and create an exciting and challenging application. Optionally, you could also compare performance vs a reference phone without SME2.
59+
Utilise the resources and learning paths below and create an exciting and challenging application. As a bonus, you could compare performance vs a reference phone without SME2.
5760

5861
## Resources from Arm and our partners
5962

0 commit comments

Comments
 (0)