Skip to content

Commit 9507d46

Browse files
authored
Merge branch 'main' into main
2 parents f858222 + 2e43b25 commit 9507d46

File tree

65 files changed

+2916
-634
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

65 files changed

+2916
-634
lines changed

.wordlist.txt

Lines changed: 78 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4590,4 +4590,81 @@ xdp
45904590
xhci
45914591
JFR
45924592
conv
4593-
servlet
4593+
servlet
4594+
tv
4595+
gpiozero
4596+
lgpio
4597+
TinyLlama
4598+
Superscalar
4599+
automations
4600+
gemma
4601+
tinyllama
4602+
pinout
4603+
Makatia
4604+
Omusilibwa
4605+
EdgeAI
4606+
Raspi
4607+
abyz
4608+
fidel
4609+
javascript
4610+
makatia
4611+
uk
4612+
STDDEV
4613+
Stdev
4614+
BytesToBytesMap
4615+
HashMap
4616+
LongToUnsafeRowMap
4617+
Stdev
4618+
UnsafeRow
4619+
UnsafeRowhash
4620+
agg
4621+
arrayEqual
4622+
codegen
4623+
hashmap
4624+
hugeMethodLimit
4625+
ints
4626+
kurtosis
4627+
stddev
4628+
wholestage
4629+
RDD
4630+
TEEs
4631+
paravirtualization
4632+
WholeStageCodegen
4633+
Wholestage
4634+
zshrc
4635+
hadoop
4636+
CBL
4637+
DataFrame
4638+
exitCode
4639+
Gerganov's
4640+
Radoslav
4641+
rgerganov
4642+
NSS
4643+
spatio
4644+
upsampling
4645+
UE
4646+
VGF
4647+
NNE
4648+
RDG
4649+
Configurator
4650+
RHI
4651+
RHIs
4652+
NNERuntimeRDGMLExtensionsForVulkan
4653+
Unreal's
4654+
ORT
4655+
MLEmulationLayerForVulkan
4656+
RenderDoc's
4657+
vgf
4658+
dataflow
4659+
Sandboxed
4660+
sandboxed
4661+
Termina
4662+
LXC
4663+
Crostini
4664+
ChromeOS's
4665+
crosh
4666+
Sommelier
4667+
chromeos
4668+
linuxcontainers
4669+
4670+

assets/contributors.csv

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,4 +96,5 @@ William Liang,,,wyliang,,
9696
Waheed Brown,Arm,https://github.com/armwaheed,https://www.linkedin.com/in/waheedbrown/,,
9797
Aryan Bhusari,Arm,,https://www.linkedin.com/in/aryanbhusari,,
9898
Ken Zhang,Insyde,,,,
99-
Ann Cheng,Arm,anncheng-arm,hello-ann,,
99+
Ann Cheng,Arm,anncheng-arm,hello-ann,,
100+
Fidel Makatia Omusilibwa,,,,,

content/install-guides/fm_fvp/fvp.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,15 +11,20 @@ multi_install: false # Set to true if first page of multi-page artic
1111
multitool_install_part: true # Set to true if a sub-page of a multi-page article, else false
1212
layout: installtoolsall # DO NOT MODIFY. Always true for tool install articles
1313
---
14-
Arm Fixed Virtual Platforms (FVPs) are provided as a library of ready to use platforms.
1514

16-
{{% notice Arm Development Tools%}}
17-
An appropriate subset of the FVP library is installed with [Arm Development Studio](/install-guides/armds) and [Keil MDK](/install-guides/mdk) Professional Edition.
15+
{{% notice Note %}}
16+
Arm Fixed Virtual Platforms (FVPs) were available as a library of ready to use platforms (and as a component of Arm Development Studio) up until version 11.28.
17+
18+
From 11.29 onwards the FVPs are provided solely as part of Arm Development Studio.
19+
20+
This install guide is only applicable to the legacy FVP library.
21+
22+
See the [Arm Development Studio Install Guide](/install-guides/armds) and the [Introduction to FVPs](https://developer.arm.com/documentation/110379/1129/Introduction-to-FVPs) documentation.
1823
{{% /notice %}}
1924

2025
## Download installer packages
2126

22-
You can download the FVP library installer from the [Product Download Hub](https://developer.arm.com/downloads/view/FM000A).
27+
You can download the FVP library installer from the [Product Download Hub](https://developer.arm.com/downloads/view/FMFVP).
2328

2429
Linux (AArch64 and x86) and Windows (x86 only) hosts are supported.
2530

@@ -47,11 +52,6 @@ For full list of available options, use:
4752

4853
FVPs are license managed. License setup instructions are available in the [Arm Licensing install guide](/install-guides/license).
4954

50-
{{% notice Arm Development Tools%}}
51-
The FVPs provided with Arm Development Studio and/or Keil MDK Professional Edition use the license of that product, not that of the FVP library.
52-
{{% /notice %}}
53-
54-
5555
## Verify installation
5656

5757
To verify everything is working OK, navigate to the install directory, and launch any of the supplied FVP executables. No additional command options are needed.

content/install-guides/llm-jetson-xavier.md

Lines changed: 0 additions & 69 deletions
This file was deleted.

content/learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/2-env-setup.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ Run the commands below to set up the ExecuTorch internal dependencies:
5050

5151
```bash
5252
git submodule sync
53-
git submodule update --init
53+
git submodule update --init --recursive
5454
./install_executorch.sh
5555
```
5656

Lines changed: 30 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
2-
title: Overview
2+
title: Run LLMs locally on Raspberry Pi 5 for Edge AI
3+
34
weight: 2
45

56
### FIXED, DO NOT MODIFY
@@ -8,66 +9,65 @@ layout: learningpathall
89

910
## Overview
1011

11-
This Learning Path walks you through deploying an efficient large language model (LLM) locally on the Raspberry Pi 5, powered by an Arm Cortex-A76 CPU. This will allow you to control your smart home using natural language, without relying on cloud services. With rapid advances in Generative AI and the power of Arm Cortex-A processors, you can now run advanced language models directly in your home on the Raspberry Pi 5.
12+
This Learning Path walks you through deploying an efficient large language model (LLM) locally on the Raspberry Pi 5, powered by an Arm Cortex-A76 CPU. This setup enables you to control your smart home using natural language without relying on cloud services. With rapid advances in generative AI and the power of Arm Cortex-A processors, you can now run advanced language models directly in your home on the Raspberry Pi 5.
1213

13-
You will create a fully local, privacy-first smart home system that leverages the strengths of Arm Cortex-A architecture. The system can achieve 15+ tokens per second inference speeds using optimized models like TinyLlama and Qwen, while maintaining the energy efficiency that makes Arm processors a good fit for always-on applications.
14+
You will create a fully local, privacy-first smart home system that leverages the strengths of Arm Cortex-A architecture. The system can achieve 15+ tokens per second inference speeds using optimized models like TinyLlama and Qwen, while maintaining the energy efficiency that makes Arm processors well suited for always-on applications.
1415

15-
## Why Arm Cortex-A for Edge AI?
16+
## Why Arm Cortex-A76 makes Raspberry Pi 5 ideal for Edge AI
1617

1718
The Raspberry Pi 5's Arm Cortex-A76 processor can manage high-performance computing tasks like AI inference. Key architectural features include:
1819

19-
- The **superscalar architecture** allows the processor to execute multiple instructions in parallel, improving throughput for compute-heavy tasks.
20-
- **128-bit NEON SIMD support** accelerates matrix and vector operations, which are common in the inner loops of language model inference.
21-
- The **multi-level cache hierarchy** helps reduce memory latency and improves data access efficiency during runtime.
22-
- The **thermal efficiency** enables sustained performance without active cooling, making it ideal for compact or always-on smart home setups.
20+
- **Superscalar architecture**: Executes multiple instructions in parallel, improving throughput for compute-heavy tasks
21+
- **128-bit NEON SIMD support**: Accelerates matrix and vector operations, common in the inner loops of language model inference
22+
- **Multi-level cache hierarchy**: Reduces memory latency and improves data access efficiency during runtime
23+
- **Thermal efficiency**: Enables sustained performance without active cooling, making it ideal for compact or always-on smart home setups
2324

24-
These characteristics make the Raspberry Pi 5 well-suited for workloads like smart home assistants, where responsiveness, efficiency, and local processing are important. Running LLMs locally on Arm-based devices brings several practical benefits. Privacy is preserved, since conversations and routines never leave the device. With optimized inference, the system can offer responsiveness under 100 ms, even on resource-constrained hardware. It remains fully functional in offline scenarios, continuing to operate when internet access is unavailable. Developers also gain flexibility to customize models and automations. Additionally, software updates and an active ecosystem continue to improve performance over time.
25+
These characteristics make the Raspberry Pi 5 well suited for workloads like smart home assistants, where responsiveness, efficiency, and local processing are important. Running LLMs locally on Arm-based devices brings several practical benefits. Privacy is preserved, since conversations and routines never leave the device. With optimized inference, the system can offer responsiveness under 100 ms, even on resource-constrained hardware. It remains fully functional in offline scenarios, continuing to operate when internet access is unavailable. Developers also gain flexibility to customize models and automations. Additionally, software updates and an active ecosystem continue to improve performance over time.
2526

26-
## Arm Ecosystem Advantages
27+
## Leverage the Arm ecosystem for Raspberry Pi Edge AI
2728

2829
For the stack in this setup, Raspberry Pi 5 benefits from the extensive developer ecosystem:
2930

3031
- Optimized compilers including GCC and Clang with Arm-specific enhancements
3132
- Native libraries such as gpiozero and lgpio are optimized for Raspberry Pi
32-
- Community support from open-source projects where developers are contributing Arm-optimized code
33-
- Arm maintains a strong focus on backward compatibility, which reduces friction when updating kernels or deploying across multiple Arm platforms
33+
- Community support from open-source projects where developers contribute Arm-optimized code
34+
- Backward compatibility in Arm architecture reduces friction when updating kernels or deploying across platforms
3435
- The same architecture powers smartphones, embedded controllers, edge devices, and cloud infrastructure—enabling consistent development practices across domains
3536

36-
## Performance Benchmarks on Raspberry Pi 5
37+
## Performance benchmarks on Raspberry Pi 5
3738

3839
The table below shows inference performance for several quantized models running on a Raspberry Pi 5. Measurements reflect single-threaded CPU inference with typical prompt lengths and temperature settings suitable for command-based interaction.
3940

40-
| Model | Tokens/Sec | Avg Latency (ms) |
41+
| Model | Tokens/sec | Avg latency (ms) |
4142
| ------------------- | ---------- | ---------------- |
4243
| qwen:0.5b | 17.0 | 8,217 |
4344
| tinyllama:1.1b | 12.3 | 9,429 |
4445
| deepseek-coder:1.3b | 7.3 | 22,503 |
4546
| gemma2:2b | 4.1 | 23,758 |
4647
| deepseek-r1:7b | 1.6 | 64,797 |
4748

49+
## LLM benchmark insights on Raspberry Pi 5
4850

49-
What does this table tell us? Here are some performance insights:
50-
51-
- Qwen 0.5B and TinyLlama 1.1B deliver fast token generation and low average latency, making them suitable for real-time interactions like voice-controlled smart home commands.
52-
- DeepSeek-Coder 1.3B and Gemma 2B trade off some speed for improved language understanding, which can be useful for more complex task execution or context-aware prompts.
53-
- DeepSeek-R1 7B offers advanced reasoning capabilities with acceptable latency, which may be viable for offline summarization, planning, or low-frequency tasks.
51+
- Qwen 0.5B and TinyLlama 1.1B deliver fast token generation and low average latency, making them suitable for real-time interactions such as voice-controlled smart home commands
52+
- DeepSeek-Coder 1.3B and Gemma 2B trade some speed for improved language understanding, which can be useful for complex tasks or context-aware prompts
53+
- DeepSeek-R1 7B offers advanced reasoning capabilities with acceptable latency, which may be viable for offline summarization, planning, or low-frequency tasks
5454

55-
## Supported Arm-Powered Devices
55+
## Supported Arm-powered devices
5656

57-
This Learning Path focuses on the Raspberry Pi 5, but you can adapt the concepts and code to other Arm-powered devices:
57+
This Learning Path focuses on the Raspberry Pi 5, but you can adapt the concepts and code to other Arm-powered devices.
5858

59-
### Recommended Platforms
59+
## Recommended platforms
6060

61-
| Platform | CPU | RAM | GPIO Support | Model Size Suitability |
62-
|------------------|----------------------------------|----------------|-------------------------------|-----------------------------|
63-
| **Raspberry Pi 5** | Arm Cortex-A76 quad-core @ 2.4GHz | Up to 16GB | Native `lgpio` (high-performance) | Large models (8–16GB) |
64-
| **Raspberry Pi 4** | Arm Cortex-A72 quad-core @ 1.8GHz | Up to 8GB | Compatible with `gpiozero` | Small to mid-size models |
65-
| **Other Arm Devices** | Arm Cortex-A | 4GB min (8GB+ recommended) | Requires physical GPIO pins | Varies by RAM |
61+
| Platform | CPU | RAM | GPIO support | Model size suitability |
62+
| ------------------- | -------------------------------- | -------------- | ------------------------------ | --------------------------- |
63+
| **Raspberry Pi 5** | Arm Cortex-A76 quad-core @ 2.4GHz | Up to 16GB | Native `lgpio` (high-performance) | Large models (8–16GB) |
64+
| **Raspberry Pi 4** | Arm Cortex-A72 quad-core @ 1.8GHz | Up to 8GB | Compatible with `gpiozero` | Small to mid-size models |
65+
| **Other Arm devices** | Arm Cortex-A | 4GB min (8GB+ recommended) | Requires physical GPIO pins | Varies by RAM |
6666

67-
Additionally, the platform must:
67+
Additionally, the platform must meet the following requirements:
6868

6969
- GPIO pins available for hardware control
70-
- Use Python 3.8 or newer
70+
- Python 3.8 or newer
7171
- Ability to run [Ollama](https://ollama.com/)
7272

73-
Continue to the next section to start building a smart home system that highlights how Arm-based processors can enable efficient, responsive, and private AI applications at the edge.
73+
In the next section, you’ll set up the software dependencies needed to start building your privacy-first smart home system on Raspberry Pi 5.

0 commit comments

Comments
 (0)