Skip to content

Commit 4336f46

Browse files
Merge pull request #2112 from jasonrandrews/review
Review ONNX demo
2 parents a3e099f + 2ca120a commit 4336f46

File tree

2 files changed

+5
-5
lines changed

2 files changed

+5
-5
lines changed

content/learning-paths/servers-and-cloud-computing/onnx/_demo.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,13 @@ title: Run a Phi-4-mini chatbot powered by ONNX Runtime
33
weight: 2
44

55
overview: |
6-
This Learning Path shows you how to use a 32-core Azure Dpls_v6 instance powered by an Arm Neoverse-N2 CPU to build a simple chatbot server that you can then use to provide a chatbot to serve a small number of concurrent users.
6+
This Learning Path shows you how to use a 32-core Azure Dpls_v6 instance powered by an Arm Neoverse N2 CPU to build a simple chatbot that you can use to serve a small number of concurrent users.
77
8-
This architecture is suitable for businesses looking to deploy the latest Generative AI technologies with RAG capabilities using their existing CPU compute capacity and deployment pipelines.
8+
This architecture is suitable for deploying the latest Generative AI technologies with RAG capabilities using their existing CPU compute capacity and deployment pipelines.
99
10-
The demo uses the ONNX runtime, which Arm has enhanced with its own Kleidi technologies. Further optimizations are achieved by using the smaller Phi-4-mini model, which has been optimized at INT4 quantization to minimize memory usage.
10+
The demo uses the ONNX runtime, which Arm has integrated with KleidiAI. Further optimizations are achieved by using the smaller Phi-4-mini model, which has been optimized at INT4 quantization to minimize memory usage.
1111
12-
Chat with the chatbot LLM below to see the performance for yourself, and then follow the Learning Path to build your own Generative AI service on Arm Neoverse.
12+
Chat with the LLM below to see the performance for yourself, and then follow the Learning Path to build your own Generative AI service on Arm Neoverse.
1313
1414
1515
demo_steps:

content/learning-paths/servers-and-cloud-computing/onnx/_index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ who_is_this_for: This is an advanced topic for developers, ML engineers, and clo
77

88
learning_objectives:
99
- Quantize and run the Phi-4-mini model with ONNX Runtime on Azure.
10-
- Analyze performance on Arm Neoverse-N2 based Azure Cobalt 100 VMs.
10+
- Analyze performance on Arm Neoverse N2 based Azure Cobalt 100 VMs.
1111

1212
prerequisites:
1313
- An [Arm-based instance](/learning-paths/servers-and-cloud-computing/csp/) from an appropriate cloud service provider. This Learning Path has been tested on an Azure Cobalt 100 virtual machine.

0 commit comments

Comments
 (0)