Skip to content

Commit d407b67

Browse files
Merge pull request #1939 from jasonrandrews/review
Review and remove draft from ONNX with Phi-3.5 on Cobalt 100
2 parents 3aac54a + 9724dde commit d407b67

File tree

4 files changed

+10
-15
lines changed

4 files changed

+10
-15
lines changed

content/learning-paths/servers-and-cloud-computing/onnx/_index.md

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,5 @@
11
---
2-
title: Deploy Phi-3.5 Vision with ONNX Runtime on Azure Cobalt 100 on Arm
3-
4-
5-
6-
draft: true
7-
cascade:
8-
draft: true
2+
title: Deploy Phi-3.5 Vision with ONNX Runtime on Azure Cobalt 100
93

104
minutes_to_complete: 30
115

@@ -16,7 +10,7 @@ learning_objectives:
1610
- Analyze performance on Arm Neoverse-N2 based Azure Cobalt 100 VMs.
1711

1812
prerequisites:
19-
- An [Arm-based instance](/learning-paths/servers-and-cloud-computing/csp/) from an appropriate cloud service provider. This Learning Path has been tested on a Microsoft Azure Cobalt 100 virtual machine with 32 cores, 8GB of RAM, and 32GB of disk space.
13+
- An [Arm-based instance](/learning-paths/servers-and-cloud-computing/csp/) from an appropriate cloud service provider. This Learning Path has been tested on an Azure Cobalt 100 virtual machine.
2014
- Basic understanding of Python and machine learning concepts.
2115
- Familiarity with ONNX Runtime and Azure cloud services.
2216
- Knowledge of Large Language Model (LLM) fundamentals.

content/learning-paths/servers-and-cloud-computing/onnx/analysis.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,19 +7,22 @@ layout: learningpathall
77

88
## Try a text-only prompt
99

10-
To begin, skip the image prompt and input the text prompt as shown in the example below:
10+
To begin, skip the image prompt by pressing return and then input the text prompt as shown in the example below:
11+
1112
![output](output.png)
1213

1314
Now exit the server.
1415

1516
Next, download a sample image from the internet using the following `wget` command:
17+
1618
```bash
1719
wget https://cdn.pixabay.com/photo/2020/06/30/22/34/dog-5357794__340.jpg
1820
```
1921

2022
## Try an image + text prompt
2123

22-
After downloading the image, provide the image file name when prompted, followed by the text prompt, as demonstrated in the example below:
24+
After downloading the image, run the server again and provide the image file name when prompted, followed by the text prompt, as demonstrated in the example below:
25+
2326
![image_output](image_output.png)
2427

2528
## Observe performance metrics

content/learning-paths/servers-and-cloud-computing/onnx/chatbot.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ layout: learningpathall
77

88
## Create the chatbot server script
99

10-
Create a Python script called `phi3v.py` with the following content.
10+
Create a Python script called `phi3v.py` with the code below.
1111

1212
This script launches a chatbot server using the Phi-3.5 vision model and ONNX Runtime.
1313

content/learning-paths/servers-and-cloud-computing/onnx/setup.md

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,10 @@ In this Learning Path, you'll run quantized Phi models with ONNX Runtime on Micr
1313

1414
Specifically, you'll deploy the Phi-3.5 vision model on Arm-based servers running Ubuntu 24.04 LTS.
1515

16-
1716
{{% notice Note %}}
18-
These instructions have been tested on a 32-core Azure `Dpls_v6` instance.
17+
These instructions have been tested on a 32-core Azure `Dpls_v6` instance with 32 cores, 64GB of RAM, and 32GB of disk space.
1918
{{% /notice %}}
2019

21-
2220
You will learn how to build and configure ONNX Runtime to enable efficient LLM inference on Arm CPUs.
2321

2422
This Learning Path walks you through the following tasks:
@@ -79,7 +77,7 @@ Clone and build the `onnxruntime-genai` repository, which includes the Kleidi AI
7977
Ensure you're using Python 3.12 to match the cp312 wheel format.
8078
{{% /notice %}}
8179

82-
This build includes optimizations from Kleidi AI for efficient inference on Arm CPUs.
80+
This build includes optimizations from KleidiAI for efficient inference on Arm CPUs.
8381

8482
## Download and quantize the model
8583

0 commit comments

Comments
 (0)