You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/laptops-and-desktops/win_on_arm_build_onnxruntime/1-dev-env-setup.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,7 +20,7 @@ Specifically, you'll learn how to:
20
20
The short-context version accepts shorter (4K) prompts and generates shorter outputs than the long-context (128K) version. It also consumes less memory.
21
21
{{% /notice %}}
22
22
23
-
## Set up your Development Environment
23
+
## Set up your development environment
24
24
25
25
Your first task is to prepare a development environment with the required software.
Copy file name to clipboardExpand all lines: content/learning-paths/laptops-and-desktops/win_on_arm_build_onnxruntime/3-build-onnxruntime-generate-api.md
+15-11Lines changed: 15 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,21 +6,23 @@ weight: 4
6
6
layout: learningpathall
7
7
---
8
8
9
-
## Compile the ONNX Runtime Generate() API for Windows on Arm
9
+
## Build the ONNX Runtime Generate() API for Windows on Arm
10
10
11
11
The Generate() API in ONNX Runtime is designed for text generation tasks using models like Phi-3. It implements the generative AI loop for ONNX models, including:
12
12
- Pre- and post-processing.
13
-
- Inference with ONNX Runtime- logits processing.
13
+
- Inference with ONNX Runtime (including logits processing).
14
14
- Search and sampling.
15
15
- KV cache management.
16
16
17
-
You can learn more by reading the [ONNX Runtime Generate() API page](https://onnxruntime.ai/docs/genai/).
17
+
{{% notice Learning Tip %}}
18
+
You can learn more about this area by reading the [ONNX Runtime Generate() API documentation](https://onnxruntime.ai/docs/genai/).
19
+
{{% /notice %}}
18
20
19
-
In this section you will learn how to build the Generate API() from source.
21
+
In this section, you'll build the Generate API() from source.
20
22
21
23
22
-
### Clone onnxruntime-genai Repo
23
-
Within your Windows Developer Command Prompt for Visual Studio, checkout the source repo:
24
+
### Clone the onnxruntime-genai repository
25
+
From your **Windows Developer Command Prompt for Visual Studio**, clone the repository and checkout the following tested commit:
24
26
25
27
```bash
26
28
cd C:\Users\%USERNAME%
@@ -35,18 +37,20 @@ You might be able to use later commits. These steps have been tested with the co
35
37
{{% /notice %}}
36
38
37
39
### Build for Windows on Arm
38
-
The build command below has a ---config argument, which takes the following options:
39
-
-```Release``` builds release build
40
-
-```Debug``` builds binaries with debug symbols
41
-
-```RelWithDebInfo``` builds release binaries with debug info
40
+
The build script uses a ---config argument, which supports the following options:
41
+
-```Release``` builds release build.
42
+
-```Debug``` builds binaries with debug symbols.
43
+
-```RelWithDebInfo``` builds release binaries with debug info.
42
44
43
-
You will build the `Release` variant of the ONNX Runtime Generate() API:
45
+
To build the `Release` variant of the ONNX Runtime Generate() API:
44
46
45
47
```bash
46
48
pip install requests
47
49
python build.py --config Release --skip_tests
48
50
```
49
51
52
+
### Verify the output
53
+
50
54
When the build is complete, confirm the ONNX Runtime Generate() API Dynamic Link Library has been created:
0 commit comments