You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-onnx.md
+11-13Lines changed: 11 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
title: ONNX models
2
+
title: ONNX Runtime and models
3
3
titleSuffix: Azure Machine Learning
4
4
description: Learn how using the Open Neural Network Exchange (ONNX) can help optimize inference of your machine learning models.
5
5
services: machine-learning
@@ -9,27 +9,25 @@ ms.topic: concept-article
9
9
ms.author: mopeakande
10
10
author: msakande
11
11
ms.reviewer: kritifaujdar
12
-
ms.date: 09/27/2024
12
+
ms.date: 09/30/2024
13
13
#customer intent: As a data scientist, I want to learn about ONNX so I can use it to optimize the inference of my machine learning models.
14
14
---
15
15
16
16
# ONNX and Azure Machine Learning
17
17
18
18
This article describes how the [Open Neural Network Exchange (ONNX)](https://onnx.ai) can help optimize the inference of your machine learning models. Inference or model scoring is the process of using a deployed model to generate predictions on production data.
19
19
20
-
Optimizing machine learning models for inference requires you to tune the model and the inference library to make the most of hardware capabilities. This task becomes complex if you want to get optimal performance on different kinds of platforms such as cloud, edge, CPU, or GPU, because each platform has different capabilities and characteristics. The complexity increases if you need to run models from various frameworks on different platforms. It can be time-consuming to optimize all the different combinations of frameworks and hardware.
20
+
Optimizing machine learning models for inference requires you to tune the model and the inference library to make the most of hardware capabilities. This task becomes complex if you want to get optimal performance on different platforms such as cloud, edge, CPU, or GPU, because each platform has different capabilities and characteristics. The complexity increases if you need to run models from various frameworks on different platforms. It can be time-consuming to optimize all the different combinations of frameworks and hardware.
21
21
22
-
A useful solution is to train your model one time in your preferred framework and then run it anywhere on the cloud or edge. ONNX can help with this solution.
22
+
A useful solution is to train your model one time in your preferred framework, and then export or convert it to ONNX so it can run anywhere on the cloud or edge. Microsoft and a community of partners created ONNX as an open standard for representing machine learning models. You can export or convert models from [many frameworks](https://onnx.ai/supported-tools) to the standard ONNX format. Supported frameworks include TensorFlow, PyTorch, scikit-learn, Keras, Chainer, MXNet, and MATLAB. You can run models in the ONNX format on various platforms and devices.
23
23
24
-
Microsoft and a community of partners created ONNX as an open standard for representing machine learning models. You can export or convert models from [many frameworks](https://onnx.ai/supported-tools), including TensorFlow, PyTorch, scikit-learn, Keras, Chainer, MXNet, and MATLAB, to the standard ONNX format. You can run models in the ONNX format on various platforms and devices.
24
+
This ONNX flow diagram shows available frameworksand deployment options.
25
25
26
-
The following ONNX flow diagram shows available frameworks and deployment options.
[ONNX Runtime](https://onnxruntime.ai) is a high-performance inference engine for deploying ONNX models to production. ONNX Runtime is optimized for both cloud and edge, and works on Linux, Windows, and MacOS. ONNX is written in C++, but also has C, Python, C#, Java, and JavaScript (Node.js) APIs for use in those environments.
30
+
[ONNX Runtime](https://onnxruntime.ai) is a high-performance inference engine for deploying ONNX models to production. ONNX Runtime is optimized for both cloud and edge, and works on Linux, Windows, and macOS. ONNX is written in C++, but also has C, Python, C#, Java, and JavaScript (Node.js) APIs to use in those environments.
33
31
34
32
ONNX Runtime supports both deep neural networks (DNN) and traditional machine learning models, and it integrates with accelerators on different hardware such as TensorRT on Nvidia GPUs, OpenVINO on Intel processors, and DirectML on Windows. By using ONNX Runtime, you can benefit from extensive production-grade optimizations, testing, and ongoing improvements.
35
33
@@ -48,11 +46,11 @@ You can obtain ONNX models in several ways:
48
46
- Get a pretrained ONNX model from the [ONNX Model Zoo](https://github.com/onnx/models).
49
47
- Generate a customized ONNX model from [Azure AI Custom Vision service](/azure/ai-services/custom-vision-service/).
50
48
51
-
You can represent many models as ONNX models, including image classification, object detection, and text processing models. If you can't convert your model successfully, file a GitHub issue in the repository of the converter you used.
49
+
You can represent many models as ONNX, including image classification, object detection, and text processing models. If you can't convert your model successfully, file a GitHub issue in the repository of the converter you used.
52
50
53
51
## ONNX model deployment in Azure
54
52
55
-
You can deploy, manage, and monitor your ONNX models with Azure Machine Learning. Use the standard [MLOps deployment workflow](concept-model-management-and-deployment.md) with ONNX Runtime to create a REST endpoint hosted in the cloud.
53
+
You can deploy, manage, and monitor your ONNX models in Azure Machine Learning. Using a standard [MLOps deployment workflow](concept-model-management-and-deployment.md#deploy-models-as-endpoints) with ONNX Runtime, you can create a REST endpoint hosted in the cloud.
56
54
57
55
## Python packages for ONNX Runtime
58
56
@@ -65,7 +63,7 @@ pip install onnxruntime # CPU build
65
63
pip install onnxruntime-gpu # GPU build
66
64
```
67
65
68
-
To call ONNX Runtime in your Python script, use:
66
+
To call ONNX Runtime in your Python script, use the following code:
69
67
70
68
```python
71
69
import onnxruntime
@@ -74,7 +72,7 @@ session = onnxruntime.InferenceSession("path to model")
74
72
75
73
The documentation accompanying the model usually tells you the inputs and outputs for using the model. You can also use a visualization tool such as [Netron](https://github.com/lutzroeder/Netron) to view the model.
76
74
77
-
ONNX Runtime lets you query the model metadata, inputs, and outputs as follows:
75
+
ONNX Runtime lets you query the model metadata, inputs, and outputs, as follows:
0 commit comments