You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-onnx.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,15 +18,15 @@ ms.date: 03/18/2024
18
18
19
19
Learn how use of the [Open Neural Network Exchange](https://onnx.ai) (ONNX) can help to optimize the inference of your machine learning model. _Inference_ or _model scoring_, is the process of using a deployed model to generate predictions on production data.
20
20
21
-
Optimizing machine learning models for inference requires you to tune the model and the inference library to make the most of the hardware capabilities. This task becomes complex if you want to get optimal performance on different kinds of platforms such as cloud or edge, CPU or GPU, and so on, since each platform has different capabilities and characteristics. The complexity increases if you have models from various frameworks that need to run on different platforms. It can be time-consuming to optimize all the different combinations of frameworks and hardware. Therefore, a useful solution is to train once in your preferred framework and then run anywhere on the cloud or edge—this solution is where ONNX comes in.
21
+
Optimizing machine learning models for inference requires you to tune the model and the inference library to make the most of the hardware capabilities. This task becomes complex if you want to get optimal performance on different kinds of platforms such as cloud or edge, CPU or GPU, and so on, since each platform has different capabilities and characteristics. The complexity increases if you have models from various frameworks that need to run on different platforms. It can be time-consuming to optimize all the different combinations of frameworks and hardware. Therefore, a useful solution is to train your model once in your preferred framework and then run it anywhere on the cloud or edge—this solution is where ONNX comes in.
22
22
23
23
## What is ONNX?
24
24
25
25
Microsoft and a community of partners created ONNX as an open standard for representing machine learning models. Models from [many frameworks](https://onnx.ai/supported-tools) including TensorFlow, PyTorch, scikit-learn, Keras, Chainer, MXNet, and MATLAB can be exported or converted to the standard ONNX format. Once the models are in the ONNX format, they can be run on various platforms and devices.
26
26
27
-
[ONNX Runtime](https://onnxruntime.ai) is a high-performance inference engine for deploying ONNX models to production. It's optimized for both cloud and edge and works on Linux, Windows, and Mac. While ONNX is written in C++, it also has C, Python, C#, Java, and JavaScript (Node.js) APIs for usage in many environments. ONNX Runtime supports both deep neural networks (DNN) and traditional machine learning models, and it integrates with accelerators on different hardware such as TensorRT on Nvdia GPUs, OpenVINO on Intel processors, and DirectML on Windows. By using ONNX Runtime, you can benefit from the extensive production-grade optimizations, testing, and ongoing improvements.
27
+
[ONNX Runtime](https://onnxruntime.ai) is a high-performance inference engine for deploying ONNX models to production. It's optimized for both cloud and edge and works on Linux, Windows, and Mac. While ONNX is written in C++, it also has C, Python, C#, Java, and JavaScript (Node.js) APIs for usage in many environments. ONNX Runtime supports both deep neural networks (DNN) and traditional machine learning models, and it integrates with accelerators on different hardware such as TensorRT on Nvidia GPUs, OpenVINO on Intel processors, and DirectML on Windows. By using ONNX Runtime, you can benefit from the extensive production-grade optimizations, testing, and ongoing improvements.
28
28
29
-
ONNX Runtime is used in high-scale Microsoft services such as Bing, Office, and Azure AI. Performance gains depend on many factors, but these Microsoft services have seen an __average 2x performance gain on CPU__. In addition to Azure Machine Learning services, ONNX Runtime also runs in other products that support Machine Learning workloads, including:
29
+
ONNX Runtime is used in high-scale Microsoft services such as Bing, Office, and Azure AI. Although performance gains depend on many factors, these Microsoft services report an __average 2x performance gain on CPU__. In addition to Azure Machine Learning services, ONNX Runtime also runs in other products that support Machine Learning workloads, including:
30
30
31
31
-__Windows__: The runtime is built into Windows as part of [Windows Machine Learning](/windows/ai/windows-ml/) and runs on hundreds of millions of devices.
32
32
-__Azure SQL product family__: Run native scoring on data in [Azure SQL Edge](../azure-sql-edge/onnx-overview.md) and [Azure SQL Managed Instance](/azure/azure-sql/managed-instance/machine-learning-services-overview).
@@ -43,7 +43,7 @@ You can obtain ONNX models in several ways:
43
43
- Get a pretrained ONNX model from the [ONNX Model Zoo](https://github.com/onnx/models).
44
44
- Generate a customized ONNX model from [Azure AI Custom Vision service](../ai-services/custom-vision-service/index.yml).
45
45
46
-
Many models, including image classification, object detection, and text processing models can be represented as ONNX models. If you run into an issue with a model that can't be converted successfully, file an issue in the GitHub repository of the respective converter that you used. You can continue using your existing model format until the issue is addressed.
46
+
Many models, including image classification, object detection, and text processing models can be represented as ONNX models. If you run into an issue with a model that can't be converted successfully, file a GitHub issue in the repository of the converter that you used. You can continue using your existing model format until the issue is addressed.
0 commit comments