You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jun 3, 2025. It is now read-only.
Copy file name to clipboardExpand all lines: src/content/get-started/install.mdx
+7-5Lines changed: 7 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,19 +7,21 @@ index: 0
7
7
8
8
# Installation
9
9
10
-
The Deep Sparse Platform is made up of core libraries that are available as Python APIs and CLIs.
10
+
The Neural Magic Platform is made up of core libraries that are available as Python APIs and CLIs.
11
11
All Python APIs and CLIs are installed through pip utilizing [PyPI](https://pypi.org/user/neuralmagic/).
12
-
It is recommended to install in a [virtual environment](https://docs.python.org/3/library/venv.html) to encapsulate your local environment.
12
+
We recommend you install in a [virtual environment](https://docs.python.org/3/library/venv.html) to encapsulate your local environment.
13
13
14
-
## Quick Start
14
+
## Installing the Neural Magic Platform
15
15
16
-
To begin using the Deep Sparse Platform, run the following commands which install standard setups for deployment with the [DeepSparse Engine](../../products/deepsparse) and model training/optimization with [SparseML](../../products/sparseml):
16
+
To begin using the Neural Magic Platform, run the following command, which installs standard setups for deployment with [DeepSparse](../../products/deepsparse) and model training/optimization with [SparseML](../../products/sparseml):
Copy file name to clipboardExpand all lines: src/content/get-started/install/deepsparse-ent.mdx
+4-7Lines changed: 4 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,19 +13,15 @@ The engine accepts models in the open-source [ONNX format](https://onnx.ai/), wh
13
13
Currently, DeepSparse is tested on Python 3.7-3.10, ONNX 1.5.0-1.10.1, ONNX opset version 11+ and is [manylinux compliant](https://peps.python.org/pep-0513/).
14
14
It is limited to Linux systems running on x86 CPU architectures.
15
15
16
-
The DeepSparse Engine is available in two editions:
17
-
1.[**The Community Edition**](/products/deepsparse) is open-source and free for evaluation, research, and non-production use with our [Engine Community License](https://neuralmagic.com/legal/engine-license-agreement/).
18
-
2.[**The Enterprise Edition**](/products/deepsparse-ent) requires a Trial License or [can be fully licensed](https://neuralmagic.com/legal/master-software-license-and-service-agreement/) for production, commercial applications.
19
-
20
-
## General Install
16
+
## Installing DeepSparse Enterprise
21
17
22
18
Use the following command to install with pip:
23
19
24
20
```bash
25
21
pip install deepsparse-ent
26
22
```
27
23
28
-
## Server Install
24
+
## Installing the Server
29
25
30
26
The [DeepSparse Server](/use-cases/deploying-deepsparse/deepsparse-server) allows you to serve models and pipelines through an HTTP interface using the deepsparse.server CLI.
31
27
To install, use the following extra option:
@@ -34,7 +30,7 @@ To install, use the following extra option:
34
30
pip install deepsparse-ent[server]
35
31
```
36
32
37
-
## YOLO Install
33
+
## Installing YOLO
38
34
39
35
The [Ultralytics YOLOv5](/use-cases/object-detection/deploying) models require extra dependencies for deployment.
40
36
To use YOLO models, install with the following extra option:
@@ -43,3 +39,4 @@ To use YOLO models, install with the following extra option:
43
39
pip install deepsparse-ent[yolo] # just yolo requirements
44
40
pip install deepsparse-ent[yolo,server] # both yolo + server requirements
Copy file name to clipboardExpand all lines: src/content/get-started/install/deepsparse.mdx
+6-9Lines changed: 6 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,22 +10,18 @@ index: 1000
10
10
The [DeepSparse Engine](/products/deepsparse) enables GPU-class performance on CPUs, leveraging sparsity within models to reduce FLOPs and the unique cache hierarchy on CPUs to reduce memory movement.
11
11
The engine accepts models in the open-source [ONNX format](https://onnx.ai/), which are easily created from PyTorch and TensorFlow models.
12
12
13
-
Currently, DeepSparse is tested on Python 3.7-3.10, ONNX 1.5.0-1.10.1, ONNX opset version 11+ and is [manylinux compliant](https://peps.python.org/pep-0513/).
14
-
It is limited to Linux systems running on x86 CPU architectures.
13
+
Currently, DeepSparse is tested on Python 3.7-3.10, ONNX 1.5.0-1.10.1, and ONNX opset version 11+. It is [manylinux compliant](https://peps.python.org/pep-0513/).
14
+
DeepSparse is limited to Linux systems running on x86 CPU architectures.
15
15
16
-
The DeepSparse Engine is available in two editions:
17
-
1.[**The Community Edition**](/products/deepsparse) is open-source and free for evaluation, research, and non-production use with our [Engine Community License](https://neuralmagic.com/legal/engine-license-agreement/).
18
-
2.[**The Enterprise Edition**](/products/deepsparse-ent) requires a Trial License or [can be fully licensed](https://neuralmagic.com/legal/master-software-license-and-service-agreement/) for production, commercial applications.
19
-
20
-
## General Installation
16
+
## Installing DeepSparse Community
21
17
22
18
Use the following command to install the Community Edition with pip:
23
19
24
20
```bash
25
21
pip install deepsparse
26
22
```
27
23
28
-
## Server Install
24
+
## Installing the Server
29
25
30
26
The [DeepSparse Server](/use-cases/deploying-deepsparse/deepsparse-server) allows you to serve models and pipelines through an HTTP interface using the deepsparse.server CLI.
31
27
To install, use the following extra option:
@@ -34,7 +30,7 @@ To install, use the following extra option:
34
30
pip install deepsparse[server]
35
31
```
36
32
37
-
## YOLO Install
33
+
## Installing YOLO
38
34
39
35
The [Ultralytics YOLOv5](/use-cases/object-detection/deploying) models require extra dependencies for deployment.
40
36
To use YOLO models, install with the following extra option:
@@ -43,3 +39,4 @@ To use YOLO models, install with the following extra option:
43
39
pip install deepsparse[yolo] # just yolo requirements
44
40
pip install deepsparse[yolo,server] # both yolo + server requirements
Copy file name to clipboardExpand all lines: src/content/get-started/try-a-model.mdx
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,9 +10,9 @@ index: 2000
10
10
DeepSparse Engine supports fast inference on CPUs for sparse and dense models. For sparse models in particular, it achieves GPU-level performance in many use cases.
11
11
12
12
Around the engine, the DeepSparse package includes various utilities to simplify benchmarking performance and model deployment. For instance:
13
-
1. Trained models are passed in the open ONNX file format, enabling easy exporting from common packages like PyTorch, Keras, and TensorFlow.
14
-
2. Benchmaking latency and performance is available via a single CLI call, with various arguments to test scenarios.
15
-
3.`Pipelines` utilities wrap the model execution with input pre-processing and output post-processing, simplifying deployment and adding functionality like multi-stream, bucketing and dynamic shape.
13
+
- Trained models are passed in the open ONNX file format, enabling easy exporting from common packages like PyTorch, Keras, and TensorFlow.
14
+
- Benchmaking latency and performance is available via a single CLI call, with various arguments to test scenarios.
15
+
-Pipelines utilities wrap the model execution with input pre-processing and output post-processing, simplifying deployment and adding functionality like multi-stream, bucketing, and dynamic shape.
16
16
17
17
## Use Case Examples
18
18
@@ -35,4 +35,4 @@ The examples below walk through use cases leveraging DeepSparse for testing and
35
35
## Other Use Cases
36
36
37
37
More documentation, models, use cases, and examples are continually being added.
38
-
If you don't see one you're interested in, search the [DeepSparse Github repo](https://github.com/neuralmagic/deepsparse), the [SparseML Github repo](https://github.com/neuralmagic/sparseml), the[SparseZoo website](https://sparsezoo.neuralmagic.com/), or ask in the [Neural Magic Slack](https://join.slack.com/t/discuss-neuralmagic/shared_invite/zt-q1a1cnvo-YBoICSIw3L1dmQpjBeDurQ).
38
+
If you don't see one you're interested in, search the [DeepSparse Github repo](https://github.com/neuralmagic/deepsparse), [SparseML Github repo](https://github.com/neuralmagic/sparseml), or[SparseZoo website](https://sparsezoo.neuralmagic.com/). Or, ask in the [Neural Magic Slack](https://join.slack.com/t/discuss-neuralmagic/shared_invite/zt-q1a1cnvo-YBoICSIw3L1dmQpjBeDurQ).
Copy file name to clipboardExpand all lines: src/content/get-started/try-a-model/custom-use-case.mdx
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,19 +9,19 @@ index: 3000
9
9
10
10
This page explains how to run a model on the DeepSparse Engine for a custom task inside a Python API called `Pipelines.`
11
11
12
-
`Pipelines`wrap key utilities around the DeepSparse Engine for easy testing and deployment.
12
+
`Pipelines`wraps key utilities around the DeepSparse Engine for easy testing and deployment.
13
13
14
14
The DeepSparse Engine supports many operators within ONNX, enabling performance for most models and use cases outside of the ones available on the SparseZoo.
15
-
The `CustomTaskPipeline` enables you to wrap your model with custom pre and post-processing functions for simple deployment and benchmarking.
15
+
The `CustomTaskPipeline` enables you to wrap your model with custom pre-processing and post-processing functions for simple deployment and benchmarking.
16
16
In this way, the simplicity of `Pipelines` is combined with the performance of DeepSparse for arbitrary use cases.
17
17
18
-
## Install Requirements
18
+
## Installation Requirements
19
19
20
-
This example requires [DeepSparse General Install](/get-started/install/deepsparse) and [SparseML Torchvision Install](/get-started/install/sparseml).
20
+
This example requires [DeepSparse General Installation](/get-started/install/deepsparse) and [SparseML Torchvision Installation](/get-started/install/sparseml).
21
21
22
22
## Model Setup
23
23
24
-
For custom model deployment, first export your model to the ONNX model format (create a `model.onnx` file).
24
+
For custom model deployment, export your model to the ONNX model format (create a `model.onnx` file).
25
25
SparseML has available wrappers for ONNX export classes and APIs for a more straightforward export process.
26
26
A sample export utilizing this API for a MobileNetV2 TorchVision model is given below.
27
27
@@ -41,15 +41,15 @@ Examples for both are given below.
41
41
42
42
## Inference Pipelines
43
43
44
-
The `model.onnx` file can be passed into a DeepSparse `CustomTaskPipeline` utilizing the `model_path` argument alongside optional pre and post-processing functions.
44
+
The `model.onnx` file can be passed into a DeepSparse `CustomTaskPipeline` utilizing the `model_path` argument alongside optional pre-processing and post-processing functions.
45
45
46
46
A sample image is downloaded that will be run through the example to test the `Pipeline`.
Copy file name to clipboardExpand all lines: src/content/get-started/try-a-model/cv-object-detection.mdx
+18-18Lines changed: 18 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,33 +9,33 @@ index: 2000
9
9
10
10
This page explains how to run a trained model on the DeepSparse Engine for Object Detection inside a Python API called `Pipelines.`
11
11
12
-
`Pipelines`wrap key utilities around the DeepSparse Engine for easy testing and deployment.
12
+
`Pipelines`wraps key utilities around the DeepSparse Engine for easy testing and deployment.
13
13
14
-
The object detection `Pipeline`, for example, wraps a trained model with the proper preprocessing and postprocessing pipelines such as NMS.
15
-
This enables the passing of raw images and receiving the bounding boxes from the DeepSparse Engine without any extra effort.
16
-
With all of this built on top of the DeepSparse Engine, the simplicity of `Pipelines` is combined with GPU-class performance on CPUs for sparse models.
14
+
The object detection `Pipeline`, for example, wraps a trained model with the proper pre-processing and post-processing pipelines such as NMS.
15
+
This enables the passing of raw images and receiving the bounding boxes from DeepSparse without any extra effort.
16
+
With all of this built on top of DeepSparse, the simplicity of `Pipelines` is combined with GPU-class performance on CPUs for sparse models.
17
17
18
-
## Install Requirements
18
+
## Installation Requirements
19
19
20
-
This example requires [DeepSparse YOLO Install](/get-started/install/deepsparse).
20
+
This example requires [DeepSparse YOLO Installation](/get-started/install/deepsparse).
21
21
22
22
## Model Setup
23
23
24
24
The object detection `Pipeline` uses Ultralytics YOLOv5 standards and configurations for model setup.
25
-
The possible files/variables that can be passed in are the following:
26
-
-`model.onnx` - The exported YOLOv5 model in the ONNX format.
27
-
-`model.yaml` - The Ultralytics model config file containing configuration information about the model and its post-processing.
25
+
The possible files/variables that can be passed in are:
26
+
-`model.onnx` - Exported YOLOv5 model in the ONNX format.
27
+
-`model.yaml` - Ultralytics model configuration file containing configuration information about the model and its post-processing.
28
28
-`class_names` - A list, dictionary, or file containing the index to class name mappings for the trained model.
29
29
30
30
`model.onnx` is the only required file.
31
-
The pipeline will default to a standard setup for the COCO dataset if the model config file or class names are not provided.
31
+
The pipeline will default to a standard setup for the COCO dataset if the model configuration file or class names are not provided.
32
32
33
33
There are two options for passing these files to DeepSparse:
34
34
35
35
<details>
36
-
<summary><b>1) Using The SparseZoo</b></summary>
36
+
<summary><b>1) Using the SparseZoo</b></summary>
37
37
38
-
This pathway is relevant if you want to use a pre-sparsified state-of-the-art model off the shelpf.
38
+
This pathway is relevant if you want to use a pre-sparsified state-of-the-art model off the shelf.
39
39
40
40
SparseZoo is a repository of pre-trained and pre-sparsified models. DeepSparse supports SparseZoo stubs as inputs for automatic download and inclusion into easy testing and deployment.
41
41
These models include dense and sparsified versions of YOLOv5 trained on the COCO dataset for performant and general detection, among others.
@@ -54,12 +54,12 @@ These SparseZoo stubs can be passed as arguments to the `Pipeline` constructor i
54
54
</details>
55
55
56
56
<details>
57
-
<summary><b>2) Using a Custom Local Model</b></summary>
57
+
<summary><b>2) Using a custom local model</b></summary>
58
58
59
59
This pathway is relevant if you want to use a model fine-tuned on your data with SparseML or a custom model.
60
60
61
61
There are three steps to using a local model with `Pipelines`:
62
-
1. Create the `model.onnx` file (if you trained with SparseML, use the [ONNX export script](https://github.com/neuralmagic/sparseml/tree/main/integrations/ultralytics-yolov5#exporting-the-sparse-model-to-onnx))
62
+
1. Create the `model.onnx` file (if you trained with SparseML, use the [ONNX export script](https://github.com/neuralmagic/sparseml/tree/main/integrations/ultralytics-yolov5#exporting-the-sparse-model-to-onnx)).
63
63
2. Collect the `model.yaml` file and `class_names` listed above.
64
64
3. Pass the local paths of the files in place of the SparseZoo stubs.
65
65
@@ -69,9 +69,9 @@ The examples below use the SparseZoo stubs. Pass the path to the local model in
69
69
70
70
## Inference Pipelines
71
71
72
-
With the object detection model setup, it can then be passed into a DeepSparse `Pipeline` utilizing the `model_path` argument.
72
+
With the object detection model set up, the model can be passed into a DeepSparse `Pipeline` utilizing the `model_path` argument.
73
73
The SparseZoo stub for the sparse-quantized YOLOv5l model given at the beginning is used in the sample code below.
74
-
It will automatically download the necessary files for the model from the SparseZoo and then compile them on your local machine in the DeepSparse engine.
74
+
It will automatically download the necessary files for the model from the SparseZoo and then compile them on your local machine in DeepSparse.
75
75
Once compiled, the model `Pipeline` is ready for inference with images.
76
76
77
77
First, a sample image is downloaded that will be run through the example to test the pipeline.
@@ -80,7 +80,7 @@ First, a sample image is downloaded that will be run through the example to test
0 commit comments