Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Commit d24a017

Browse files
authored
updates: FAQs, taglines (#7)
* Add files via upload: thumbnail for video as GH can't embed videos * Provided updated faqs, project taglines
1 parent db0073f commit d24a017

File tree

3 files changed

+24
-11
lines changed

3 files changed

+24
-11
lines changed

docs/source/faqs.md

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,7 @@ The DeepSparse Engine, created by Neural Magic, is a general purpose engine for
3030

3131
Learn more about Neural Magic and the DeepSparse Engine (formerly known as the Neural Magic Inference Engine).
3232

33-
<iframe width="560" height="315" src="https://www.youtube.com/embed/zJy_8uPZd0o" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen>
34-
</iframe>
33+
[![Watch the Why Neural Magic video](why-nm.jpg)](https://youtu.be/zJy_8uPZd0o)
3534

3635
**How does Neural Magic make it work?**
3736

@@ -53,7 +52,7 @@ We are open to opportunities to expand our support footprint for different CPU-b
5352

5453
**Do you run on ARM architecture?**
5554

56-
We currently do not support ARM and it’s on the Neural Magic roadmap; however, we’d still like to hear your use cases[Contact us to continue the conversation](https://neuralmagic.com/contact/).
55+
We currently do not support ARM and it’s on the Neural Magic roadmap; however, we’d still like to hear your use cases. [Contact us to continue the conversation](https://neuralmagic.com/contact/).
5756

5857
**To what use cases is the Deep Sparse Platform best suited?**
5958

@@ -63,6 +62,14 @@ We focus on the models and use cases related to computer vision due to cost sens
6362

6463
Today, we offer support for CNN-based computer vision models, specifically classification and object detection model types. We are continuously adding models to [our supported model list and SparseZoo](https://docs.neuralmagic.com/sparsezoo). Additionally, we are investigating model architectures beyond computer vision such as NLP models like BERT.
6564

65+
**Is dynamic shape supported?**
66+
67+
Dynamic shape is currently not supported; be sure to use models with fixed inputs and compile the model for a particular batch size. Dynamic shape and dynamic batch sizes are on the Neural Magic roadmap; [subscribe for updates](https://neuralmagic.com/subscribe/).
68+
69+
**Can multiple model inferences be executed?**
70+
71+
Model inferences are executed as a single stream; concurrent execution is unsupported at this time.
72+
6673
___
6774

6875
## Benchmarking FAQs
@@ -73,7 +80,7 @@ This will vary but, in some cases, we are seeing 3x-10x savings. Typically, we o
7380

7481
**Do you have benchmarks to compare and contrast?**
7582

76-
Yes. Check out our [benchmark demo video](https://neuralmagic.com/blog/neural-magic-demo/) or [contact us to](https://neuralmagic.com/contact/) discuss your particular performance requirements. If you’d rather observe performance for yourself, [head over to the Neural Magic GitHub repo](https://github.com/neuralmagic) to check out our tools and generate your own benchmarks in your environment.
83+
Yes. Check out our [benchmark demo video](https://neuralmagic.com/blog/neural-magic-demo/) or [contact us](https://neuralmagic.com/contact/) to discuss your particular performance requirements. If you’d rather observe performance for yourself, [head over to the Neural Magic GitHub repo](https://github.com/neuralmagic) to check out our tools and generate your own benchmarks in your environment.
7784

7885
**Do you publish ML Perf inference benchmarks?**
7986

@@ -83,11 +90,15 @@ ___
8390

8491
## Infrastructure FAQs
8592

93+
**Which instruction sets are supported and do we have to enable certain settings?**
94+
95+
AVX-512, AVX2, and VNNI. The DeepSparse Engine will automatically utilize the most effective available instruction set for the task. Generally, if AVX-512 is available then we have no reason to use AVX2 instruction set. AVX-512 VNNI only comes into use for quantized models i.e., INT8 or UINT8.
96+
8697
**Are you suitable for edge deployments (i.e., in-store devices, cameras)?**
8798

8899
Yes, absolutely. We can run anywhere you have a CPU with x86 instructions, including on bare metal, in the cloud, on-prem, or at the edge. Additionally, our model optimization tools are able to reduce the footprint of models across all architectures. We only guarantee performance in the DeepSparse Engine.
89100

90-
We’d love to hear from users highly interested in ML performance. If you want to chat about your use cases or how others are leveraging the Deep Sparse Platform, [please reach out](mailto: [email protected]). Or simply head over to the [Neural Magic GitHub repo](https://github.com/neuralmagic) and check out our tools.
101+
We’d love to hear from users highly interested in ML performance. If you want to chat about your use cases or how others are leveraging the Deep Sparse Platform, [please reach out]([email protected]). Or simply head over to the [Neural Magic GitHub repo](https://github.com/neuralmagic) and check out our tools.
91102

92103
**Do you have available solutions or applications on the Microsoft/Azure platform?**
93104

docs/source/getstarted.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,10 @@ Neural Magic’s vision is no-hardware AI, shattering the hardware barriers that
2222

2323
To deliver on this vision, there are several components to the Deep Sparse Platform:
2424

25-
1. [Sparsify](https://docs.neuralmagic.com/sparsify): Open-source, easy-to-use interface to automatically sparsify and quantize deep learning models for CPUs & GPUs.
26-
2. [SparseML](https://docs.neuralmagic.com/sparseml): Open-source libraries and optimization algorithms for CPUs & GPUs, enabling integration with a few lines of code.
27-
3. [SparseZoo](https://docs.neuralmagic.com/sparsezoo): Open-source neural network model repository for highly sparse and sparse-quantized models with matching pruning recipes for CPUs and GPUs.
28-
4. [DeepSparse Engine](https://docs.neuralmagic.com/deepsparse): Free CPU runtime that runs sparse models at GPU speeds.
25+
1. [Sparsify](https://docs.neuralmagic.com/sparsify): Easy-to-use UI for automatically sparsifying neural networks and creating sparsification recipes for better inference performance and a smaller footprint
26+
2. [SparseML](https://docs.neuralmagic.com/sparseml): Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models
27+
3. [SparseZoo](https://docs.neuralmagic.com/sparsezoo): Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes
28+
4. [DeepSparse Engine](https://docs.neuralmagic.com/deepsparse): Neural network inference engine that delivers GPU-class performance for sparsified models on CPUs
2929

3030
Our inference engine and model optimization technologies enable companies to use ubiquitous and unconstrained CPU resources to achieve performance breakthroughs, at scale, with all the flexibility of software.
3131

@@ -62,11 +62,13 @@ Today, we offer support for convolutional neural network-based computer vision m
6262

6363
We are continuously exploring models to add to our supported [model list](https://docs.neuralmagic.com/sparsezoo/models.html) and SparseZoo including model architectures beyond computer vision. Popular NLP models such as BERT are on the Neural Magic roadmap; [subscribe for updates](http://neuralmagic.com/subscribe).
6464

65-
### PyTorch and ONNX
65+
### Notes
66+
67+
#### PyTorch and ONNX
6668

6769
Sparsify and the DeepSparse Engine inputs are standardized on the ONNX format. PyTorch has native ONNX export and requires fewer steps than other supported frameworks, such as [Keras or TensorFlow](https://docs.neuralmagic.com/sparseml/quicktour.html#exporting-to-onnx). If you have flexibility in frameworks, consider PyTorch to start.
6870

69-
### Model Considerations
71+
#### Model Considerations
7072

7173
Dynamic shape is currently not supported; be sure to use models with fixed inputs and compile the model for a particular batch size. Dynamic shape and dynamic batch sizes are on the Neural Magic roadmap; [subscribe for updates](http://neuralmagic.com/subscribe).
7274

docs/source/why-nm.jpg

151 KB
Loading

0 commit comments

Comments
 (0)