You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jun 3, 2025. It is now read-only.
@@ -44,25 +44,37 @@ limitations under the License.
44
44
45
45
## Overview
46
46
47
-
SparseML is a toolkit that includes APIs, CLIs, scripts and libraries that apply state-of-the-art optimization algorithms such as [pruning](https://neuralmagic.com/blog/pruning-overview/) and [quantization](https://arxiv.org/abs/1609.07061) to any neural network. General, recipe-driven approaches built around these optimizations enable the simplification of creating faster and smaller models for the ML performance community at large.
47
+
SparseML is a toolkit that includes APIs, CLIs, scripts and libraries that apply state-of-the-art sparsification algorithms such as pruning and quantization to any neural network.
48
+
General, recipe-driven approaches built around these algorithms enable the simplification of creating faster and smaller models for the ML performance community at large.
48
49
49
-
SparseML is integrated for easy model optimizations within the [PyTorch](https://pytorch.org/),
50
-
[Keras](https://keras.io/), and [TensorFlow V1](http://tensorflow.org/) ecosystems currently.
50
+
This repository contains integrations within the [PyTorch](https://pytorch.org/), [Keras](https://keras.io/), and [TensorFlow V1](http://tensorflow.org/) ecosystems, allowing for seamless model sparsification.
51
51
52
-
### Related Products
52
+
##Sparsification
53
53
54
-
-[DeepSparse](https://github.com/neuralmagic/deepsparse): CPU inference engine that delivers unprecedented performance for sparse models
55
-
-[SparseZoo](https://github.com/neuralmagic/sparsezoo): Neural network model repository for highly sparse models and optimization recipes
56
-
-[Sparsify](https://github.com/neuralmagic/sparsify): Easy-to-use autoML interface to optimize deep neural networks for better inference performance and a smaller footprint
54
+
Sparsification is the process of taking a trained deep learning model and removing redundant information from the overprecise and over-parameterized network resulting in a faster and smaller model.
55
+
Techniques for sparsification are all encompassing including everything from inducing sparsity using [pruning](https://neuralmagic.com/blog/pruning-overview/) and [quantization](https://arxiv.org/abs/1609.07061) to enabling naturally occurring sparsity using [activation sparsity](http://proceedings.mlr.press/v119/kurtz20a.html) or [winograd/FFT](https://arxiv.org/abs/1509.09308).
56
+
When implemented correctly, these techniques result in significantly more performant and smaller models with limited to no effect on the baseline metrics.
57
+
For example, pruning plus quantization can give over [7x improvements in performance](https://neuralmagic.com/blog/benchmark-resnet50-with-deepsparse) while recovering to nearly the same baseline accuracy.
58
+
59
+
The Deep Sparse product suite builds on top of sparsification enabling you to easily apply the techniques to your datasets and models using recipe-driven approaches.
60
+
Recipes encode the directions for how to sparsify a model into a simple, easily editable format.
61
+
- Download a sparsification recipe and sparsified model from the [SparseZoo](https://github.com/neuralmagic/sparsezoo).
62
+
- Alternatively, create a recipe for your model using [Sparsify](https://github.com/neuralmagic/sparsify).
63
+
- Apply your recipe with only a few lines of code using [SparseML](https://github.com/neuralmagic/sparseml).
64
+
- Finally, for GPU-level performance on CPUs, deploy your sparse-quantized model with the [DeepSparse Engine](https://github.com/neuralmagic/deepsparse).
To enable flexibility, ease of use, and repeatability, optimizing a model is generally done using a recipe file.
61
-
The files encode the instructions needed for modifying the model and/or training process as a list of modifiers.
73
+
To enable flexibility, ease of use, and repeatability, sparsifying a model is generally done using a recipe.
74
+
The recipes encode the instructions needed for modifying the model and/or training process as a list of modifiers.
62
75
Example modifiers can be anything from setting the learning rate for the optimizer to gradual magnitude pruning.
63
76
The files are written in [YAML](https://yaml.org/) and stored in YAML or [markdown](https://www.markdownguide.org/) files using [YAML front matter](https://assemble.io/docs/YAML-front-matter.html).
64
-
The rest of the SparseML system is coded to parse the recipe files into a native format for the desired framework
65
-
and apply the modifications to the model and training pipeline.
77
+
The rest of the SparseML system is coded to parse the recipes into a native format for the desired framework and apply the modifications to the model and training pipeline.
66
78
67
79
A sample recipe for pruning a model generally looks like the following:
More information on the available recipes, formats, and arguments can be found [here](https://github.com/neuralmagic/sparseml/blob/main/docs/optimization-recipes.md). Additionally, all code implementations of the modifiers under the `optim` packages for the frameworks are documented with example YAML formats.
106
+
More information on the available recipes, formats, and arguments can be found [here](https://github.com/neuralmagic/sparseml/blob/main/docs/source/recipes.md). Additionally, all code implementations of the modifiers under the `optim` packages for the frameworks are documented with example YAML formats.
95
107
96
108
Pre-configured recipes and the resulting models can be explored and downloaded from the [SparseZoo](https://github.com/neuralmagic/sparsezoo). Also, [Sparsify](https://github.com/neuralmagic/sparsify) enables autoML style creation of optimization recipes for use with SparseML.
97
109
98
110
For a more in-depth read, check out [SparseML documentation](https://docs.neuralmagic.com/sparseml/).
99
111
100
-
### PyTorch Optimization
112
+
### PyTorch Sparsification
101
113
102
-
The PyTorch optimization libraries are located under the `sparseml.pytorch.optim` package.
103
-
Inside are APIs designed to make model optimization as easy as possible by integrating seamlessly into PyTorch training pipelines.
114
+
The PyTorch sparsification libraries are located under the `sparseml.pytorch.optim` package.
115
+
Inside are APIs designed to make model sparsification as easy as possible by integrating seamlessly into PyTorch training pipelines.
104
116
105
-
The integration is done using the `ScheduledOptimizer` class. It is intended to wrap your current optimizer and its step function. The step function then calls into the `ScheduledModifierManager` class which can be created from a recipe file. With this setup, the training process can then be modified as desired to optimize the model.
117
+
The integration is done using the `ScheduledOptimizer` class.
118
+
It is intended to wrap your current optimizer and its step function.
119
+
The step function then calls into the `ScheduledModifierManager` class which can be created from a recipe file.
120
+
With this setup, the training process can then be modified as desired to sparsify the model.
106
121
107
122
To enable all of this, the integration code you'll need to write is only a handful of lines:
The TensorFlow optimization libraries for TensorFlow version 1.X are located under the `sparseml.tensorflow_v1.optim` package. Inside are APIs designed to make model optimization as easy as possible by integrating seamlessly into TensorFlow V1 training pipelines.
175
+
The TensorFlow sparsification libraries for TensorFlow version 1.X are located under the `sparseml.tensorflow_v1.optim` package.
176
+
Inside are APIs designed to make model sparsification as easy as possible by integrating seamlessly into TensorFlow V1 training pipelines.
161
177
162
178
The integration is done using the `ScheduledModifierManager` class which can be created from a recipe file.
163
-
This class handles modifying the TensorFlow graph for the desired optimizations.
164
-
With this setup, the training process can then be modified as desired to optimize the model.
179
+
This class handles modifying the TensorFlow graph for the desired algorithms.
180
+
With this setup, the training process can then be modified as desired to sparsify the model.
Session-based pipelines need a little bit more as compared to estimator-based pipelines; however,
186
202
it is still designed to require only a few lines of code for integration.
187
203
After graph creation, the manager's `create_ops` method must be called.
188
-
This will modify the graph as needed for the optimizations and return modifying ops and extras.
204
+
This will modify the graph as needed for the algorithms and return modifying ops and extras.
189
205
After creating the session and training normally, call into `session.run` with the modifying ops after each step.
190
206
Modifying extras contain objects such as tensorboard summaries of the modifiers to be used if desired.
191
207
Finally, once completed, `complete_graph` must be called to remove the modifying ops for saving and export.
@@ -289,7 +305,7 @@ Install with pip using:
289
305
pip install sparseml
290
306
```
291
307
292
-
Then if you would like to explore any of the [scripts](https://github.com/neuralmagic/sparseml/blob/main/scripts/), [notebooks](https://github.com/neuralmagic/sparseml/blob/main/notebooks/), or [examples](https://github.com/neuralmagic/sparseml/blob/main/examples/)
308
+
Then if you would like to explore any of the [scripts](https://github.com/neuralmagic/sparseml/blob/main/scripts/), [notebooks](https://github.com/neuralmagic/sparseml/blob/main/notebooks/), or [integrations](https://github.com/neuralmagic/sparseml/blob/main/integrations/)
293
309
clone the repository and install any additional dependencies as required.
294
310
295
311
#### Supported Framework Versions
@@ -343,7 +359,7 @@ Note, TensorFlow V1 is no longer being built for newer operating systems such as
343
359
344
360
## Contributing
345
361
346
-
We appreciate contributions to the code, examples, and documentation as well as bug reports and feature requests! [Learn how here](https://github.com/neuralmagic/sparseml/blob/main/CONTRIBUTING.md).
362
+
We appreciate contributions to the code, examples, integrations, and documentation as well as bug reports and feature requests! [Learn how here](https://github.com/neuralmagic/sparseml/blob/main/CONTRIBUTING.md).
0 commit comments