You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+21-5Lines changed: 21 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@
10
10
11
11
## Introduction
12
12
13
-
FMS Model Optimizer is a framework for developing reduced precision neural network models. Quantization techniques, such as [quantization-aware-training (QAT)](https://arxiv.org/abs/2407.11062), [post-training quantization (PTQ)](https://arxiv.org/abs/2102.05426), and several other optimization techniques on popular deep learning workloads are supported.
13
+
FMS Model Optimizer is a framework for developing reduced precision neural network models. [Quantization](https://www.ibm.com/think/topics/quantization) techniques, such as [quantization-aware-training (QAT)](https://arxiv.org/abs/2407.11062), [post-training quantization (PTQ)](https://arxiv.org/abs/2102.05426), and several other optimization techniques on popular deep learning workloads are supported.
14
14
15
15
## Highlights
16
16
@@ -37,10 +37,11 @@ FMS Model Optimizer is a framework for developing reduced precision neural netwo
37
37
38
38
1.**🐧 Linux system with Nvidia GPU (V100/A100/H100)**
39
39
2. Python 3.9 to Python 3.11
40
+
40
41
📋 Python 3.12 is currently not supported due to PyTorch Dynamo constraint
41
42
3. CUDA >=12
42
43
43
-
*Optional packages based on optimization functionalities required:*
44
+
*Optional packages based on optimization functionality required:*
44
45
45
46
-**GPTQ** is a popular compression method for LLMs:
46
47
-[auto_gptq](https://pypi.org/project/auto-gptq/) or build from [source](https://github.com/AutoGPTQ/AutoGPTQ)
@@ -50,9 +51,10 @@ FMS Model Optimizer is a framework for developing reduced precision neural netwo
50
51
- Clone the [CUTLASS](https://github.com/NVIDIA/cutlass) repository
51
52
-`PyTorch 2.3.1` (as newer version will cause issue for the custom CUDA kernel used in these examples)
52
53
-**FP8** is a reduced precision format like **INT8**:
- To enable compute graph plotting function (mostly for troubleshooting purpose):
57
+
-[matplotlib](https://matplotlib.org/)
56
58
-[graphviz](https://graphviz.org/)
57
59
-[pygraphviz](https://pygraphviz.github.io/)
58
60
@@ -61,7 +63,7 @@ FMS Model Optimizer is a framework for developing reduced precision neural netwo
61
63
62
64
### Installation
63
65
64
-
We recommend using a Python virtual environment with Python 3.10+. Here is how to setup a virtual environment using [Python venv](https://docs.python.org/3/library/venv.html):
66
+
We recommend using a Python virtual environment with Python 3.9+. Here is how to setup a virtual environment using [Python venv](https://docs.python.org/3/library/venv.html):
> If you use [pyenv](https://github.com/pyenv/pyenv), [Conda Miniforge](https://github.com/conda-forge/miniforge) or other such tools for Python version management, create the virtual environment with that tool instead of venv. Otherwise, you may have issues with installed packages not being found as they are linked to your Python version management tool and not `venv`.
73
75
74
-
To install `fms_mo` package from source:
76
+
There are 2 ways to install the FMS Model Optimizer as follows:
77
+
78
+
#### From Release
79
+
80
+
To install from release ([PyPi package](https://pypi.org/project/fms-model-optimizer/)):
0 commit comments