You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch/README.MD
+40-84Lines changed: 40 additions & 84 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,7 @@
2
2
3
3
The sample is a getting started tutorial for the Intel® Neural Compressor (INC), and demonstrates how to perform INT8 quantization on a Hugging Face BERT model. This sample shows how to achieve performance boosts using Intel hardware.
4
4
5
-
| Property | Description
5
+
| Area | Description
6
6
|:--- |:---
7
7
| What you will learn | How to quantize a BERT model using Intel® Neural Compressor
8
8
| Time to complete | 20 minutes
@@ -48,41 +48,14 @@ You will need to download and install the following toolkits, tools, and compone
48
48
Required AI Tools: **Intel® Neural Compressor, Intel® Extension of PyTorch***.
49
49
<br>If you have not already, select and install these Tools via via [AI Tools Selector](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-tools-selector.html). AI and Analytics samples are validated on AI Tools Offline Installer. It is recommended to select Offline Installer option in AI Tools Selector.
50
50
51
-
>**Note**: If Docker option is chosen in AI Tools Selector, refer to [Working with Preset Containers](https://github.com/intel/ai-containers/tree/main/preset) to learn how to run the docker and samples.
52
-
53
-
**2. (Offline Installer) Activate the AI Tools bundle base environment**
54
-
55
-
If the default path is used during the installation of AI Tools:
cd oneAPI-samples/AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch
53
+
pip install -r requirements.txt
74
54
```
55
+
**Install Jupyter Notebook** by running `pip install notebook`. Alternatively, see [Installing Jupyter](https://jupyter.org/install) for detailed installation instructions.
75
56
76
-
**5. Install dependencies**
57
+
## Run the `Getting Started with Intel® Neural Compressor for Quantization` Sample
77
58
78
-
>**Note**: Before running the following commands, make sure your Conda/Python environment with AI Tools installed is activated
79
-
```
80
-
pip install -r requirements.txt
81
-
pip install notebook
82
-
```
83
-
For Jupyter Notebook, refer to [Installing Jupyter](https://jupyter.org/install) for detailed installation instructions.
84
-
85
-
## Run the Sample
86
59
>**Note**: Before running the sample, make sure [Environment Setup](https://github.com/oneapi-src/oneAPI-samples/tree/master/AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch#environment-setup) is completed.
87
60
88
61
Go to the section which corresponds to the installation method chosen in [AI Tools Selector](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-tools-selector.html) to see relevant instructions:
@@ -91,91 +64,74 @@ Go to the section which corresponds to the installation method chosen in [AI Too
91
64
*[Docker](#docker)
92
65
93
66
### AI Tools Offline Installer (Validated)
94
-
95
-
**1. Register Conda kernel to Jupyter Notebook kernel**
96
-
97
-
If the default path is used during the installation of AI Tools:
67
+
1. If you have not already done so, activate the AI Tools bundle base environment.
68
+
If you used the default location to install AI Tools, open a terminal and type the following
cd oneAPI-samples/AI-and-Analytics/Getting-Started-Samples
133
105
```
134
-
**2. Launch Jupyter Notebook**
135
-
106
+
2. Launch Jupyter Notebook:
107
+
> **Note**: You might need to register Conda kernel to Jupyter Notebook kernel,
108
+
feel free to check [the instruction](https://github.com/IntelAI/models/tree/master/docs/notebooks/perf_analysis#option-1-conda-environment-creation)
136
109
```
137
110
jupyter notebook --ip=0.0.0.0
138
111
```
139
-
**3. Follow the instructions to open the URL with the token in your browser**
140
-
141
-
**4. Select the Notebook**
142
-
112
+
4. Follow the instructions to open the URL with the token in your browser.
113
+
5. Select the Notebook:
143
114
```
144
115
quantize_with_inc.ipynb
145
116
```
146
-
**5. Change the kernel to `<your-env-name>`**
147
-
148
-
**6. Run every cell in the Notebook in sequence**
117
+
6. Run every cell in the Notebook in sequence.
149
118
150
119
### Docker
151
120
AI Tools Docker images already have Get Started samples pre-installed. Refer to [Working with Preset Containers](https://github.com/intel/ai-containers/tree/main/preset) to learn how to run the docker and samples.
152
-
121
+
153
122
## Example Output
154
123
You should see an image showing the performance comparison and analysis between FP32 and INT8.
155
124
>**Note**: The image shown below is an example of a general performance comparison for inference speedup obtained by quantization. (Your results might be different.)
156
125
157
126

158
-
159
127
## Related Samples
160
-
161
128
*[Fine-tuning Text Classification Model with Intel® Neural Compressor (INC)](https://github.com/oneapi-src/oneAPI-samples/tree/master/AI-and-Analytics/Features-and-Functionality/INC_QuantizationAwareTraining_TextClassification)
162
129
*[Optimize PyTorch* Models using Intel® Extension for PyTorch* (IPEX)](https://github.com/oneapi-src/oneAPI-samples/tree/master/AI-and-Analytics/Features-and-Functionality/INC_QuantizationAwareTraining_TextClassification)
163
-
164
130
## License
165
131
166
132
Code samples are licensed under the MIT license. See
Third party program Licenses can be found here: [third-party-programs.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/third-party-programs.txt).
172
136
173
137
*Other names and brands may be claimed as the property of others. [Trademarks](https://www.intel.com/content/www/us/en/legal/trademarks.html)
0 commit comments