You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch/README.MD
+87-43Lines changed: 87 additions & 43 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,7 @@
2
2
3
3
The sample is a getting started tutorial for the Intel® Neural Compressor (INC), and demonstrates how to perform INT8 quantization on a Hugging Face BERT model. This sample shows how to achieve performance boosts using Intel hardware.
4
4
5
-
| Area | Description
5
+
| Property | Description
6
6
|:--- |:---
7
7
| What you will learn | How to quantize a BERT model using Intel® Neural Compressor
8
8
| Time to complete | 20 minutes
@@ -39,7 +39,7 @@ The sample contains one Jupyter Notebook and one Python script. It can be run us
39
39
|:--- |:---
40
40
|`dataset.py` | The script provides a PyTorch* Dataset class that tokenizes text data
41
41
42
-
### Setup your environment for the offline installer
42
+
### Environment Setup
43
43
44
44
You will need to download and install the following toolkits, tools, and components to use the sample.
45
45
@@ -48,14 +48,41 @@ You will need to download and install the following toolkits, tools, and compone
48
48
Required AI Tools: **Intel® Neural Compressor, Intel® Extension of PyTorch***.
49
49
<br>If you have not already, select and install these Tools via via [AI Tools Selector](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-tools-selector.html). AI and Analytics samples are validated on AI Tools Offline Installer. It is recommended to select Offline Installer option in AI Tools Selector.
50
50
51
-
**2. Install dependencies**
51
+
>**Note**: If Docker option is chosen in AI Tools Selector, refer to [Working with Preset Containers](https://github.com/intel/ai-containers/tree/main/preset) to learn how to run the docker and samples.
52
+
53
+
**2. (Offline Installer) Activate the AI Tools bundle base environment**
54
+
55
+
If the default path is used during the installation of AI Tools:
cd oneAPI-samples/AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch
54
74
```
55
-
**Install Jupyter Notebook** by running `pip install notebook`. Alternatively, see [Installing Jupyter](https://jupyter.org/install) for detailed installation instructions.
56
75
57
-
## Run the `Getting Started with Intel® Neural Compressor for Quantization` Sample
76
+
**5. Install dependencies**
58
77
78
+
>**Note**: Before running the following commands, make sure your Conda/Python environment with AI Tools installed is activated
79
+
```
80
+
pip install -r requirements.txt
81
+
pip install notebook
82
+
```
83
+
For Jupyter Notebook, refer to [Installing Jupyter](https://jupyter.org/install) for detailed installation instructions.
84
+
85
+
## Run the Sample
59
86
>**Note**: Before running the sample, make sure [Environment Setup](https://github.com/oneapi-src/oneAPI-samples/tree/master/AI-and-Analytics/Getting-Started-Samples/INC-Quantization-Sample-for-PyTorch#environment-setup) is completed.
60
87
61
88
Go to the section which corresponds to the installation method chosen in [AI Tools Selector](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-tools-selector.html) to see relevant instructions:
@@ -64,74 +91,91 @@ Go to the section which corresponds to the installation method chosen in [AI Too
64
91
*[Docker](#docker)
65
92
66
93
### AI Tools Offline Installer (Validated)
67
-
1. If you have not already done so, activate the AI Tools bundle base environment.
68
-
If you used the default location to install AI Tools, open a terminal and type the following
4. Follow the instructions to open the URL with the token in your browser.
113
-
5. Select the Notebook:
139
+
**3. Follow the instructions to open the URL with the token in your browser**
140
+
141
+
**4. Select the Notebook**
142
+
114
143
```
115
-
optimize_pytorch_models_with_ipex.ipynb
144
+
quantize_with_inc.ipynb
116
145
```
117
-
6. Run every cell in the Notebook in sequence.
146
+
**5. Change the kernel to `<your-env-name>`**
147
+
148
+
**6. Run every cell in the Notebook in sequence**
118
149
119
150
### Docker
120
151
AI Tools Docker images already have Get Started samples pre-installed. Refer to [Working with Preset Containers](https://github.com/intel/ai-containers/tree/main/preset) to learn how to run the docker and samples.
121
-
152
+
122
153
## Example Output
123
154
You should see an image showing the performance comparison and analysis between FP32 and INT8.
124
155
>**Note**: The image shown below is an example of a general performance comparison for inference speedup obtained by quantization. (Your results might be different.)
125
156
126
157

158
+
127
159
## Related Samples
160
+
128
161
*[Fine-tuning Text Classification Model with Intel® Neural Compressor (INC)](https://github.com/oneapi-src/oneAPI-samples/tree/master/AI-and-Analytics/Features-and-Functionality/INC_QuantizationAwareTraining_TextClassification)
129
162
*[Optimize PyTorch* Models using Intel® Extension for PyTorch* (IPEX)](https://github.com/oneapi-src/oneAPI-samples/tree/master/AI-and-Analytics/Features-and-Functionality/INC_QuantizationAwareTraining_TextClassification)
163
+
130
164
## License
131
165
132
166
Code samples are licensed under the MIT license. See
133
-
[License.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/License.txt) for details.
Third party program Licenses can be found here: [third-party-programs.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/third-party-programs.txt).
0 commit comments