Skip to content

Commit 4f58547

Browse files
authored
Update README.md
1 parent f2882b6 commit 4f58547

File tree

1 file changed

+66
-20
lines changed
  • AI-and-Analytics/Getting-Started-Samples/INC-Sample-for-Tensorflow

1 file changed

+66
-20
lines changed

AI-and-Analytics/Getting-Started-Samples/INC-Sample-for-Tensorflow/README.md

Lines changed: 66 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,34 +1,35 @@
1-
# `Intel® Neural Compressor (INC) TensorFlow* Getting Started*` Sample
1+
# `Intel® Neural Compressor TensorFlow* Getting Started*` Sample
22

3-
The `Intel® Neural Compressor (INC) TensorFlow* Getting Started*` Sample demonstrates using the Intel® Neural Compressor (INC), which is part of the Intel® AI Tools with the with Intel® Optimizations for TensorFlow* to speed up inference by simplifying the process of converting the FP32 model to INT8/BF16.
3+
The `Intel® Neural Compressor TensorFlow* Getting Started*` Sample demonstrates using the Intel® Neural Compressor, which is part of the Intel® AI Tools with the with Intel® Optimizations for TensorFlow* to speed up inference by simplifying the process of converting the FP32 model to INT8/BF16.
44

5-
| Area | Description
5+
| Property | Description
66
|:--- |:---
7-
| What you will learn | How to use Intel® Neural Compressor (INC) tool to quantize the AI model based on TensorFlow* and speed up the inference on Intel® Xeon® CPUs
8-
| Time to complete | 10 minutes
97
| Category | Getting Started
8+
| What you will learn | How to use Intel® Neural Compressor tool to quantize the AI model based on TensorFlow* and speed up the inference on Intel® Xeon® CPUs
9+
| Time to complete | 10 minutes
10+
1011

1112
## Purpose
1213

13-
This sample shows the process of building a convolutional neural network (CNN) model to recognize handwritten numbers and demonstrates how to increase the inference performance by using Intel® Neural Compressor (INC). Low-precision optimizations can speed up inference. Intel® Neural Compressor (INC) simplifies the process of converting the FP32 model to INT8/BF16. At the same time, Intel® Neural Compressor (INC) tunes the quantization method to reduce the accuracy loss, which is a big blocker for low-precision inference.
14+
This sample shows the process of building a convolutional neural network (CNN) model to recognize handwritten numbers and demonstrates how to increase the inference performance by using Intel® Neural Compressor. Low-precision optimizations can speed up inference. Intel® Neural Compressor simplifies the process of converting the FP32 model to INT8/BF16. At the same time, Intel® Neural Compressor tunes the quantization method to reduce the accuracy loss, which is a big blocker for low-precision inference.
1415

1516
You can achieve higher inference performance by converting the FP32 model to INT8 or BF16 model. Additionally, Intel® Deep Learning Boost (Intel® DL Boost) in Intel® Xeon® Scalable processors and Xeon® processors provides hardware acceleration for INT8 and BF16 models.
1617

17-
You will learn how to train a CNN model with Keras and TensorFlow*, use Intel® Neural Compressor (INC) to quantize the model, and compare the performance to see the benefit of Intel® Neural Compressor (INC).
18+
You will learn how to train a CNN model with Keras and TensorFlow*, use Intel® Neural Compressor to quantize the model, and compare the performance to see the benefit of Intel® Neural Compressor.
1819

1920
## Prerequisites
2021

2122
| Optimized for | Description
2223
|:--- |:---
2324
| OS | Ubuntu* 20.04 (or newer) <br> Windows 11, 10*
2425
| Hardware | Intel® Core™ Gen10 Processor <br> Intel® Xeon® Scalable Performance processors
25-
| Software | Intel® Neural Compressor (INC), Intel Optimization for TensorFlow
26+
| Software | Intel® Neural Compressor, Intel Optimization for TensorFlow
2627

27-
### Intel® Neural Compressor (INC) and Sample Code Versions
28+
### Intel® Neural Compressor and Sample Code Versions
2829

29-
>**Note**: See the [Intel® Neural Compressor (INC)](https://github.com/intel/neural-compressor) GitHub repository for more information and recent changes.
30+
>**Note**: See the [Intel® Neural Compressor](https://github.com/intel/neural-compressor) GitHub repository for more information and recent changes.
3031
31-
This sample is updated regularly to match the Intel® Neural Compressor (INC) version in the latest Intel® AI Tools release. If you want to get the sample code for an earlier toolkit release, check out the corresponding git tag.
32+
This sample is updated regularly to match the Intel® Neural Compressor version in the latest Intel® AI Tools release. If you want to get the sample code for an earlier toolkit release, check out the corresponding git tag.
3233

3334
1. List the available git tags.
3435
```
@@ -63,14 +64,14 @@ You will need to download and install the following toolkits, tools, and compone
6364
The sample demonstrates how to:
6465

6566
- Use Keras from TensorFlow* to build and train a CNN model.
66-
- Define a function and class for Intel® Neural Compressor (INC) to
67+
- Define a function and class for Intel® Neural Compressor to
6768
quantize the CNN model.
68-
- The Intel® Neural Compressor (INC) can run on any Intel® CPU to quantize the AI model.
69+
- The Intel® Neural Compressor can run on any Intel® CPU to quantize the AI model.
6970
- The quantized AI model has better inference performance than the FP32 model on Intel CPUs.
7071
- Specifically, the latest Intel® Xeon® Scalable processors and Xeon® processors provide hardware acceleration for such tasks.
7172
- Test the performance of the FP32 model and INT8 (quantization) model.
7273

73-
## Prepare the Environment
74+
## Environment Setup
7475
If you have already set up the PIP or Conda environment and installed AI Tools go directly to Run the Notebook.
7576

7677
### On Linux* (Only applicable to AI Tools Offline Installer)
@@ -79,9 +80,13 @@ If you have already set up the PIP or Conda environment and installed AI Tools g
7980

8081
When working with the command-line interface (CLI), you should configure the oneAPI toolkits using environment variables. Set up your CLI environment by sourcing the `setvars` script every time you open a new terminal window. This practice ensures that your compiler, libraries, and tools are ready for development.
8182

82-
#### Activate Conda
83+
#### Setup Conda Environment
8384

84-
You can list the available conda environments using a command similar to the following
85+
You can list the available conda environments using a command similar to the following.
86+
87+
##### Option 1: Clone Conda Environment from AI Toolkit Conda Environment
88+
89+
Please confirm to install Intel AI Toolkit!
8590

8691
```
8792
conda info -e
@@ -115,11 +120,27 @@ tensorflow-2.3.0 /opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.
115120
```
116121
source activate usr_tensorflow
117122
```
118-
2. Install Intel® Neural Compressor (INC) from the local channel.
123+
2. Install Intel® Neural Compressor from the local channel.
119124
```
120125
conda install -c ${ONEAPI_ROOT}/conda_channel neural-compressor -y --offline
121126
```
122127
128+
##### Option 2: Create Conda Environment
129+
130+
Configure Conda for **user_tensorflow** by entering commands similar to the following:
131+
```
132+
conda deactivate
133+
conda env remove -n user_tensorflow
134+
conda create -n user_tensorflow python=3.9 -y
135+
conda activate user_tensorflow
136+
conda install -n user_tensorflow pycocotools -c esri -y
137+
conda install -n user_tensorflow neural-compressor tensorflow -c conda-forge -c intel -y
138+
conda install -n user_tensorflow jupyter runipy notebook -y
139+
conda install -c anaconda ipykernel
140+
python -m ipykernel install --user --nam=user_tensorflow
141+
```
142+
143+
123144
#### Configure Jupyter Notebook
124145
125146
1. Create a new kernel for the Jupyter notebook based on your activated conda environment.
@@ -133,7 +154,7 @@ tensorflow-2.3.0 /opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.
133154
134155
#### Configure Conda
135156
136-
1. Configure Conda for **user_tensorflow** by entering commands similar to the following:
157+
Configure Conda for **user_tensorflow** by entering commands similar to the following:
137158
```
138159
conda deactivate
139160
conda env remove -n user_tensorflow
@@ -142,9 +163,11 @@ tensorflow-2.3.0 /opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.
142163
conda install -n user_tensorflow pycocotools -c esri -y
143164
conda install -n user_tensorflow neural-compressor tensorflow -c conda-forge -c intel -y
144165
conda install -n user_tensorflow jupyter runipy notebook -y
166+
conda install -c anaconda ipykernel
167+
python -m ipykernel install --user --nam=user_tensorflow
145168
```
146169
147-
## Run the `Intel® Neural Compressor (INC) TensorFlow* Getting Started*` Sample
170+
## Run the `Intel® Neural Compressor TensorFlow* Getting Started*` Sample
148171
149172
> **Note**: If you have not already done so, set up your CLI
150173
> environment by sourcing the `setvars` script in the root of your oneAPI installation.
@@ -160,7 +183,7 @@ tensorflow-2.3.0 /opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.
160183
>
161184
> For more information on configuring environment variables, see *[Use the setvars Script with Linux* or macOS*](https://www.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/oneapi-development-environment-setup/use-the-setvars-script-with-linux-or-macos.html)* or *[Use the setvars Script with Windows*](https://www.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/oneapi-development-environment-setup/use-the-setvars-script-with-windows.html)*.
162185
163-
### Steps for Intel AI Tools Offline Installer
186+
### Active Conda Environment
164187
165188
1. Ensure you activate the conda environment.
166189
```
@@ -212,11 +235,34 @@ tensorflow-2.3.0 /opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.
212235
4. Change the kernel to **user_tensorflow**.
213236
5. Run every cell in the Notebook in sequence.
214237
238+
## Example Output
239+
240+
You should see log print and images showing the performance comparison with absolute and relative data and analysis between FP32 and INT8.
241+
242+
Following is an example. Your data should be different with them.
243+
244+
```
245+
#absolute data
246+
throughputs_times [1, 2.51508607887295]
247+
latencys_times [1, 0.38379207710795576]
248+
accuracys_times [0, -0.009999999999990905]
249+
250+
#relative data
251+
throughputs_times [1, 2.51508607887295]
252+
latencys_times [1, 0.38379207710795576]
253+
accuracys_times [0, -0.009999999999990905]
254+
```
255+
256+
![Absolute Performance](img/inc_ab_perf_data.png)
257+
![Relative Performance](img/inc_re_perf_data.png)
215258
216259
#### Troubleshooting
217260
218261
If you receive an error message, troubleshoot the problem using the **Diagnostics Utility for Intel® oneAPI Toolkits**. The diagnostic utility provides configuration and system checks to help find missing dependencies, permissions errors, and other issues. See the [Diagnostics Utility for Intel® oneAPI Toolkits User Guide](https://www.intel.com/content/www/us/en/develop/documentation/diagnostic-utility-user-guide/top.html) for more information on using the utility.
219262
263+
## Related Samples
264+
265+
[Pytorch `Getting Started with Intel® Neural Compressor for Quantization` Sample](../INC-Quantization-Sample-for-PyTorch)
220266
221267
## License
222268

0 commit comments

Comments
 (0)