ptq_static

Step-by-Step

This example load an image classification model from ONNX Model Zoo and confirm its accuracy and speed based on ILSVR2012 validation Imagenet dataset. You need to download this dataset yourself.

Prerequisite

1. Environment

pip install onnx-neural-compressor
pip install -r requirements.txt

2. Prepare Model

python prepare_model.py --output_model='resnet50-v1-12.onnx'

3. Prepare Dataset

Download dataset ILSVR2012 validation Imagenet dataset.

Download label:

wget http://dl.caffe.berkeleyvision.org/caffe_ilsvrc12.tar.gz
tar -xvzf caffe_ilsvrc12.tar.gz val.txt

Run

1. Quantization

Quantize model with QLinearOps:

bash run_quant.sh --input_model=path/to/model \  # model path as *.onnx
                   --dataset_location=/path/to/imagenet \
                   --label_path=/path/to/val.txt \
                   --output_model=path/to/save

Quantize model with QDQ mode:

bash run_quant.sh --input_model=path/to/model \  # model path as *.onnx
                   --dataset_location=/path/to/imagenet \
                   --label_path=/path/to/val.txt \
                   --output_model=path/to/save \
                   --quant_format=QDQ

2. Benchmark

bash run_benchmark.sh --input_model=path/to/model \  # model path as *.onnx
                      --dataset_location=/path/to/imagenet \
                      --label_path=/path/to/val.txt \
                      --mode=performance # or accuracy