Skip to content

Commit a952dac

Browse files
author
Roman Donchenko
committed
Document the model quantizer
1 parent 294e033 commit a952dac

File tree

1 file changed

+98
-1
lines changed

1 file changed

+98
-1
lines changed

tools/downloader/README.md

Lines changed: 98 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@ based on configuration files in the models' directories.
1111
* `converter.py` (model converter) converts the models that are not in the
1212
Inference Engine IR format into that format using Model Optimizer.
1313

14+
* `quantizer.py` (model quantizer) quantizes full-precision models in the IR
15+
format into low-precision versions using Post-Training Optimization Toolkit.
16+
1417
* `info_dumper.py` (model information dumper) prints information about the models
1518
in a stable machine-readable format.
1619

@@ -296,6 +299,100 @@ To do this, use the `--dry_run` option:
296299
See the "Shared options" section for information on other options accepted by
297300
the script.
298301

302+
Model quantizer usage
303+
---------------------
304+
305+
Before you run the model quantizer, you must prepare a directory with
306+
the datasets required for the quantization process. This directory will be
307+
referred to as `<DATASET_DIR>` below. See the "Dataset directory layout"
308+
section for information on the expected contents of that directory.
309+
310+
The basic usage is to run the script like this:
311+
312+
```sh
313+
./quantizer.py --all --dataset_dir <DATASET_DIR>
314+
```
315+
316+
This will quantize all models for which quantization is supported. Other models
317+
are ignored.
318+
319+
The `--all` option can be replaced with other filter options to quantize only
320+
a subset of models. See the "Shared options" section.
321+
322+
The current directory must be the root of a tree of model files create by the model
323+
converter. To specify a different model tree path, use the `--model_dir` option:
324+
325+
```sh
326+
./quantizer.py --all --dataset_dir <DATASET_DIR> --model_dir my/model/directory
327+
```
328+
329+
By default, the quantized models are placed into the same model tree. To place them
330+
into a different directory tree, use the `-o`/`--output_dir` option:
331+
332+
```sh
333+
./quantizer.py --all --dataset_dir <DATASET_DIR> --output_dir my/output/directory
334+
```
335+
336+
By default, the script will produce models in every precision that is supported
337+
as a quantization output. To only produce models in a specific precision, use
338+
the `--precisions` option:
339+
340+
```sh
341+
./quantizer.py --all --dataset_dir <DATASET_DIR> --precisions=FP16-INT8
342+
```
343+
344+
The script will attempt to locate Post-Training Optimization Toolkit using
345+
the environment variables set by the OpenVINO&trade; toolkit's `setupvars.sh`/`setupvars.bat`
346+
script. You can override this heuristic with the `--pot` option:
347+
348+
```sh
349+
./quantizer.py --all --dataset_dir <DATASET_DIR> --pot my/openvino/path/post_training_optimization_toolkit/main.py
350+
```
351+
352+
By default, the script will run Post-Training Optimization Toolkit using the same
353+
Python executable that was used to run the script itself. To use a different
354+
Python executable, use the `-p`/`--python` option:
355+
356+
```sh
357+
./quantizer.py --all --dataset_dir <DATASET_DIR> --python my/python
358+
```
359+
360+
It's possible to specify a target device for Post-Training Optimization Toolkit
361+
to optimize for, by using the `--target_device` option:
362+
363+
```sh
364+
./quantizer.py --all --dataset_dir <DATASET_DIR> --target_device VPU
365+
```
366+
367+
The supported values are those accepted by the "target_device" option in
368+
Post-Training Optimization Toolkit's config files. If this option is unspecified,
369+
Post-Training Optimization Toolkit's default is used.
370+
371+
The script can print the quantization commands without actually running them.
372+
To do this, use the `--dry_run` option:
373+
374+
```sh
375+
./quantizer.py --all --dataset_dir <DATASET_DIR> --dry_run
376+
```
377+
378+
With this option specified, the configuration file for Post-Training Optimization
379+
Toolkit will still be created, so that you can inspect it.
380+
381+
See the "Shared options" section for information on other options accepted by
382+
the script.
383+
384+
### Dataset directory layout
385+
386+
Currently, all models for which quantization is supported require the
387+
[ILSVRC 2012](http://image-net.org/challenges/LSVRC/2012/index) validation
388+
dataset. This means that `<DATASET_DIR>` must contain the following entries:
389+
390+
* A subdirectory named `ILSVRC2012_img_val` containing the ILSVRC 2012
391+
validation images. To obtain these images, follow the
392+
[instructions at the ILSVRC 2012 website](http://image-net.org/challenges/LSVRC/2012/signup).
393+
394+
* `val.txt` from <http://dl.caffe.berkeleyvision.org/caffe_ilsvrc12.tar.gz>.
395+
299396
Model information dumper usage
300397
------------------------------
301398

@@ -364,7 +461,7 @@ describing a single model. Each such object has the following keys:
364461
Shared options
365462
--------------
366463

367-
The are certain options that both tools accept.
464+
The are certain options that all tools accept.
368465

369466
`-h`/`--help` can be used to print a help message:
370467

0 commit comments

Comments
 (0)