@@ -11,6 +11,9 @@ based on configuration files in the models' directories.
11
11
* ` converter.py ` (model converter) converts the models that are not in the
12
12
Inference Engine IR format into that format using Model Optimizer.
13
13
14
+ * ` quantizer.py ` (model quantizer) quantizes full-precision models in the IR
15
+ format into low-precision versions using Post-Training Optimization Toolkit.
16
+
14
17
* ` info_dumper.py ` (model information dumper) prints information about the models
15
18
in a stable machine-readable format.
16
19
@@ -296,6 +299,100 @@ To do this, use the `--dry_run` option:
296
299
See the "Shared options" section for information on other options accepted by
297
300
the script.
298
301
302
+ Model quantizer usage
303
+ ---------------------
304
+
305
+ Before you run the model quantizer, you must prepare a directory with
306
+ the datasets required for the quantization process. This directory will be
307
+ referred to as ` <DATASET_DIR> ` below. See the "Dataset directory layout"
308
+ section for information on the expected contents of that directory.
309
+
310
+ The basic usage is to run the script like this:
311
+
312
+ ``` sh
313
+ ./quantizer.py --all --dataset_dir < DATASET_DIR>
314
+ ```
315
+
316
+ This will quantize all models for which quantization is supported. Other models
317
+ are ignored.
318
+
319
+ The ` --all ` option can be replaced with other filter options to quantize only
320
+ a subset of models. See the "Shared options" section.
321
+
322
+ The current directory must be the root of a tree of model files create by the model
323
+ converter. To specify a different model tree path, use the ` --model_dir ` option:
324
+
325
+ ``` sh
326
+ ./quantizer.py --all --dataset_dir < DATASET_DIR> --model_dir my/model/directory
327
+ ```
328
+
329
+ By default, the quantized models are placed into the same model tree. To place them
330
+ into a different directory tree, use the ` -o ` /` --output_dir ` option:
331
+
332
+ ``` sh
333
+ ./quantizer.py --all --dataset_dir < DATASET_DIR> --output_dir my/output/directory
334
+ ```
335
+
336
+ By default, the script will produce models in every precision that is supported
337
+ as a quantization output. To only produce models in a specific precision, use
338
+ the ` --precisions ` option:
339
+
340
+ ``` sh
341
+ ./quantizer.py --all --dataset_dir < DATASET_DIR> --precisions=FP16-INT8
342
+ ```
343
+
344
+ The script will attempt to locate Post-Training Optimization Toolkit using
345
+ the environment variables set by the OpenVINO&trade ; toolkit's ` setupvars.sh ` /` setupvars.bat `
346
+ script. You can override this heuristic with the ` --pot ` option:
347
+
348
+ ``` sh
349
+ ./quantizer.py --all --dataset_dir < DATASET_DIR> --pot my/openvino/path/post_training_optimization_toolkit/main.py
350
+ ```
351
+
352
+ By default, the script will run Post-Training Optimization Toolkit using the same
353
+ Python executable that was used to run the script itself. To use a different
354
+ Python executable, use the ` -p ` /` --python ` option:
355
+
356
+ ``` sh
357
+ ./quantizer.py --all --dataset_dir < DATASET_DIR> --python my/python
358
+ ```
359
+
360
+ It's possible to specify a target device for Post-Training Optimization Toolkit
361
+ to optimize for, by using the ` --target_device ` option:
362
+
363
+ ``` sh
364
+ ./quantizer.py --all --dataset_dir < DATASET_DIR> --target_device VPU
365
+ ```
366
+
367
+ The supported values are those accepted by the "target_device" option in
368
+ Post-Training Optimization Toolkit's config files. If this option is unspecified,
369
+ Post-Training Optimization Toolkit's default is used.
370
+
371
+ The script can print the quantization commands without actually running them.
372
+ To do this, use the ` --dry_run ` option:
373
+
374
+ ``` sh
375
+ ./quantizer.py --all --dataset_dir < DATASET_DIR> --dry_run
376
+ ```
377
+
378
+ With this option specified, the configuration file for Post-Training Optimization
379
+ Toolkit will still be created, so that you can inspect it.
380
+
381
+ See the "Shared options" section for information on other options accepted by
382
+ the script.
383
+
384
+ ### Dataset directory layout
385
+
386
+ Currently, all models for which quantization is supported require the
387
+ [ ILSVRC 2012] ( http://image-net.org/challenges/LSVRC/2012/index ) validation
388
+ dataset. This means that ` <DATASET_DIR> ` must contain the following entries:
389
+
390
+ * A subdirectory named ` ILSVRC2012_img_val ` containing the ILSVRC 2012
391
+ validation images. To obtain these images, follow the
392
+ [ instructions at the ILSVRC 2012 website] ( http://image-net.org/challenges/LSVRC/2012/signup ) .
393
+
394
+ * ` val.txt ` from < http://dl.caffe.berkeleyvision.org/caffe_ilsvrc12.tar.gz > .
395
+
299
396
Model information dumper usage
300
397
------------------------------
301
398
@@ -364,7 +461,7 @@ describing a single model. Each such object has the following keys:
364
461
Shared options
365
462
--------------
366
463
367
- The are certain options that both tools accept.
464
+ The are certain options that all tools accept.
368
465
369
466
` -h ` /` --help ` can be used to print a help message:
370
467
0 commit comments