Skip to content

[GSoC] Blockwise Quantization Tool #265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 31, 2024
Merged

[GSoC] Blockwise Quantization Tool #265

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1f4cbba
Blockwise quantization tool
DaniAffCH Jul 7, 2024
846d893
add missing type hints
DaniAffCH Jul 7, 2024
a536245
add min python version check
DaniAffCH Jul 8, 2024
d83d6b2
refactoring
DaniAffCH Jul 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion tools/quantize/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Install dependencies before trying quantization:
pip install -r requirements.txt
```

## Usage
## Quantization Usage

Quantize all models in the Zoo:
```shell
Expand Down Expand Up @@ -52,6 +52,16 @@ models = dict(
python quantize-inc.py model1
```

## Blockwise quantization usage

`block_quantize.py` requires Python>=3.7

To perform weight-only blockwise quantization:

```shell
python block_quantize.py --input_model INPUT_MODEL.onnx --output_model OUTPUT_MODEL.onnx --block_size {block size} --bits {8,16}
```

## Dataset
Some models are quantized with extra datasets.
- [MP-PalmDet](../../models/palm_detection_mediapipe) and [MP-HandPose](../../models/handpose_estimation_mediapipe) are quantized with evaluation set of [FreiHAND](https://lmb.informatik.uni-freiburg.de/resources/datasets/FreihandDataset.en.html). Download the dataset from [this link](https://lmb.informatik.uni-freiburg.de/data/freihand/FreiHAND_pub_v2_eval.zip). Unpack it and replace `path/to/dataset` with the path to `FreiHAND_pub_v2_eval/evaluation/rgb`.
Loading