Skip to content

Commit a00f830

Browse files
ceci3ceci3
authored
add source free (PaddlePaddle#974)
* add source-free * add source-free * add asp * update * add fleet * add docs * rename * fix unittest
1 parent d66afeb commit a00f830

27 files changed

+2097
-187
lines changed

‎demo/auto-compression/README.md‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# 使用预测模型进行量化训练示例
2+
3+
预测模型保存接口：
4+
动态图使用``paddle.jit.save``保存；
5+
静态图使用``paddle.static.save_inference_model``保存。
6+
7+
本示例将介绍如何使用预测模型进行蒸馏量化训练，
8+
首先使用接口``paddleslim.quant.quant_aware_with_infermodel``训练量化模型，
9+
训练完成后，使用接口``paddleslim.quant.export_quant_infermodel``将训好的量化模型导出为预测模型。
10+
11+
## 分类模型量化训练流程
12+
13+
### 1. 准备数据
14+
15+
在``demo``文件夹下创建``data``文件夹，将``ImageNet``数据集解压在``data``文件夹下，解压后``data/ILSVRC2012``文件夹下应包含以下文件：
16+
- ``'train'``文件夹，训练图片
17+
- ``'train_list.txt'``文件
18+
- ``'val'``文件夹，验证图片
19+
- ``'val_list.txt'``文件
20+
21+
### 2. 准备需要量化的模型
22+
23+
飞桨图像识别套件PaddleClas是飞桨为工业界和学术界所准备的一个图像识别任务的工具集，本示例使用该套件产出imagenet分类模型。
24+
#### 2.1 下载PaddleClas release/2.3分支代码
25+
<https://github.com/PaddlePaddle/PaddleClas/archive/refs/heads/release/2.3.zip>
26+
解压后，进入PaddleClas目录
27+
```
28+
cd PaddleClas-release-2.3
29+
```
30+
#### 2.2 下载MobileNetV2预训练模型
31+
在PaddleClas根目录创建``pretrained``文件夹：
32+
```
33+
mkdir pretrained
34+
```
35+
36+
下载预训练模型
37+
分类预训练模型库地址 <https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/algorithm_introduction/ImageNet_models.md>
38+
MobileNetV2预训练模型地址 <https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_pretrained.pdparams>
39+
执行下载命令：
40+
```
41+
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV2_pretrained.pdparams -O ./pretrained/MobileNetV2_pretrained.pdparams
42+
```
43+
44+
#### 2.3 导出预测模型
45+
PaddleClas代码库根目录执行如下命令，导出预测模型
46+
```
47+
python tools/export_model.py \
48+
-c ppcls/configs/ImageNet/MobileNetV2/MobileNetV2.yaml \
49+
-o Global.pretrained_model=pretrained/MobileNetV2_pretrained \
50+
-o Global.save_inference_dir=infermodel_mobilenetv2
51+
```
52+
#### 2.4 测试模型精度
53+
拷贝``infermodel_mobilenetv2``文件夹到``PaddleSlim/demo/auto-compression/``文件夹。
54+
```
55+
cd PaddleSlim/demo/auto-compression/
56+
```
57+
使用[eval.py](../quant_post/eval.py)脚本得到模型的分类精度：
58+
```
59+
python ../quant_post/eval.py --model_path infermodel_mobilenetv2 --model_name inference.pdmodel --params_name inference.pdiparams
60+
```
61+
精度输出为:
62+
```
63+
top1_acc/top5_acc= [0.71918 0.90568]
64+
```
65+
66+
### 3. 进行多策略融合压缩
67+
68+
每一个小章节代表一种多策略融合压缩，不代表需要串行执行。
69+
70+
### 3.1 进行量化蒸馏压缩
71+
蒸馏量化训练示例脚本为[demo_imagenet.py](./demo_imagenet.py)，使用接口``paddleslim.auto_compression.AutoCompression``对模型进行量化训练。运行命令为：
72+
```
73+
python demo_imagenet.py \
74+
--model_dir='infermodel_mobilenetv2' \
75+
--model_filename='inference.pdmodel' \
76+
--params_filename='./inference.pdiparams' \
77+
--save_dir='./save_qat_mbv2/' \
78+
--devices='gpu' \
79+
--batch_size=64 \
80+
--config_path='./configs/CV/mbv2_qat_dis.yaml'
81+
```
82+
83+
### 3.2 进行离线量化超参搜索压缩
84+
离线量化超参搜索压缩示例脚本为[demo_imagenet.py](./demo_imagenet.py)，使用接口``paddleslim.auto_compression.AutoCompression``对模型进行压缩。运行命令为：
85+
```
86+
python demo_imagenet.py \
87+
--model_dir='infermodel_mobilenetv2' \
88+
--model_filename='inference.pdmodel' \
89+
--params_filename='./inference.pdiparams' \
90+
--save_dir='./save_qat_mbv2/' \
91+
--devices='gpu' \
92+
--batch_size=64 \
93+
--config_path='./configs/CV/mbv2_ptq_hpo.yaml'
94+
```
95+
96+
### 3.3 进行剪枝蒸馏策略融合压缩
97+
注意：本示例为对BERT模型进行ASP稀疏。
98+
首先参考[脚本](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/examples/language_model/bert#%E9%A2%84%E6%B5%8B)得到可部署的模型。
99+
剪枝蒸馏压缩示例脚本为[demo_glue.py](./demo_glue.py)，使用接口``paddleslim.auto_compression.AutoCompression``对模型进行压缩。运行命令为：
100+
```
101+
python demo_glue.py \
102+
--model_dir='./static_bert_models/' \
103+
--model_filename='bert.pdmodel' \
104+
--params_filename='bert.pdiparams' \
105+
--save_dir='./save_asp_bert/' \
106+
--devices='gpu' \
107+
--batch_size=32 \
108+
--task='sst-2' \
109+
--config_path='./configs/NLP/bert_asp_dis.yaml'
110+
```
111+
112+
### 3.4 进行非结构化稀疏蒸馏策略融合压缩
113+
非结构化稀疏蒸馏压缩示例脚本为[demo_imagenet.py](./demo_imagenet.py)，使用接口``paddleslim.auto_compression.AutoCompression``对模型进行压缩。运行命令为：
114+
```
115+
python demo_imagenet.py \
116+
--model_dir='infermodel_mobilenetv2' \
117+
--model_filename='inference.pdmodel' \
118+
--params_filename='./inference.pdiparams' \
119+
--save_dir='./save_qat_mbv2/' \
120+
--devices='gpu' \
121+
--batch_size=64 \
122+
--config_path='./configs/CV/xxx.yaml'
123+
```

‎demo/auto-compression/configs/CV/mbv2_ptq_hpo.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
HyperParameterOptimization:
2+
batch_num:
3+
- 4
4+
- 16
5+
bias_correct:
6+
- true
7+
hist_percent:
8+
- 0.999
9+
- 0.99999
10+
max_quant_count: 20
11+
ptq_algo:
12+
- KL
13+
- hist
14+
weight_quantize_type:
15+
- channel_wise_abs_max
16+
Quantization:
17+
activation_bits: 8
18+
quantize_op_types:
19+
- conv2d
20+
- depthwise_conv2d
21+
- mul
22+
weight_bits: 8

‎demo/auto-compression/configs/CV/mbv2_qat_dis.yaml‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
Distillation:
2+
distill_lambda: 1.0
3+
distill_loss: l2_loss
4+
distill_node_pair:
5+
- teacher_conv2d_54.tmp_0
6+
- conv2d_54.tmp_0
7+
- teacher_conv2d_55.tmp_0
8+
- conv2d_55.tmp_0
9+
- teacher_conv2d_57.tmp_0
10+
- conv2d_57.tmp_0
11+
- teacher_elementwise_add_0
12+
- elementwise_add_0
13+
- teacher_conv2d_61.tmp_0
14+
- conv2d_61.tmp_0
15+
- teacher_elementwise_add_1
16+
- elementwise_add_1
17+
- teacher_elementwise_add_2
18+
- elementwise_add_2
19+
- teacher_conv2d_67.tmp_0
20+
- conv2d_67.tmp_0
21+
- teacher_elementwise_add_3
22+
- elementwise_add_3
23+
- teacher_elementwise_add_4
24+
- elementwise_add_4
25+
- teacher_elementwise_add_5
26+
- elementwise_add_5
27+
- teacher_conv2d_75.tmp_0
28+
- conv2d_75.tmp_0
29+
- teacher_elementwise_add_6
30+
- elementwise_add_6
31+
- teacher_elementwise_add_7
32+
- elementwise_add_7
33+
- teacher_conv2d_81.tmp_0
34+
- conv2d_81.tmp_0
35+
- teacher_elementwise_add_8
36+
- elementwise_add_8
37+
- teacher_elementwise_add_9
38+
- elementwise_add_9
39+
- teacher_conv2d_87.tmp_0
40+
- conv2d_87.tmp_0
41+
- teacher_linear_1.tmp_0
42+
- linear_1.tmp_0
43+
merge_feed: true
44+
teacher_model_dir: ./MobileNetV2_ssld_infer
45+
teacher_model_filename: inference.pdmodel
46+
teacher_params_filename: inference.pdiparams
47+
Quantization:
48+
activation_bits: 8
49+
is_full_quantize: false
50+
not_quant_pattern:
51+
- skip_quant
52+
quantize_op_types:
53+
- conv2d
54+
- depthwise_conv2d
55+
weight_bits: 8
56+
TrainConfig:
57+
epochs: 1
58+
eval_iter: 1000
59+
learning_rate: 0.0001
60+
optimizer: SGD
61+
origin_metric: 0.765
62+
weight_decay: 4.0e-05

‎demo/auto-compression/configs/NLP/bert_asp_dis.yaml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
Distillation:
2+
distill_lambda: 1.0
3+
distill_loss: l2_loss
4+
distill_node_pair:
5+
- teacher_tmp_9
6+
- tmp_9
7+
- teacher_tmp_12
8+
- tmp_12
9+
- teacher_tmp_15
10+
- tmp_15
11+
- teacher_tmp_18
12+
- tmp_18
13+
- teacher_tmp_21
14+
- tmp_21
15+
- teacher_tmp_24
16+
- tmp_24
17+
- teacher_tmp_27
18+
- tmp_27
19+
- teacher_tmp_30
20+
- tmp_30
21+
- teacher_tmp_33
22+
- tmp_33
23+
- teacher_tmp_36
24+
- tmp_36
25+
- teacher_tmp_39
26+
- tmp_39
27+
- teacher_tmp_42
28+
- tmp_42
29+
- teacher_linear_147.tmp_1
30+
- linear_147.tmp_1
31+
merge_feed: true
32+
teacher_model_dir: ../auto-compression_origin/static_bert_models
33+
teacher_model_filename: bert.pdmodel
34+
teacher_params_filename: bert.pdiparams
35+
Prune:
36+
prune_algo: asp
37+
TrainConfig:
38+
epochs: 3
39+
eval_iter: 1000
40+
learning_rate: 2.0e-05
41+
optim_args:
42+
weight_decay: 0.0
43+
optimizer: AdamW
44+
origin_metric: 0.93

‎demo/auto-compression/configs/NLP/bert_ptq_hpo.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
HyperParameterOptimization:
2+
batch_num:
3+
- 4
4+
- 16
5+
bias_correct:
6+
- true
7+
hist_percent:
8+
- 0.999
9+
- 0.99999
10+
max_quant_count: 20
11+
ptq_algo:
12+
- KL
13+
- hist
14+
weight_quantize_type:
15+
- channel_wise_abs_max
16+
Quantization:
17+
activation_bits: 8
18+
quantize_op_types:
19+
- conv2d
20+
- depthwise_conv2d
21+
- mul
22+
weight_bits: 8

‎demo/auto-compression/configs/NLP/bert_qat_dis.yaml‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
Distillation:
2+
distill_lambda: 1.0
3+
distill_loss: l2_loss
4+
distill_node_pair:
5+
- teacher_tmp_9
6+
- tmp_9
7+
- teacher_tmp_12
8+
- tmp_12
9+
- teacher_tmp_15
10+
- tmp_15
11+
- teacher_tmp_18
12+
- tmp_18
13+
- teacher_tmp_21
14+
- tmp_21
15+
- teacher_tmp_24
16+
- tmp_24
17+
- teacher_tmp_27
18+
- tmp_27
19+
- teacher_tmp_30
20+
- tmp_30
21+
- teacher_tmp_33
22+
- tmp_33
23+
- teacher_tmp_36
24+
- tmp_36
25+
- teacher_tmp_39
26+
- tmp_39
27+
- teacher_tmp_42
28+
- tmp_42
29+
- teacher_linear_147.tmp_1
30+
- linear_147.tmp_1
31+
merge_feed: true
32+
teacher_model_dir: ../auto-compression_origin/static_bert_models
33+
teacher_model_filename: bert.pdmodel
34+
teacher_params_filename: bert.pdiparams
35+
Quantization:
36+
activation_bits: 8
37+
is_full_quantize: false
38+
not_quant_pattern:
39+
- skip_quant
40+
quantize_op_types:
41+
- conv2d
42+
- depthwise_conv2d
43+
- mul
44+
- matmul
45+
weight_bits: 8
46+
TrainConfig:
47+
epochs: 3
48+
eval_iter: 1000
49+
learning_rate: 0.0001
50+
optimizer: SGD
51+
optim_args:
52+
weight_decay: 4.0e-05
53+
origin_metric: 0.93

0 commit comments

Comments
 (0)