在文本识别训练中，PP-OCRv3 模型使用了GTC策略，其中SAR分支参数量大，当训练数据为简单场景时模型容易过拟合，导致微调效果不佳，建议去除GTC策略 #14707

o0ii0o · 2025-02-18T05:31:33Z

o0ii0o
Feb 18, 2025

如果配置文件是ch_PP-OCRv3_rec_distillation.yml，具体该怎么改？
配置文件内容如下：
Global:
debug: false
use_gpu: true
epoch_num: 800
log_smooth_window: 20
print_batch_step: 10
save_model_dir: ./output/rec_ppocr_v3_distillation
save_epoch_step: 3
eval_batch_step: [0, 2000]
cal_metric_during_train: true
pretrained_model:
checkpoints:
save_inference_dir:
use_visualdl: false
infer_img: doc/imgs_words/ch/word_1.jpg
character_dict_path: ppocr/utils/ppocr_keys_v1.txt
max_text_length: &max_text_length 25
infer_mode: false
use_space_char: true
distributed: true
save_res_path: ./output/rec/predicts_ppocrv3_distillation.txt
d2s_train_image_shape: [3, 48, -1]

Optimizer:
name: Adam
beta1: 0.9
beta2: 0.999
lr:
name: Piecewise
decay_epochs : [700]
values : [0.0005, 0.00005]
warmup_epoch: 5
regularizer:
name: L2
factor: 3.0e-05

Architecture:
model_type: &model_type "rec"
name: DistillationModel
algorithm: Distillation
Models:
Teacher:
pretrained:
freeze_params: false
return_all_feats: true
model_type: *model_type
algorithm: SVTR_LCNet
Transform:
Backbone:
name: MobileNetV1Enhance
scale: 0.5
last_conv_stride: [1, 2]
last_pool_type: avg
last_pool_kernel_size: [2, 2]
Head:
name: MultiHead
head_list:
- CTCHead:
Neck:
name: svtr
dims: 64
depth: 2
hidden_dims: 120
use_guide: True
Head:
fc_decay: 0.00001
- SARHead:
enc_dim: 512
max_text_length: *max_text_length
Student:
pretrained:
freeze_params: false
return_all_feats: true
model_type: *model_type
algorithm: SVTR_LCNet
Transform:
Backbone:
name: MobileNetV1Enhance
scale: 0.5
last_conv_stride: [1, 2]
last_pool_type: avg
last_pool_kernel_size: [2, 2]
Head:
name: MultiHead
head_list:
- CTCHead:
Neck:
name: svtr
dims: 64
depth: 2
hidden_dims: 120
use_guide: True
Head:
fc_decay: 0.00001
- SARHead:
enc_dim: 512
max_text_length: *max_text_length
Loss:
name: CombinedLoss
loss_config_list:

DistillationDMLLoss:
weight: 1.0
act: "softmax"
use_log: true
model_name_pairs:
- ["Student", "Teacher"]
  key: head_out
  multi_head: True
  dis_head: ctc
  name: dml_ctc
DistillationDMLLoss:
weight: 0.5
act: "softmax"
use_log: true
model_name_pairs:
- ["Student", "Teacher"]
  key: head_out
  multi_head: True
  dis_head: sar
  name: dml_sar
DistillationDistanceLoss:
weight: 1.0
mode: "l2"
model_name_pairs:
- ["Student", "Teacher"]
  key: backbone_out
DistillationCTCLoss:
weight: 1.0
model_name_list: ["Student", "Teacher"]
key: head_out
multi_head: True
DistillationSARLoss:
weight: 1.0
model_name_list: ["Student", "Teacher"]
key: head_out
multi_head: True

PostProcess:
name: DistillationCTCLabelDecode
model_name: ["Student", "Teacher"]
key: head_out
multi_head: True

Metric:
name: DistillationMetric
base_metric_name: RecMetric
main_indicator: acc
key: "Student"
ignore_space: False

Train:
dataset:
name: SimpleDataSet
data_dir: ./train_data/
ext_op_transform_idx: 1
label_file_list:
- ./train_data/train_list.txt
transforms:
- DecodeImage:
img_mode: BGR
channel_first: false
- RecConAug:
prob: 0.5
ext_data_num: 2
image_shape: [48, 320, 3]
max_text_length: *max_text_length
- RecAug:
- MultiLabelEncode:
- RecResizeImg:
image_shape: [3, 48, 320]
- KeepKeys:
keep_keys:
- image
- label_ctc
- label_sar
- length
- valid_ratio
loader:
shuffle: true
batch_size_per_card: 128
drop_last: true
num_workers: 4
Eval:
dataset:
name: SimpleDataSet
data_dir: ./train_data
label_file_list:
- ./train_data/val_list.txt
transforms:
- DecodeImage:
img_mode: BGR
channel_first: false
- MultiLabelEncode:
- RecResizeImg:
image_shape: [3, 48, 320]
- KeepKeys:
keep_keys:
- image
- label_ctc
- label_sar
- length
- valid_ratio
loader:
shuffle: false
drop_last: false
batch_size_per_card: 128
num_workers: 4

GreatV · 2025-02-18T05:33:49Z

GreatV
Feb 18, 2025
Maintainer

在 PP-OCRv3 模型中，GTC（Global Text Context）策略通过引入 SAR（Sequence-Aware Representation）分支来增强模型的文本识别能力。然而，当训练数据为简单场景时，SAR 分支的参数量较大，容易导致模型过拟合，进而影响微调效果。为了去除 GTC 策略，您需要对配置文件 ch_PP-OCRv3_rec_distillation.yml 进行以下修改：

1. 修改 `Architecture` 部分

在 Architecture 部分，您需要移除 Teacher 和 Student 模型中的 SARHead 分支，只保留 CTCHead 分支。具体修改如下：

Architecture:
  model_type: &model_type "rec"
  name: DistillationModel
  algorithm: Distillation
  Models:
    Teacher:
      pretrained:
      freeze_params: false
      return_all_feats: true
      model_type: *model_type
      algorithm: SVTR_LCNet
      Transform:
      Backbone:
        name: MobileNetV1Enhance
        scale: 0.5
        last_conv_stride: [1, 2]
        last_pool_type: avg
        last_pool_kernel_size: [2, 2]
      Head:
        name: MultiHead
        head_list:
          - CTCHead:
              Neck:
                name: svtr
                dims: 64
                depth: 2
                hidden_dims: 120
                use_guide: True
              Head:
                fc_decay: 0.00001
          # 移除 SARHead 分支
    Student:
      pretrained:
      freeze_params: false
      return_all_feats: true
      model_type: *model_type
      algorithm: SVTR_LCNet
      Transform:
      Backbone:
        name: MobileNetV1Enhance
        scale: 0.5
        last_conv_stride: [1, 2]
        last_pool_type: avg
        last_pool_kernel_size: [2, 2]
      Head:
        name: MultiHead
        head_list:
          - CTCHead:
              Neck:
                name: svtr
                dims: 64
                depth: 2
                hidden_dims: 120
                use_guide: True
              Head:
                fc_decay: 0.00001
          # 移除 SARHead 分支

2. 修改 `Loss` 部分

在 Loss 部分，您需要移除与 SARHead 相关的损失函数，包括 DistillationSARLoss 和 DistillationDMLLoss 中的 sar 部分。具体修改如下：

Loss:
  name: CombinedLoss
  loss_config_list:
  - DistillationDMLLoss:
      weight: 1.0
      act: "softmax"
      use_log: true
      model_name_pairs:
      - ["Student", "Teacher"]
      key: head_out
      multi_head: True
      dis_head: ctc
      name: dml_ctc
  # 移除 dml_sar 部分
  - DistillationDistanceLoss:
      weight: 1.0
      mode: "l2"
      model_name_pairs:
      - ["Student", "Teacher"]
      key: backbone_out
  - DistillationCTCLoss:
      weight: 1.0
      model_name_list: ["Student", "Teacher"]
      key: head_out
      multi_head: True
  # 移除 DistillationSARLoss

3. 修改 `PostProcess` 部分

在 PostProcess 部分，您需要确保只使用 CTC 相关的解码器。由于 SARHead 已被移除，PostProcess 部分无需额外修改。

4. 修改 `Train` 和 `Eval` 部分

在 Train 和 Eval 部分，您需要移除与 SAR 相关的标签和数据预处理步骤。具体修改如下：

Train:
  dataset:
    name: SimpleDataSet
    data_dir: ./train_data/
    ext_op_transform_idx: 1
    label_file_list:
    - ./train_data/train_list.txt
    transforms:
    - DecodeImage:
        img_mode: BGR
        channel_first: false
    - RecConAug:
        prob: 0.5
        ext_data_num: 2
        image_shape: [48, 320, 3]
        max_text_length: *max_text_length
    - RecAug:
    - MultiLabelEncode:
    - RecResizeImg:
        image_shape: [3, 48, 320]
    - KeepKeys:
        keep_keys:
        - image
        - label_ctc
        - length
        - valid_ratio
  loader:
    shuffle: true
    batch_size_per_card: 128
    drop_last: true
    num_workers: 4

Eval:
  dataset:
    name: SimpleDataSet
    data_dir: ./train_data
    label_file_list:
    - ./train_data/val_list.txt
    transforms:
    - DecodeImage:
        img_mode: BGR
        channel_first: false
    - MultiLabelEncode:
    - RecResizeImg:
        image_shape: [3, 48, 320]
    - KeepKeys:
        keep_keys:
        - image
        - label_ctc
        - length
        - valid_ratio
  loader:
    shuffle: false
    drop_last: false
    batch_size_per_card: 128
    num_workers: 4

总结

通过以上修改，您已经成功移除了 PP-OCRv3 模型中的 GTC 策略，仅保留了 CTCHead 分支。这将有助于减少模型在简单场景下的过拟合问题，并提升微调效果。

Response generated by 🤖 feifei-bot | deepseek-chat

⚠️ This is an AI-generated response. Please verify any critical information.

4 replies

shesung Feb 26, 2025

改了报错

o0ii0o Feb 26, 2025
Author

请问你是什么问题，我现在训练模型遇到了问题想请教一下

shesung Feb 27, 2025

照这样改 config，会提示 assert len(self.head_list) >= 2

o0ii0o Feb 27, 2025
Author

那看来遇到的问题不一样，加个微信谈吧，k17870086362

nissansz · 2025-02-18T09:57:25Z

nissansz
Feb 18, 2025

你是什么语种？准确率能达到多少了？

2 replies

o0ii0o Feb 18, 2025
Author

是中文繁体，准确率只有68%，但是我的数据都是合成的，场景比较简单所以想试试去除SAR分支能不能提高

nissansz Feb 18, 2025

可以发样本来试试？微信nlanguage

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

在文本识别训练中，PP-OCRv3 模型使用了GTC策略，其中SAR分支参数量大，当训练数据为简单场景时模型容易过拟合，导致微调效果不佳，建议去除GTC策略 #14707

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments 6 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

在文本识别训练中，PP-OCRv3 模型使用了GTC策略，其中SAR分支参数量大，当训练数据为简单场景时模型容易过拟合，导致微调效果不佳，建议去除GTC策略 #14707

Uh oh!

o0ii0o Feb 18, 2025

Replies: 2 comments · 6 replies

Uh oh!

GreatV Feb 18, 2025 Maintainer

1. 修改 Architecture 部分

2. 修改 Loss 部分

3. 修改 PostProcess 部分

4. 修改 Train 和 Eval 部分

总结

Uh oh!

shesung Feb 26, 2025

Uh oh!

o0ii0o Feb 26, 2025 Author

Uh oh!

shesung Feb 27, 2025

Uh oh!

o0ii0o Feb 27, 2025 Author

Uh oh!

nissansz Feb 18, 2025

Uh oh!

o0ii0o Feb 18, 2025 Author

Uh oh!

nissansz Feb 18, 2025

o0ii0o
Feb 18, 2025

Replies: 2 comments 6 replies

GreatV
Feb 18, 2025
Maintainer

1. 修改 `Architecture` 部分

2. 修改 `Loss` 部分

3. 修改 `PostProcess` 部分

4. 修改 `Train` 和 `Eval` 部分

o0ii0o Feb 26, 2025
Author

o0ii0o Feb 27, 2025
Author

nissansz
Feb 18, 2025

o0ii0o Feb 18, 2025
Author