寻求工业领域特殊字符的解决方案

[figoshi]

🔎 Search before asking

• I have searched the PaddleOCR Docs and found no similar bug report.
• I have searched the PaddleOCR Issues and found no similar bug report.
• I have searched the PaddleOCR Discussions and found no similar bug report.

🐛 Bug (问题描述)

目前使用的是PPV5的识别和检测模型,表格结构用的PPStructureV3

当前对于V5的识别结果比较满意,尤其是(文本识别准确率和手写文字准确率)

期望:
参加下图1=>工业领域有很多的特殊字符,但是PP基本上识别不出来。
图2为之后希望训练的特殊字符全集。
期望能增加特殊字符的准确率,且务必不要丢失(文本识别准确率和手写文字准确率)
目前希望针对PPV5的识别模型进行下优化

问题
能否咨询下详细的解决方案路线呢
1.包括应该准备哪些数据集合,希望不要准备太多数据,类似于十几万上百万那种的,有点顶不住~
(循序渐进一次性优化1-5个字符都行)

2.使用蒸馏还是微调(对于我们的问题他们的区别是什么)

3.有没有PPV5的方案解决方案的文档(只想用V5,因为手写文字准确率太好了)

4.我尝试使用V5的文档进行了下微调(v5rec识别模型),(数据准备5000个中文语料中,以10:1掺杂了特殊字符的图片)

参考文档:
https://paddlepaddle.github.io/PaddleOCR/main/version3.x/module_usage/text_recognition.html#_5

yml如下(基本没有大改)
https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml

发现训练完之后的模型,手写识别的能力和以前文本识别的准确率都下降巨多(基本等于全遗忘)

5.所以希望寻求最正确的解决姿势,万分感谢~

图1
https://private-user-images.githubusercontent.com/60286286/450706559-b4e78340-b6b3-4120-afe9-8b1a547d9969.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDg5NDU3MTAsIm5iZiI6MTc0ODk0NTQxMCwicGF0aCI6Ii82MDI4NjI4Ni80NTA3MDY1NTktYjRlNzgzNDAtYjZiMy00MTIwLWFmZTktOGIxYTU0N2Q5OTY5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA2MDMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNjAzVDEwMTAxMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTYwOWNiNDg2MTFhNWM5N2Q5YjI5NjM2ZDg4YzFiOGQzMzc2YTBjNjU4ZWFiMmExMzExZGE1M2UwODk4ODMxMzMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.e8LyChqw49HRYeC0IJjJCEty9A7ftiQoSwHa67QLlT8
图2
https://private-user-images.githubusercontent.com/60286286/450692402-3ed60961-c129-4c73-b510-5036c71cb80d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDg5NDU3MTAsIm5iZiI6MTc0ODk0NTQxMCwicGF0aCI6Ii82MDI4NjI4Ni80NTA2OTI0MDItM2VkNjA5NjEtYzEyOS00YzczLWI1MTAtNTAzNmM3MWNiODBkLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA2MDMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNjAzVDEwMTAxMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTg5MmM3N2RmYTZmYzVlOTYzM2EyZjgzY2E4NmY4YTJhYWNhMDI1YWRjZWU3ZmFiNjdmMjA1NDgxNDZiMDcxNTgmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.6Zo_Mui4wgECkyDZoQLfRXfbSKkceOouqIrlXJccU84

🏃‍♂️ Environment (运行环境)

paddleocr3.0-gpu

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

Global:
model_name: PP-OCRv5_server_rec # To use static model for inference.
debug: false
use_gpu: true
epoch_num: 75
log_smooth_window: 20
print_batch_step: 10
save_model_dir: ./output/PP-OCRv5_server_rec
save_epoch_step: 1
eval_batch_step: [0, 2000]
cal_metric_during_train: true
calc_epoch_interval: 1
pretrained_model:
checkpoints:
save_inference_dir:
use_visualdl: false
infer_img: doc/imgs_words/ch/word_1.jpg
character_dict_path: ./ppocr/utils/dict/ppocrv5_dict.txt
max_text_length: &max_text_length 25
infer_mode: false
use_space_char: true
distributed: true
save_res_path: ./output/rec/predicts_ppocrv5.txt
d2s_train_image_shape: [3, 48, 320]

Optimizer:
name: Adam
beta1: 0.9
beta2: 0.999
lr:
name: Cosine
learning_rate: 0.0005
warmup_epoch: 1
regularizer:
name: L2
factor: 3.0e-05

Architecture:
model_type: rec
algorithm: SVTR_HGNet
Transform:
Backbone:
name: PPHGNetV2_B4
text_rec: True
Head:
name: MultiHead
head_list:
- CTCHead:
Neck:
name: svtr
dims: 120
depth: 2
hidden_dims: 120
kernel_size: [1, 3]
use_guide: True
Head:
fc_decay: 0.00001
- NRTRHead:
nrtr_dim: 384
max_text_length: *max_text_length

Loss:
name: MultiLoss
loss_config_list:
- CTCLoss:
- NRTRLoss:

PostProcess:
name: CTCLabelDecode

Metric:
name: RecMetric
main_indicator: acc

Train:
dataset:
name: MultiScaleDataSet
ds_width: false
data_dir: ./train_data/
ext_op_transform_idx: 1
label_file_list:
- ./train_data/train_list.txt
transforms:
- DecodeImage:
img_mode: BGR
channel_first: false
- RecAug:
- MultiLabelEncode:
gtc_encode: NRTRLabelEncode
- KeepKeys:
keep_keys:
- image
- label_ctc
- label_gtc
- length
- valid_ratio
sampler:
name: MultiScaleSampler
scales: [[320, 32], [320, 48], [320, 64]]
first_bs: &bs 128
fix_bs: false
divided_factor: [8, 16] # w, h
is_training: True
loader:
shuffle: true
batch_size_per_card: *bs
drop_last: true
num_workers: 16
Eval:
dataset:
name: SimpleDataSet
data_dir: ./train_data
label_file_list:
- ./train_data/val_list.txt
transforms:
- DecodeImage:
img_mode: BGR
channel_first: false
- MultiLabelEncode:
gtc_encode: NRTRLabelEncode
- RecResizeImg:
image_shape: [3, 48, 320]
- KeepKeys:
keep_keys:
- image
- label_ctc
- label_gtc
- length
- valid_ratio
loader:
shuffle: false
drop_last: false
batch_size_per_card: 128
num_workers: 4

[veryy]

数据集多少 什么样的

[figoshi]

数据集多少 什么样的

准备了5000张左右图片如
/val1.png 工程名称 纯中文:5/10
/val2.png KZ1B 英文数字单元:2/10
/val3.png 700m 单位:2/10
/val4.png 4(特殊字符)25 包含特殊字符1/10

数据集中没有手写文字的数据集合和标点符号
数据集中所出现的所有中文字典也就3000多个不同字

但是用户表格 会是 各种天花乱坠 中英文+手写字符+特殊字符所组成的数据
每个表格结构也不一样(但是现在表格结构效果可以接受,就不考虑在这方面优化了)

目前只是特殊字符识别效果不好,我应该如何准备数据集?(大概体量如何,数据集中是否要包含所有中/数字/字母/标点符号/特殊字符)

能否只提升特殊字符识别的效果,而不影响现在V5的能力呢

感谢~

[julesarts]

may you can train a model to detect the specific chars and build a classify model to deal with the new chars.

[figoshi]

How can I assemble and merge the data results of the old model(ppocrV5) with the special character data results of the new model?

Because the V5 model is already very powerful, we don't want to lose some of its excellent abilities (such as handwriting recognition, various excellent Chinese/English/punctuation recognition, etc.)

Thanks

[julesarts]

First of all, I am trying to understand your question. Do you mean that you receive two results , one from the ppocrV5 and another result from a new trained model. you need to choose a better result from them. I think you need an AI model for decision-making. I recommend Gemma or DeepseekR1. With these tools, you can add prompts to help you choose the best result for your business. If your current situation doesn't allow you to deploy a AI model , may you need to build a word map or to train a decistion-making algorithm.

…

________________________________ 发件人: figoshi ***@***.***> 发送时间: 2025年6月9日 13:02 收件人: PaddlePaddle/PaddleOCR ***@***.***> 抄送: jules ***@***.***>; Comment ***@***.***> 主题: Re: [PaddlePaddle/PaddleOCR] 寻求工业领域特殊字符的解决方案 (Discussion #15568) How can I assemble and merge the data results of the old model(ppocvV5) with the special character data results of the new model? thanks ― Reply to this email directly, view it on GitHub<#15568 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BTB2M3MPUOI5KCDFSFKQEW33CUIN5AVCNFSM6AAAAAB6375TSWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNBQGYYTCOA>. You are receiving this because you commented.Message ID: ***@***.***>

[figoshi]

#15552