SER + RE模型串联，报错：Corrupt JPEG data: premature end of data segment

[Jacoo520]

• windows 11
• pyhton：3.8
• cuda：11.8
• cudnn：8.6
• paddlepaddle-gpu:2.4.2
• paddleocr:2.7.0

关键信息抽取，SER + RE模型串联运行报错：Corrupt JPEG data: premature end of data segment

• 运行命令：

python3 ./tools/infer_kie_token_ser_re.py \
-c configs/kie/vi_layoutxlm/re_vi_layoutxlm_xfund_zh.yml \
-o Architecture.Backbone.checkpoints=./pretrained_model/re_vi_layoutxlm_xfund_pretrained/best_accuracy \
Global.infer_img=./train_data/XFUND/zh_val/image/zh_val_42.jpg \
-c_ser configs/kie/vi_layoutxlm/ser_vi_layoutxlm_xfund_zh.yml \
-o_ser Architecture.Backbone.checkpoints=./pretrained_model/ser_vi_layoutxlm_xfund_pretrained/best_accuracy

• 输出日志：

D:\anaconda3\envs\paddleocr\lib\site-packages\setuptools\sandbox.py:13: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
D:\anaconda3\envs\paddleocr\lib\site-packages\pkg_resources\__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('mpl_toolkits')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
D:\anaconda3\envs\paddleocr\lib\site-packages\pkg_resources\__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
[2024/03/09 17:41:56] ppocr INFO: Architecture :
[2024/03/09 17:41:56] ppocr INFO:     Backbone :
[2024/03/09 17:41:56] ppocr INFO:         checkpoints : D:/py_xuexi/PaddleOCR/pretrained_model/ser_vi_layoutxlm_xfund_pretrained/best_accuracy
[2024/03/09 17:41:56] ppocr INFO:         mode : vi
[2024/03/09 17:41:56] ppocr INFO:         name : LayoutXLMForSer
[2024/03/09 17:41:56] ppocr INFO:         num_classes : 7
[2024/03/09 17:41:56] ppocr INFO:         pretrained : True
[2024/03/09 17:41:56] ppocr INFO:     Transform : None
[2024/03/09 17:41:56] ppocr INFO:     algorithm : LayoutXLM
[2024/03/09 17:41:56] ppocr INFO:     model_type : kie
[2024/03/09 17:41:56] ppocr INFO: Eval :
[2024/03/09 17:41:56] ppocr INFO:     dataset :
[2024/03/09 17:41:56] ppocr INFO:         data_dir : train_data/XFUND/zh_val/image
[2024/03/09 17:41:56] ppocr INFO:         label_file_list : ['train_data/XFUND/zh_val/val.json']
[2024/03/09 17:41:56] ppocr INFO:         name : SimpleDataSet
[2024/03/09 17:41:56] ppocr INFO:         transforms :
[2024/03/09 17:41:56] ppocr INFO:             DecodeImage :
[2024/03/09 17:41:56] ppocr INFO:                 channel_first : False
[2024/03/09 17:41:56] ppocr INFO:                 img_mode : RGB
[2024/03/09 17:41:56] ppocr INFO:             VQATokenLabelEncode :
[2024/03/09 17:41:56] ppocr INFO:                 algorithm : LayoutXLM
[2024/03/09 17:41:56] ppocr INFO:                 class_path : train_data/XFUND/class_list_xfun.txt
[2024/03/09 17:41:56] ppocr INFO:                 contains_re : False
[2024/03/09 17:41:56] ppocr INFO:                 order_method : tb-yx
[2024/03/09 17:41:56] ppocr INFO:                 use_textline_bbox_info : True
[2024/03/09 17:41:56] ppocr INFO:             VQATokenPad :
[2024/03/09 17:41:56] ppocr INFO:                 max_seq_len : 512
[2024/03/09 17:41:56] ppocr INFO:                 return_attention_mask : True
[2024/03/09 17:41:56] ppocr INFO:             VQASerTokenChunk :
[2024/03/09 17:41:56] ppocr INFO:                 max_seq_len : 512
[2024/03/09 17:41:56] ppocr INFO:             Resize :
[2024/03/09 17:41:56] ppocr INFO:                 size : [224, 224]
[2024/03/09 17:41:56] ppocr INFO:             NormalizeImage :
[2024/03/09 17:41:56] ppocr INFO:                 mean : [123.675, 116.28, 103.53]
[2024/03/09 17:41:56] ppocr INFO:                 order : hwc
[2024/03/09 17:41:56] ppocr INFO:                 scale : 1
[2024/03/09 17:41:56] ppocr INFO:                 std : [58.395, 57.12, 57.375]
[2024/03/09 17:41:56] ppocr INFO:             ToCHWImage : None
[2024/03/09 17:41:56] ppocr INFO:             KeepKeys :
[2024/03/09 17:41:56] ppocr INFO:                 keep_keys : ['input_ids', 'bbox', 'attention_mask', 'token_type_ids', 'image', 'labels']
[2024/03/09 17:41:56] ppocr INFO:     loader :
[2024/03/09 17:41:56] ppocr INFO:         batch_size_per_card : 8
[2024/03/09 17:41:56] ppocr INFO:         drop_last : False
[2024/03/09 17:41:56] ppocr INFO:         num_workers : 4
[2024/03/09 17:41:56] ppocr INFO:         shuffle : False
[2024/03/09 17:41:56] ppocr INFO: Global :
[2024/03/09 17:41:56] ppocr INFO:     amp_custom_white_list : ['scale', 'concat', 'elementwise_add']
[2024/03/09 17:41:56] ppocr INFO:     cal_metric_during_train : False
[2024/03/09 17:41:56] ppocr INFO:     d2s_train_image_shape : [3, 224, 224]
[2024/03/09 17:41:56] ppocr INFO:     distributed : False
[2024/03/09 17:41:56] ppocr INFO:     epoch_num : 200
[2024/03/09 17:41:56] ppocr INFO:     eval_batch_step : [0, 19]
[2024/03/09 17:41:56] ppocr INFO:     infer_img : D:/py_xuexi/PaddleOCR/ppstructure/docs/kie/input/zh_val_42.jpg
[2024/03/09 17:41:56] ppocr INFO:     kie_det_model_dir : None
[2024/03/09 17:41:56] ppocr INFO:     kie_rec_model_dir : None
[2024/03/09 17:41:56] ppocr INFO:     log_smooth_window : 10
[2024/03/09 17:41:56] ppocr INFO:     print_batch_step : 10
[2024/03/09 17:41:56] ppocr INFO:     save_epoch_step : 2000
[2024/03/09 17:41:56] ppocr INFO:     save_inference_dir : None
[2024/03/09 17:41:56] ppocr INFO:     save_model_dir : ./output/ser_vi_layoutxlm_xfund_zh
[2024/03/09 17:41:56] ppocr INFO:     save_res_path : ./output/ser/xfund_zh/res
[2024/03/09 17:41:56] ppocr INFO:     seed : 2022
[2024/03/09 17:41:56] ppocr INFO:     use_gpu : True
[2024/03/09 17:41:56] ppocr INFO:     use_visualdl : False
[2024/03/09 17:41:56] ppocr INFO: Loss :
[2024/03/09 17:41:56] ppocr INFO:     key : backbone_out
[2024/03/09 17:41:56] ppocr INFO:     name : VQASerTokenLayoutLMLoss
[2024/03/09 17:41:56] ppocr INFO:     num_classes : 7
[2024/03/09 17:41:56] ppocr INFO: Metric :
[2024/03/09 17:41:56] ppocr INFO:     main_indicator : hmean
[2024/03/09 17:41:56] ppocr INFO:     name : VQASerTokenMetric
[2024/03/09 17:41:56] ppocr INFO: Optimizer :
[2024/03/09 17:41:56] ppocr INFO:     beta1 : 0.9
[2024/03/09 17:41:56] ppocr INFO:     beta2 : 0.999
[2024/03/09 17:41:56] ppocr INFO:     lr :
[2024/03/09 17:41:56] ppocr INFO:         epochs : 200
[2024/03/09 17:41:56] ppocr INFO:         learning_rate : 5e-05
[2024/03/09 17:41:56] ppocr INFO:         name : Linear
[2024/03/09 17:41:56] ppocr INFO:         warmup_epoch : 2
[2024/03/09 17:41:56] ppocr INFO:     name : AdamW
[2024/03/09 17:41:56] ppocr INFO:     regularizer :
[2024/03/09 17:41:56] ppocr INFO:         factor : 0.0
[2024/03/09 17:41:56] ppocr INFO:         name : L2
[2024/03/09 17:41:56] ppocr INFO: PostProcess :
[2024/03/09 17:41:56] ppocr INFO:     class_path : train_data/XFUND/class_list_xfun.txt
[2024/03/09 17:41:56] ppocr INFO:     name : VQASerTokenLayoutLMPostProcess
[2024/03/09 17:41:56] ppocr INFO: Train :
[2024/03/09 17:41:56] ppocr INFO:     dataset :
[2024/03/09 17:41:56] ppocr INFO:         data_dir : train_data/XFUND/zh_train/image
[2024/03/09 17:41:56] ppocr INFO:         label_file_list : ['train_data/XFUND/zh_train/train.json']
[2024/03/09 17:41:56] ppocr INFO:         name : SimpleDataSet
[2024/03/09 17:41:56] ppocr INFO:         ratio_list : [1.0]
[2024/03/09 17:41:56] ppocr INFO:         transforms :
[2024/03/09 17:41:56] ppocr INFO:             DecodeImage :
[2024/03/09 17:41:56] ppocr INFO:                 channel_first : False
[2024/03/09 17:41:56] ppocr INFO:                 img_mode : RGB
[2024/03/09 17:41:56] ppocr INFO:             VQATokenLabelEncode :
[2024/03/09 17:41:56] ppocr INFO:                 algorithm : LayoutXLM
[2024/03/09 17:41:56] ppocr INFO:                 class_path : train_data/XFUND/class_list_xfun.txt
[2024/03/09 17:41:56] ppocr INFO:                 contains_re : False
[2024/03/09 17:41:56] ppocr INFO:                 order_method : tb-yx
[2024/03/09 17:41:56] ppocr INFO:                 use_textline_bbox_info : True
[2024/03/09 17:41:56] ppocr INFO:             VQATokenPad :
[2024/03/09 17:41:56] ppocr INFO:                 max_seq_len : 512
[2024/03/09 17:41:56] ppocr INFO:                 return_attention_mask : True
[2024/03/09 17:41:56] ppocr INFO:             VQASerTokenChunk :
[2024/03/09 17:41:56] ppocr INFO:                 max_seq_len : 512
[2024/03/09 17:41:56] ppocr INFO:             Resize :
[2024/03/09 17:41:56] ppocr INFO:                 size : [224, 224]
[2024/03/09 17:41:56] ppocr INFO:             NormalizeImage :
[2024/03/09 17:41:56] ppocr INFO:                 mean : [123.675, 116.28, 103.53]
[2024/03/09 17:41:56] ppocr INFO:                 order : hwc
[2024/03/09 17:41:56] ppocr INFO:                 scale : 1
[2024/03/09 17:41:56] ppocr INFO:                 std : [58.395, 57.12, 57.375]
[2024/03/09 17:41:56] ppocr INFO:             ToCHWImage : None
[2024/03/09 17:41:56] ppocr INFO:             KeepKeys :
[2024/03/09 17:41:56] ppocr INFO:                 keep_keys : ['input_ids', 'bbox', 'attention_mask', 'token_type_ids', 'image', 'labels']
[2024/03/09 17:41:56] ppocr INFO:     loader :
[2024/03/09 17:41:56] ppocr INFO:         batch_size_per_card : 8
[2024/03/09 17:41:56] ppocr INFO:         drop_last : False
[2024/03/09 17:41:56] ppocr INFO:         num_workers : 4
[2024/03/09 17:41:56] ppocr INFO:         shuffle : True
[2024/03/09 17:41:56] ppocr INFO: profiler_options : None
[2024/03/09 17:41:56] ppocr INFO: train with paddle 2.4.2 and device Place(gpu:0)
W0309 17:41:57.992060  3396 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 8.9, Driver API Version: 12.0, Runtime API Version: 11.6
W0309 17:41:58.006819  3396 gpu_resources.cc:91] device: 0, cuDNN Version: 8.6.
[2024/03/09 17:42:01] ppocr INFO: resume from D:/py_xuexi/PaddleOCR/pretrained_model/ser_vi_layoutxlm_xfund_pretrained/best_accuracy
[2024-03-09 17:42:05,263] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/transformers/layoutxlm_base/sentencepiece.bpe.model and saved to C:\Users\jacoo\.paddlenlp\models\layoutxlm-base-uncased
[2024-03-09 17:42:05,719] [    INFO] - Downloading sentencepiece.bpe.model from https://bj.bcebos.com/paddlenlp/models/transformers/layoutxlm_base/sentencepiece.bpe.model
100%|██████████████████████████████████████████████████████████████████████████████| 4.83M/4.83M [00:06<00:00, 785kB/s]
[2024-03-09 17:42:13,028] [    INFO] - tokenizer config file saved in C:\Users\jacoo\.paddlenlp\models\layoutxlm-base-uncased\tokenizer_config.json
[2024-03-09 17:42:13,028] [    INFO] - Special tokens file saved in C:\Users\jacoo\.paddlenlp\models\layoutxlm-base-uncased\special_tokens_map.json
Corrupt JPEG data: premature end of data segment

[dragronstranding]

楼主，你解决了吗？我也遇到这个问题，我在用基于PaddleInference的预测，出现了和你一样的报错