SER index error: IndexError: (OutOfRange) label value should less than the shape of axis dimension when label value(2) not equal to ignore_index(-100)

[piarosebelledelapaz]

问题描述 / Problem Description

I want to train the SER model using my custom dataset which has 2 class labels in total, but I am getting an error when it comes to training. What could be the problem here?

运行环境 / Runtime Environment

• OS:
• Paddle:
• PaddleOCR:

复现代码 / Reproduction Code

完整报错 / Complete Error Message

(luks_paddleOCR) PS C:\Users\piade\LUKS\luks_ocr> python tools/train.py -c configs/kie/vi_layoutxlm/ser_vi_layoutxlm_xfund_zh.yml
[2024/06/15 13:52:58] ppocr INFO: Architecture :
[2024/06/15 13:52:58] ppocr INFO: Backbone :
[2024/06/15 13:52:58] ppocr INFO: checkpoints : None
[2024/06/15 13:52:58] ppocr INFO: mode : vi
[2024/06/15 13:52:58] ppocr INFO: name : LayoutXLMForSer
[2024/06/15 13:52:58] ppocr INFO: num_classes : 2
[2024/06/15 13:52:58] ppocr INFO: pretrained : True
[2024/06/15 13:52:58] ppocr INFO: Transform : None
[2024/06/15 13:52:58] ppocr INFO: algorithm : LayoutXLM
[2024/06/15 13:52:58] ppocr INFO: model_type : kie
[2024/06/15 13:52:58] ppocr INFO: Eval :
[2024/06/15 13:52:58] ppocr INFO: dataset :
[2024/06/15 13:52:58] ppocr INFO: data_dir : ./dataset/relation-extraction/img_re_eval
[2024/06/15 13:52:58] ppocr INFO: label_file_list : ['./dataset/relation-extraction/re_eval.txt']
[2024/06/15 13:52:58] ppocr INFO: name : SimpleDataSet
[2024/06/15 13:52:58] ppocr INFO: transforms :
[2024/06/15 13:52:58] ppocr INFO: DecodeImage :
[2024/06/15 13:52:58] ppocr INFO: channel_first : False
[2024/06/15 13:52:58] ppocr INFO: img_mode : RGB
[2024/06/15 13:52:58] ppocr INFO: VQATokenLabelEncode :
[2024/06/15 13:52:58] ppocr INFO: algorithm : LayoutXLM
[2024/06/15 13:52:58] ppocr INFO: class_path : ./dataset/relation-extraction/class_list.txt
[2024/06/15 13:52:58] ppocr INFO: contains_re : False
[2024/06/15 13:52:58] ppocr INFO: order_method : tb-yx
[2024/06/15 13:52:58] ppocr INFO: use_textline_bbox_info : True
[2024/06/15 13:52:58] ppocr INFO: VQATokenPad :
[2024/06/15 13:52:58] ppocr INFO: max_seq_len : 512
[2024/06/15 13:52:58] ppocr INFO: return_attention_mask : True
[2024/06/15 13:52:58] ppocr INFO: VQASerTokenChunk :
[2024/06/15 13:52:58] ppocr INFO: max_seq_len : 512
[2024/06/15 13:52:58] ppocr INFO: Resize :
[2024/06/15 13:52:58] ppocr INFO: size : [224, 224]
[2024/06/15 13:52:58] ppocr INFO: NormalizeImage :
[2024/06/15 13:52:58] ppocr INFO: mean : [123.675, 116.28, 103.53]
[2024/06/15 13:52:58] ppocr INFO: order : hwc
[2024/06/15 13:52:58] ppocr INFO: scale : 1
[2024/06/15 13:52:58] ppocr INFO: std : [58.395, 57.12, 57.375]
[2024/06/15 13:52:58] ppocr INFO: ToCHWImage : None
[2024/06/15 13:52:58] ppocr INFO: KeepKeys :
[2024/06/15 13:52:58] ppocr INFO: keep_keys : ['input_ids', 'bbox', 'attention_mask', 'token_type_ids', 'image', 'labels']
[2024/06/15 13:52:58] ppocr INFO: loader :
[2024/06/15 13:52:58] ppocr INFO: batch_size_per_card : 8
[2024/06/15 13:52:58] ppocr INFO: drop_last : False
[2024/06/15 13:52:58] ppocr INFO: num_workers : 4
[2024/06/15 13:52:58] ppocr INFO: shuffle : False
[2024/06/15 13:52:58] ppocr INFO: Global :
[2024/06/15 13:52:58] ppocr INFO: amp_custom_white_list : ['scale', 'concat', 'elementwise_add']
[2024/06/15 13:52:58] ppocr INFO: cal_metric_during_train : False
[2024/06/15 13:52:58] ppocr INFO: d2s_train_image_shape : [3, 224, 224]
[2024/06/15 13:52:58] ppocr INFO: distributed : False
[2024/06/15 13:52:58] ppocr INFO: epoch_num : 2
[2024/06/15 13:52:58] ppocr INFO: eval_batch_step : [0, 19]
[2024/06/15 13:52:58] ppocr INFO: infer_img : ppstructure/docs/kie/input/zh_val_42.jpg
[2024/06/15 13:52:58] ppocr INFO: kie_det_model_dir : ./pretrained_models/multilang_PP-OCR_V3_det_rot10_adam_best_model/best_model/model.pdparams
[2024/06/15 13:52:58] ppocr INFO: kie_rec_model_dir : ./pretrained_models/det_r50_db++_icdar15_train/best_accuracy.pdparams
[2024/06/15 13:52:58] ppocr INFO: log_smooth_window : 10
[2024/06/15 13:52:58] ppocr INFO: print_batch_step : 1
[2024/06/15 13:52:58] ppocr INFO: save_epoch_step : 2000
[2024/06/15 13:52:58] ppocr INFO: save_inference_dir : None
[2024/06/15 13:52:58] ppocr INFO: save_model_dir : ./output/ser_vi_layoutxlm_xfund_zh
[2024/06/15 13:52:58] ppocr INFO: save_res_path : ./output/ser/xfund_zh/res/
[2024/06/15 13:52:58] ppocr INFO: seed : 2022
[2024/06/15 13:52:58] ppocr INFO: use_gpu : False
[2024/06/15 13:52:58] ppocr INFO: use_visualdl : False
[2024/06/15 13:52:58] ppocr INFO: Loss :
[2024/06/15 13:52:58] ppocr INFO: key : backbone_out
[2024/06/15 13:52:58] ppocr INFO: name : VQASerTokenLayoutLMLoss
[2024/06/15 13:52:58] ppocr INFO: num_classes : 2
[2024/06/15 13:52:58] ppocr INFO: Metric :
[2024/06/15 13:52:58] ppocr INFO: main_indicator : hmean
[2024/06/15 13:52:58] ppocr INFO: name : VQASerTokenMetric
[2024/06/15 13:52:58] ppocr INFO: Optimizer :
[2024/06/15 13:52:58] ppocr INFO: beta1 : 0.9
[2024/06/15 13:52:58] ppocr INFO: beta2 : 0.999
[2024/06/15 13:52:58] ppocr INFO: lr :
[2024/06/15 13:52:58] ppocr INFO: epochs : 2
[2024/06/15 13:52:58] ppocr INFO: learning_rate : 5e-05
[2024/06/15 13:52:58] ppocr INFO: name : Linear
[2024/06/15 13:52:58] ppocr INFO: warmup_epoch : 2
[2024/06/15 13:52:58] ppocr INFO: name : AdamW
[2024/06/15 13:52:58] ppocr INFO: regularizer :
[2024/06/15 13:52:58] ppocr INFO: factor : 0.0
[2024/06/15 13:52:58] ppocr INFO: name : L2
[2024/06/15 13:52:58] ppocr INFO: PostProcess :
[2024/06/15 13:52:58] ppocr INFO: class_path : ./dataset/relation-extraction/class_list.txt
[2024/06/15 13:52:58] ppocr INFO: name : VQASerTokenLayoutLMPostProcess
[2024/06/15 13:52:58] ppocr INFO: Train :
[2024/06/15 13:52:58] ppocr INFO: dataset :
[2024/06/15 13:52:58] ppocr INFO: data_dir : ./dataset/relation-extraction/img_re_train
[2024/06/15 13:52:58] ppocr INFO: label_file_list : ['./dataset/relation-extraction/re_train.txt']
[2024/06/15 13:52:58] ppocr INFO: name : SimpleDataSet
[2024/06/15 13:52:58] ppocr INFO: ratio_list : [1.0]
[2024/06/15 13:52:58] ppocr INFO: transforms :
[2024/06/15 13:52:58] ppocr INFO: DecodeImage :
[2024/06/15 13:52:58] ppocr INFO: channel_first : False
[2024/06/15 13:52:58] ppocr INFO: img_mode : RGB
[2024/06/15 13:52:58] ppocr INFO: VQATokenLabelEncode :
[2024/06/15 13:52:58] ppocr INFO: algorithm : LayoutXLM
[2024/06/15 13:52:58] ppocr INFO: class_path : ./dataset/relation-extraction/class_list.txt
[2024/06/15 13:52:58] ppocr INFO: contains_re : False
[2024/06/15 13:52:58] ppocr INFO: order_method : tb-yx
[2024/06/15 13:52:58] ppocr INFO: use_textline_bbox_info : True
[2024/06/15 13:52:58] ppocr INFO: VQATokenPad :
[2024/06/15 13:52:58] ppocr INFO: max_seq_len : 512
[2024/06/15 13:52:58] ppocr INFO: return_attention_mask : True
[2024/06/15 13:52:58] ppocr INFO: VQASerTokenChunk :
[2024/06/15 13:52:58] ppocr INFO: max_seq_len : 512
[2024/06/15 13:52:58] ppocr INFO: Resize :
[2024/06/15 13:52:58] ppocr INFO: size : [224, 224]
[2024/06/15 13:52:58] ppocr INFO: NormalizeImage :
[2024/06/15 13:52:58] ppocr INFO: mean : [123.675, 116.28, 103.53]
[2024/06/15 13:52:58] ppocr INFO: order : hwc
[2024/06/15 13:52:58] ppocr INFO: scale : 1
[2024/06/15 13:52:58] ppocr INFO: std : [58.395, 57.12, 57.375]
[2024/06/15 13:52:58] ppocr INFO: ToCHWImage : None
[2024/06/15 13:52:58] ppocr INFO: KeepKeys :
[2024/06/15 13:52:58] ppocr INFO: keep_keys : ['input_ids', 'bbox', 'attention_mask', 'token_type_ids', 'image', 'labels']
[2024/06/15 13:52:58] ppocr INFO: loader :
[2024/06/15 13:52:58] ppocr INFO: batch_size_per_card : 8
[2024/06/15 13:52:58] ppocr INFO: drop_last : False
[2024/06/15 13:52:58] ppocr INFO: num_workers : 4
[2024/06/15 13:52:58] ppocr INFO: shuffle : True
[2024/06/15 13:52:58] ppocr INFO: profiler_options : None
[2024/06/15 13:52:58] ppocr INFO: train with paddle 2.5.0 and device Place(cpu)
[2024/06/15 13:52:58] ppocr INFO: Initialize indexs of datasets:['./dataset/relation-extraction/re_train.txt']
list index out of range
[2024-06-15 13:53:01,547] [ INFO] - Already cached C:\Users\piade.paddlenlp\models\layoutxlm-base-uncased\sentencepiece.bpe.model
[2024-06-15 13:53:02,098] [ INFO] - tokenizer config file saved in C:\Users\piade.paddlenlp\models\layoutxlm-base-uncased\tokenizer_config.json
[2024-06-15 13:53:02,103] [ INFO] - Special tokens file saved in C:\Users\piade.paddlenlp\models\layoutxlm-base-uncased\special_tokens_map.json
[2024/06/15 13:53:02] ppocr INFO: Initialize indexs of datasets:['./dataset/relation-extraction/re_eval.txt']
[2024-06-15 13:53:02,109] [ INFO] - Already cached C:\Users\piade.paddlenlp\models\layoutxlm-base-uncased\sentencepiece.bpe.model
[2024-06-15 13:53:02,661] [ INFO] - tokenizer config file saved in C:\Users\piade.paddlenlp\models\layoutxlm-base-uncased\tokenizer_config.json
[2024-06-15 13:53:02,661] [ INFO] - Special tokens file saved in C:\Users\piade.paddlenlp\models\layoutxlm-base-uncased\special_tokens_map.json
[2024-06-15 13:53:02,671] [ INFO] - Already cached C:\Users\piade.paddlenlp\models\vi-layoutxlm-base-uncased\model_state.pdparams
[2024-06-15 13:53:23,167] [ INFO] - Weights of LayoutXLMForTokenClassification not initialized from pretrained model: ['classifier.weight', 'classifier.bias']
[2024/06/15 13:53:27] ppocr INFO: train dataloader has 7 iters
[2024/06/15 13:53:27] ppocr INFO: valid dataloader has 1 iters
[2024/06/15 13:53:27] ppocr INFO: During the training process, after the 0th iteration, an evaluation is run every 19 iterations
Traceback (most recent call last):
File "tools/train.py", line 229, in
main(config, device, logger, vdl_writer, seed)
File "tools/train.py", line 200, in main
program.train(config, train_dataloader, valid_dataloader, device, model,
File "C:\Users\piade\LUKS\luks_ocr\tools\program.py", line 311, in train
loss = loss_class(preds, batch)
File "C:\Users\piade\anaconda3\envs\luks_paddleOCR\lib\site-packages\paddle\nn\layer\layers.py", line 1254, in call
return self.forward(*inputs, **kwargs)
File "C:\Users\piade\LUKS\luks_ocr\ppocr\losses\vqa_token_layoutlm_loss.py", line 41, in forward
loss = self.loss_class(active_output, active_label)
File "C:\Users\piade\anaconda3\envs\luks_paddleOCR\lib\site-packages\paddle\nn\layer\layers.py", line 1254, in call
return self.forward(*inputs, **kwargs)
File "C:\Users\piade\anaconda3\envs\luks_paddleOCR\lib\site-packages\paddle\nn\layer\loss.py", line 373, in forward
ret = paddle.nn.functional.cross_entropy(
File "C:\Users\piade\anaconda3\envs\luks_paddleOCR\lib\site-packages\paddle\nn\functional\loss.py", line 2707, in cross_entropy
, out = C_ops.cross_entropy_with_softmax(
IndexError: (OutOfRange) label value should less than the shape of axis dimension when label value(2) not equal to ignore_index(-100), But received label value as 2 and shape of axis dimension is 2
[Hint: Expected lbl < axis_dim, but received lbl:2 >= axis_dim:2.] (at ..\paddle\phi\kernels\funcs\cross_entropy.cc:74

可能解决方案 / Possible solutions

附件 / Appendix

[cactuer]

num_classes设置错了，由于采用BIO标注，假设字典中包含n个字段（包含other）时，则类别数为2n-1; 假设字典中包含n个字段（不含other）时，则类别数为2n+1。否则在train过程会报：IndexError: (OutOfRange) label value should less than the shape of axis dimension 。

参考官方文档：https://paddlepaddle.github.io/PaddleOCR/main/ppocr/model_train/kie.html#21