Fine tune detection model in order to detect only specific parts of image, not all parts

[pariberi]

hi,
I have some photos of bank cards and I want the detection model to detect only the card number part, the detection model of paddleocr detects all data on the card(card number , expiration date, cardholder name, bank name, ...) . I decided to fine tune the model with 13,500 photos.
In first experiments i used "configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_cml.yml" config with "en_PP-OCRv3_det_distii_train" mode as pretrained model. Initially, the precision was 13%, but after more than 1,000 epochs, the precision is still very low ( precision =18% and recall =97% ). It seems that the model still detects a lot of boxes that I don't need.
in the second experiment i used "configs/det/det_mv3_db.yml" as config file and set global.pretrained = "" , in order to train from scratch. in this scenario i achieved precision of 58% and recall of 89 %. however the precision deos not increase more.

1-How should I fine-tune the detection model in order to only detects the card number?
2- which config file and pretrained model is best for this task?

[GreatV]

To fine-tune a detection model to specifically detect only the card number part of bank cards, you need to focus on both the dataset preparation and the model configuration. Here are some steps and recommendations based on your experiments and the provided documents:

1. Dataset Preparation

• Labeling: Ensure that your dataset is accurately labeled, with bounding boxes only around the card number regions. Incorrect or noisy labels can significantly impact the model's performance.
• Data Augmentation: Apply data augmentation techniques to increase the diversity of your dataset. This can include rotations, translations, scaling, and adding noise to simulate different lighting conditions.

2. Model Configuration and Fine-Tuning

• Pretrained Models: Using a pretrained model can significantly speed up the training process and improve performance. For your task, the en_PP-OCRv3_det model is a good starting point.
• Configuration File: The configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_cml.yml configuration file is suitable for fine-tuning. However, you may need to adjust some parameters to better fit your specific task:
  • Learning Rate: Start with a lower learning rate to avoid overshooting the optimal weights. You can gradually increase it if the model is not improving.
  • Batch Size: Depending on your hardware, choose a batch size that allows for stable training. Larger batch sizes can sometimes lead to better generalization.
  • Epochs: While you have trained for over 1,000 epochs, it might be beneficial to monitor the validation loss and stop training early if the model stops improving.

3. Model Selection

• DB (Differentiable Binarization): The DB algorithm is known for its effectiveness in text detection tasks. The en_PP-OCRv3_det model uses this algorithm and is a good choice for your task.
• SAST (Shape-Aware Text Detection): If you want to experiment with other algorithms, SAST is another option. You can use the det_r50_vd_sast_icdar15.yml configuration file and the corresponding pretrained model.

4. Training from Scratch

• If you decide to train from scratch, as in your second experiment, ensure that your dataset is large and diverse enough to allow the model to learn effectively. The det_mv3_db.yml configuration file is a good choice for this approach.

5. Improving Precision

• Post-Processing: Implement post-processing steps to filter out unwanted detections. For example, you can set a confidence threshold to remove low-confidence predictions.
• Regularization: Use regularization techniques like dropout or weight decay to prevent overfitting, especially if your dataset is not very large.

Summary

• Best Config File: configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_cml.yml with the en_PP-OCRv3_det pretrained model.
• Fine-Tuning Strategy: Start with a lower learning rate, use data augmentation, and monitor validation loss to avoid overfitting.
• Post-Processing: Implement confidence thresholds and other filtering techniques to improve precision.

By following these steps, you should be able to fine-tune the detection model to focus specifically on the card number part of the bank cards.

---

Response generated by 🤖 feifei-bot | deepseek-chat

⚠️ This is an AI-generated response. Please verify any critical information.