What actually is the latest & best recognition model to use for finetuning of English text?

[l03126]

From the documentation, it mentions

It is recommended to choose the PP-OCRv3 model (configuration file: ch_PP-OCRv3_rec_distillation.yml，pre-trained model: ch_PP-OCRv3_rec_train.tar，its accuracy and generalization performance is the best pre-training model currently available.

However, in the latest paddleocr.py there is ch_PP-OCRv4_rec_infer.tar and en_PP-OCRv4_rec_infer.tar available. v4 rec models are not mentioned in the Model List as well. It looks like the Model List hasn't been updated in some time.

For my use case, the detection and recognition works well enough with 'en' :
ocr = PaddleOCR(use_angle_cls=True, lang='en')

I need to finetune the model as some characters in my font does not look like traditional characters and are misclassified by the model.

Questions:

1. Is the 'ch' model generally better even for only recognizing plain English text?
2. If where do I find the corresponding trained model file for en_PP-OCRv4_rec_infer or ch_PP-OCRv4_rec_infer? Or is it that the v3 trained models are still recommended over the latest ones?
3. Would restricting the char dictionary directly help improve the OCR results as I only have to recognize Capital letters

[GreatV]

To address your questions thoroughly and based on the provided documentation and discussions:

---

1. Is the 'ch' model generally better even for recognizing plain English text?

The ch models (e.g., ch_PP-OCRv3_rec) are designed to support both Chinese and English text recognition. While they have broader capabilities due to multilingual support, this does not necessarily mean they are better for plain English text. Here are key considerations:

• Accuracy & Specialization: Models specifically trained for English (e.g., en_PP-OCRv3_rec or en_PP-OCRv4_rec) are generally better suited for plain English text since their training data and dictionaries are tailored for English. Using a ch model for English text may introduce unnecessary complexity and potential misclassifications due to the larger character set.
• Use Case: If your use case involves only English text, the en models are more efficient and optimized. However, if mixed Chinese-English text or multilingual support is required, the ch models are ideal.

For your case (plain English text), stick with the en models (en_PP-OCRv3_rec or en_PP-OCRv4_rec).

---

2. Where do I find the corresponding trained model file for en_PP-OCRv4_rec_infer or ch_PP-OCRv4_rec_infer? Should I use V3 or V4?

Availability of V4 Models:

The en_PP-OCRv4_rec_infer and ch_PP-OCRv4_rec_infer models are mentioned in recent updates (e.g., paddleocr.py), but their official documentation (e.g., model list) has not yet been updated to include them. However, these models are available for download in the PaddleOCR GitHub repository:

• Pretrained Model for English V4 Recognition:
  en_PP-OCRv4_rec_train.tar

• Pretrained Model for Chinese V4 Recognition:
  ch_PP-OCRv4_rec_train.tar

For fine-tuning, you should use the trained model versions (*_rec_train.tar) rather than the inference model (*_rec_infer.tar).

V3 or V4 for Fine-Tuning?

• PP-OCRv3: Currently, the documentation highlights PP-OCRv3 as the recommended model due to its strong accuracy and generalization performance. It's a stable and well-documented choice for fine-tuning.
• PP-OCRv4: While V4 models are newer and may offer incremental improvements, they lack detailed official documentation and benchmarking. If you want cutting-edge performance and are comfortable dealing with potential gaps in documentation, you can experiment with V4.

For your case, if your current setup works well with en_PP-OCRv3_rec, stick with it unless you specifically need the latest advancements in V4.

---

3. Would restricting the character dictionary improve results if you only need to recognize capital letters?

Yes, restricting the character dictionary can significantly improve recognition results, especially when dealing with a limited and specific set of characters like capital letters. Here's how and why:

• Smaller Search Space: By reducing the character set to only capital letters (e.g., A-Z), the model's output probabilities are constrained to these characters, reducing confusion with lowercase letters, numbers, or special symbols.
• Improved Accuracy: A smaller dictionary minimizes misclassifications, especially for fonts where uppercase and lowercase characters are visually similar.
• How to Implement:
  1. Modify the character dictionary file used by PaddleOCR. For English, the default dictionary is ppocr/utils/en_dict.txt. Replace its content with only the uppercase letters:

     A
     B
     C
     ...
     Z
     

  2. Update the model configuration to use this custom dictionary during training or inference.

This approach is particularly useful for fine-tuning, as it aligns the model's training objectives with your specific use case (e.g., recognizing a custom font with only capital letters).

---

Additional Insights Based on Discussions

On Fine-Tuning and Pretrained Models:

1. Pretrained Models: For fine-tuning, always use the trained model (*_rec_train.tar), as these contain checkpoints optimized for further training. Inference models (*_rec_infer.tar) are meant only for deployment and cannot be fine-tuned further.
2. Fine-Tuning Framework: Refer to the fine-tuning guide, which provides step-by-step instructions.

On VisionLAN Fine-Tuning:

VisionLAN is another recognition model available in PaddleOCR. To fine-tune VisionLAN:

1. Use the pretrained VisionLAN model (download link).
2. Modify the configuration file to match VisionLAN's architecture and your dataset.
3. Training VisionLAN is modular in PaddleOCR, and you only need to switch the configuration file (e.g., replace en_PP-OCRv4_rec.yml with the VisionLAN-specific configuration).

---

Summary Recommendations

1. Model Selection:

   • Stick with en_PP-OCRv3_rec for stability and documentation.
   • If comfortable with newer models, try en_PP-OCRv4_rec.

2. Character Restriction:

   • Restrict the character dictionary to only uppercase letters for improved accuracy.

3. Pretrained Models:

   • For fine-tuning, use *_rec_train.tar trained models. Avoid *_rec_infer.tar as they're for inference only.

4. VisionLAN:

   • Use VisionLAN if you need advanced features or specific modeling capabilities. Download its pretrained model and adjust the configuration file accordingly.

Let me know if you need further clarification!

---

Response generated by feifei-bot | chatgpt-4o-latest

[FlashlightET]

I was gonna say this response looked ai AF then saw the disclaimer at the end