Bug Report: Incorrect Character Box Order for Inverted Text when return_word_box is enabled

[ajaivasudeve]

🔎 Search before asking

• I have searched the PaddleOCR Docs and found no similar bug report.
• I have searched the PaddleOCR Issues and found no similar bug report.
• I have searched the PaddleOCR Discussions and found no similar bug report.

🐛 Bug (问题描述)

Bug Report: Incorrect Character Box Order for Inverted Text
Description

When processing inverted text using PaddleOCR, the OCR engine successfully detects the correct text content (because angle cls is enabled). However, the order of the character-level bounding boxes remains inconsistent with the orientation of the original input text.

For instance:
When the text is inverted (right-to-left in the image), PaddleOCR correctly detects the text as left-to-right in content. However, the character bounding boxes should follow the original right-to-left order of the inverted input.
Conversely, when text is naturally written left-to-right, the bounding boxes are ordered correctly.

This creates confusion when the bounding box coordinates are used for downstream processing or analysis.
Steps to Reproduce

Input an image containing inverted text (right-to-left visually but left-to-right in logical content).
Use PaddleOCR to extract character-level bounding boxes and text content.
Observe the order of the bounding boxes:
    The text content is correctly detected as left-to-right.
    However, the character bounding boxes do not match the original input's visual (right-to-left) orientation.

Expected Behavior

For inverted text:

The character-level bounding boxes should follow the original orientation of the text in the input image (right-to-left).
The logical text content can still be left-to-right, as detected.

Actual Behavior

The bounding boxes are incorrectly ordered as left-to-right, regardless of the input text orientation.

Example Data

Input 1:

Original text is inverted (right-to-left).
Bounding box coordinates:

[[[249, 369], [263, 368], [263, 394], [249, 395]],
 [[267, 368], [283, 367], [283, 393], [267, 394]],
 ...
]

Detected content: left-to-right (correct).
Bounding box order: left-to-right (incorrect, should match the input's right-to-left orientation).

Input 2:

Normal left-to-right text.
Bounding box coordinates:

[[[259, 707], [277, 707], [278, 734], [260, 734]],
 [[277, 707], [296, 706], [297, 733], [278, 734]],
 ...
]

Detected content: left-to-right (correct).
Bounding box order: left-to-right (correct).

🏃‍♂️ Environment (运行环境)

kaggle notebook.

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

any image, run through OCR before and after inversion.

[GreatV]

For immediate use, you can implement a simple workaround:

1. Detect Text Orientation:

   • Use the angle classification result (cls output) to infer if the input text was inverted.

2. Reorder Bounding Boxes:

   • If the text was inverted, reverse the bounding box order manually while preserving individual box geometry.

Python Example:

def reorder_bounding_boxes(boxes, is_inverted):
    """
    Reorders bounding boxes to match the original input orientation.

    Args:
        boxes (list): List of bounding box coordinates.
        is_inverted (bool): Whether the input text was inverted.

    Returns:
        list: Adjusted bounding box coordinates.
    """
    if is_inverted:
        return boxes[::-1]
    return boxes

[ajaivasudeve]

For immediate use, you can implement a simple workaround:

1. **Detect Text Orientation**:
   
   * Use the angle classification result (`cls` output) to infer if the input text was inverted.

2. **Reorder Bounding Boxes**:
   
   * If the text was inverted, reverse the bounding box order manually while preserving individual box geometry.

Python Example:

def reorder_bounding_boxes(boxes, is_inverted):
"""
Reorders bounding boxes to match the original input orientation.

Args:
    boxes (list): List of bounding box coordinates.
    is_inverted (bool): Whether the input text was inverted.

Returns:
    list: Adjusted bounding box coordinates.
"""
if is_inverted:
    return boxes[::-1]
return boxes

Thank you, I already use a very similar workaround in my project. But the angle_cls seems to fail in some of the cases I am handling. However the recognition for these cases seems to do fine when angle_cls is enabled which i assume is due to some kind of threshold based fallback mechanism set up in the recognition pipeline.

[GreatV]

This bug report describes an issue with the character bounding box order when processing inverted text (right-to-left visual orientation) using PaddleOCR. Below is a detailed analysis and potential recommendations:

---

Issue Analysis:

1. Text Content vs. Character Bounding Boxes:

   • PaddleOCR correctly recognizes the content of inverted text as left-to-right when angle cls is enabled, which is expected behavior.
   • However, the character bounding boxes are ordered left-to-right, regardless of the original visual orientation (right-to-left). This mismatch causes inconsistencies in downstream applications that rely on the spatial order of the bounding boxes.

2. Expected Behavior:

   • For inverted text, the bounding boxes should preserve the original visual orientation (right-to-left) to match the input image, even if the recognized text content is corrected to left-to-right.

3. Root Cause:

   • PaddleOCR's character bounding box ordering might be tied to the logical direction of the recognized text content, rather than the spatial order in the input image. This could be a limitation in the OCR post-processing step, especially when return_word_box is enabled.

---

Recommendations:

1. Modify Post-Processing Logic:

   • Review the character-level bounding box ordering logic in PaddleOCR's source code. Ensure that the spatial order of bounding boxes is preserved based on the input image orientation, even if the recognized text direction is adjusted.

2. Use angle cls Metadata:

   • Since angle cls identifies the text orientation, this information can be used to re-order the bounding boxes for inverted text during post-processing. For example:
     • If the detected angle indicates a 180° rotation (or other inversion), reverse the bounding box order.

3. Custom Post-Processing Script:

   • As a temporary fix, users can implement a custom script to adjust the bounding box order based on the detected text orientation. For example:

     def reorder_bounding_boxes(bboxes, text_orientation):
         if text_orientation == 'inverted':  # Replace with actual angle cls output
             return bboxes[::-1]  # Reverse the bounding box order
         return bboxes

4. File a Feature Request:

   • If this behavior is expected to be common for specific use cases, consider filing a feature request with the PaddleOCR team to add an option for preserving input text orientation in bounding box order.

---

Related Issues/Discussions:

Although no directly related issues were found during the provided searches, this bug aligns with common OCR challenges involving text orientation and bounding box consistency. Users facing similar issues with bounding box order may benefit from contributing to this discussion or creating a new feature request in the PaddleOCR GitHub repository.

---

Conclusion:

This bug represents a mismatch between text recognition (logical content) and bounding box ordering (spatial orientation). Adjustments to the post-processing logic or a custom script can help address this issue until it is resolved in PaddleOCR. Consider contributing this feedback to the PaddleOCR team for further improvement.

---

Response generated by feifei-bot | chatgpt-4o-latest