Accuracy on cpp infer is lower compared to python infer. why ? how to fix?

[itzananthakrishnan]

I have exported the model and created a cpp infer script which will print accuracy.

#include <include/ocr_rec.h>
#include <opencv2/opencv.hpp>
#include <include/utility.h>
#include <iostream>
#include <vector>
#include <fstream>
#include <string>
#include <algorithm>

using namespace PaddleOCR;

std::string read_label(const std::string& label_path) {
    std::ifstream ifs(label_path);
    std::string label;
    std::getline(ifs, label);
    return label;
}

int main(int argc, char **argv) {
    if (argc < 5) {
        std::cerr << "Usage: " << argv[0]
                  << " image_dir model_dir dict_path labels_dir" << std::endl;
        return -1;
    }

    std::string image_dir = argv[1];
    std::string model_dir = argv[2];
    std::string dict_path = argv[3];
    std::string label_dir = argv[4];

    std::vector<cv::String> cv_all_img_names;
    cv::glob(image_dir, cv_all_img_names);
    if (cv_all_img_names.empty()) {
        std::cerr << "No images found in: " << image_dir << std::endl;
        return -1;
    }

    std::cout << "Total images found: " << cv_all_img_names.size() << std::endl;

    CRNNRecognizer recognizer(
        model_dir,            // model_dir
        true,                 // use_gpu
        0,                    // gpu_id
        4000,                 // gpu_mem
        4,                    // num_threads
        false,                // use_mkldnn
        dict_path,            // dictionary
        true,                // use_tensorrt
        "fp32",               // precision
        1,                    // rec_batch_num
        48,                   // rec_image_height (fixed)
        320                 // rec_image_width (max padded width)
    );

    std::vector<cv::Mat> img_list;
    std::vector<std::string> img_names;
    std::vector<std::string> ground_truths;

    for (const auto& img_path : cv_all_img_names) {
        cv::Mat img = cv::imread(img_path, cv::IMREAD_COLOR);
        if (img.empty()) {
            std::cerr << "Could not load image: " << img_path << std::endl;
            continue;
        }

        cv::cvtColor(img, img, cv::COLOR_BGR2RGB);

        std::string filename = Utility::basename(img_path);
        std::string label_path = label_dir + "/" + filename.substr(0, filename.find_last_of('.')) + ".txt";

        std::string gt = read_label(label_path);
        if (gt.empty()) {
            std::cerr << "Label not found or empty for: " << filename << std::endl;
            continue;
        }

        img_list.push_back(img);
        img_names.push_back(filename);
        ground_truths.push_back(gt);
    }

    if (img_list.empty()) {
        std::cerr << "No valid images found for processing." << std::endl;
        return -1;
    }

    std::vector<std::string> rec_texts(img_list.size());
    std::vector<float> rec_scores(img_list.size());
    std::vector<double> times;

    recognizer.Run(img_list, rec_texts, rec_scores, times);

    int correct = 0;
    for (size_t i = 0; i < rec_texts.size(); ++i) {
        std::string pred = rec_texts[i];
        std::string gt = ground_truths[i];

        // Clean both prediction and ground truth
        auto clean = [](std::string &s) {
            s.erase(std::remove_if(s.begin(), s.end(), ::isspace), s.end());
            std::transform(s.begin(), s.end(), s.begin(), ::toupper);
        };

        clean(pred);
        clean(gt);

        bool match = (pred == gt);
        if (match) correct++;

        std::cout << "Image: " << img_names[i]
                  << " | Prediction: " << pred
                  << " | GroundTruth: " << gt
                  << " | Score: " << rec_scores[i]
                  << " | " << (match ? "✅" : "❌") << std::endl;
    }

    float accuracy = 100.0f * correct / rec_texts.size();
    std::cout << "\nAccuracy: " << accuracy << "% (" << correct << "/" << rec_texts.size() << ")\n";

    if (times.size() >= 3) {
        std::cout << "Preprocess time: " << times[0] / 1000.0 << " s" << std::endl;
        std::cout << "Inference time: " << times[1] / 1000.0 << " s" << std::endl;
        std::cout << "Postprocess time: " << times[2] / 1000.0 << " s" << std::endl;
    }


    return 0;
}

- the accuracy i got here is 97.85 and on python infer i got 99% accuracy.

import os
import time
import sys
from paddleocr import TextRecognition
import shutil


MODEL_NAME = "PP-OCRv4_mobile_rec"
MODEL_DIR = "/inference_models/PP-OCRv4_mobile"
INPUT_DIR = sys.argv[1]
GROUNDTRUTH_DIR = sys.argv[2]
OUT_DIR = sys.argv[3]
WRONG_DIR = os.path.join(OUT_DIR, "wrong_detections")

IMAGE_EXTS = {".jpg", ".jpeg", ".png"}


os.makedirs(OUT_DIR, exist_ok=True)
os.makedirs(WRONG_DIR, exist_ok=True)

model = TextRecognition(model_name=MODEL_NAME, model_dir=MODEL_DIR)

def process_text(text):
    return text.strip().replace(" ", "").upper()

truth_files = sorted([f for f in os.listdir(GROUNDTRUTH_DIR) if f.endswith(".txt")])
image_files = sorted([f for f in os.listdir(INPUT_DIR) if os.path.splitext(f)[1].lower() in IMAGE_EXTS])
common_basenames = set(os.path.splitext(f)[0] for f in image_files) & set(os.path.splitext(f)[0] for f in truth_files)

correct = 0
wrong_examples = []
total_images = 0
start_time = time.time()

for base in sorted(common_basenames):
    img_path = os.path.join(INPUT_DIR, base + ".jpg")
    if not os.path.exists(img_path):
        for ext in IMAGE_EXTS:
            alt_path = os.path.join(INPUT_DIR, base + ext)
            if os.path.exists(alt_path):
                img_path = alt_path
                break
        else:
            continue  # skip if image not found

    outputs = model.predict(input=img_path, batch_size=25)
    texts = [res['rec_text'] for res in outputs if 'rec_text' in res]
    pred_text = process_text("".join(texts))

    gt_path = os.path.join(GROUNDTRUTH_DIR, base + ".txt")
    with open(gt_path, "r", encoding="utf-8") as f:
        gt_text = process_text(f.read())

    if pred_text == gt_text:
        correct += 1
    else:
        # Save wrong image to WRONG_DIR
        shutil.copy(img_path, os.path.join(WRONG_DIR, os.path.basename(img_path)))

        # Save OCR and GT text in .txt file for review
        wrong_text_file = os.path.join(WRONG_DIR, base + ".txt")
        with open(wrong_text_file, "w") as f:
            f.write(f"GT: {gt_text}\nOCR: {pred_text}\n")

        if len(wrong_examples) < 10:
            wrong_examples.append((base, gt_text, pred_text))

    total_images += 1

end_time = time.time()

wrong = total_images - correct
accuracy = (correct / total_images) * 100 if total_images else 0
fps = total_images / (end_time - start_time)

print("\n==========================================================\n")
print(f"Accuracy: {accuracy:.2f}%")
print(f"Correct Plates: {correct}")
print(f"Wrong Plates: {wrong}")
print("Wrong Examples:\n")
print("----------------------------------------------------------")
for fname, gt, pred in wrong_examples:
    print(f"GT: {gt} | OCR: {pred}")

print(f"\nInference FPS: {fps:.2f}\n")

how to improve the accuracy on cpp infer.

[zhang-prog]

Which branch of PaddleOCR are you using and what version of PaddlePaddle?

PS: The code formatting is chaotic and makes it difficult to read. If possible, please use backticks to enclose the code. THX!

[itzananthakrishnan]

i am using latest paddleocr version = 3.0.2 and paddlepaddle-gpu 3.0.0
ps: updated the code structure, please review

[zhang-prog]

In the Python script, the target images are sorted, but in the C++ script, they are not.

truth_files = sorted([f for f in os.listdir(GROUNDTRUTH_DIR) if f.endswith(".txt")])
image_files = sorted([f for f in os.listdir(INPUT_DIR) if os.path.splitext(f)[1].lower() in IMAGE_EXTS])

Can it be ensured that the images correspond correctly to the ground truth when using the C++ script for inference?

[itzananthakrishnan]

Yes, i updated the code to sort the images and labels. still the accuracy is same

[zhang-prog]

And, please paste the contents of the inference.yml file in the model directory.

[itzananthakrishnan]

Global:
  model_name: PP-OCRv5_mobile_rec
Hpi:
  backend_configs:
    paddle_infer:
      trt_dynamic_shapes: &id001
        x:
        - - 1
          - 3
          - 48
          - 160
        - - 1
          - 3
          - 48
          - 320
        - - 8
          - 3
          - 48
          - 3200
    tensorrt:
      dynamic_shapes: *id001
PreProcess:
  transform_ops:
  - DecodeImage:
      channel_first: false
      img_mode: BGR
  - MultiLabelEncode:
      gtc_encode: NRTRLabelEncode
  - RecResizeImg:
      image_shape:
      - 3
      - 48
      - 320
  - KeepKeys:
      keep_keys:
      - image
      - label_ctc
      - label_gtc
      - length
      - valid_ratio
PostProcess:
  name: CTCLabelDecode
  character_dict:
  - '0'
  - '1'
  - '2'
  - '3'
  - '4'
  - '5'
  - '6'
  - '7'
  - '8'
  - '9'
  - A
  - B
  - C
  - D
  - E
  - F
  - G
  - H
  - I
  - J
  - K
  - L
  - M
  - N
  - O
  - P
  - Q
  - R
  - S
  - T
  - U
  - V
  - W
  - X
  - Y
  - Z

[zhang-prog]

1. I’ve noticed a potential inconsistency regarding which model is being used. The inference.yml file refers to PP-OCRv5_mobile_rec, whereas the Python code specifies the model directory as PP-OCRv4_mobile_rec ?
2. It seems that the character_dict used might not be set to the default configuration. Could you check if the recognizer.dict_path in your C++ code aligns correctly with the settings in inference.yml ?

[itzananthakrishnan]

1. Cause the actual model i fine-tuned is en_PPOCRv4 and i changed name in inference.yml for cpp , because only PPOCRv5 model is accepted by the paddleocr cpp version. i compiled using latest paddle devolop branch, it's just a naming change nothing more
2. yes

[zhang-prog]

get it. In fact, the 3.0 branch has not yet updated the relevant C++ inference code. The current C++ inference code is derived from the 2.x branch. Although we have adapted it for Paddle 3.0, there may still be some discrepancies and usability challenges.

[itzananthakrishnan]

So, We have to address the difference in model accuracy, but we can't do anything about it for now, right?. Thanks for the reply

[zhang-prog]

And, could you please provide an image for which the Python prediction is correct but the C++ prediction is incorrect? I need it to reproduce the issue you discovered.

[itzananthakrishnan]

sure

Pred: MH45AE4943 | GT: MH45AF4943 | Score: 0.963805 | ❌

this is an example image where i get crct prediction on python for this but on cpp i get wrong.

[zhang-prog]

If your settings are correct, I suspect the discrepancy might be due to differences in the pre-processing and post-processing logic between the Python and C++ implementations. We already have plans to implement C++ inference in the PaddleOCR 3.0 OCR pipeline.