Best way to use PaddleOCR for end-to-end Key-Value Pair Extraction from ID cards

[atharvjairath]

Hi,
I am trying to build Key-value pair Extraction from ID cards (like Driving licenses etc), where the key would be Name, Address, DOB, etc.
What would be the best way to use PaddleOCR to achieve the above?

[ThanhLV97]

You should train KIE model with your dataset.

[LDOUBLEV]

Please refer to kie algorithm: https://github.com/PaddlePaddle/PaddleOCR/blob/release%2F2.5/ppstructure/docs/kie_en.md

[ThanhLV97]

@LDOUBLEV as my mention in previous issue. In order to onnx converting that need paddle format but the paddle format convertingscript is maybe wrong with kie model. so How I convert kie model to onnx format using
torchscript or something else.

[atharvjairath]

is there any way I can run a run a inference on my data using trained KIE model? @LDOUBLEV @thanh15dt2

[shahzain110]

have you found any solutions?

[AbdulDD]

you should also look into DONUT (OCR-FREE document understading) and Pix2Struct from HuggingFace.
Both are E2E models and can perform:

1. Key information extraction (extracting key and their respective values from images (like in your case))
2. Document Visual Query Answering (Doc VQA).