Requirements of a custom dataset for KIE

[aymennasri]

As the title says, I'm creating a custom dataset consisting of documents of consistent layout that doesn't change but has fields that can be sometimes empty; my questions are

• Should I still draw a box of where the answer should be using PPOCRLabel?
• Should I have the same size for all the images?
• Should I remove any foreign background excluding the document?
• Do the images have to be consistently the same format of the document? Are partial parts of the document good to include in the dataset?
• Are different documents from the same nature accepted?
• Working with paddleocr==2.8, does the structure of the dataset change? Because PPOCRLabel v3.0.0 yields a different format.
• And finally, what are the good characteristics of a good dataset?

[aymennasri]

@liuhongen1234567 @GreatV any advice?

[liuhongen1234567]

The effectiveness of these factors depends on the scenario on your side, and you need to experiment. For example, whether or not to draw bounding boxes for empty answers—both options are possible. If you choose to draw, you can directly match empty answers. If not, and you can’t find a corresponding question-answer pair, it can default to empty. Good data may require a sufficiently large variance and some variation in size, among other things. The most crucial aspect is that it performs well in your specific scenario. These strategies need to be adjusted based on your actual situation.

[aymennasri]

So there is no rules book for custom datasets except for the mentioned structure in the documentation?