Help needed for very specific health information use case

[Dr-Zorg]

We are trying to create an automated way to capture all data from a leprosy (Hansens disease) patient clinic card (images). This would allow easier patient notification, mapping and treatment followup in diverse developmental setting where leprosy is still endemic.
The forms are a complex combination of structured and unstructured data, including some hand drawings of skin lesions.
The form itself is standardized, and the details just gets filled in.
I am more from a medical background, so the AI learning curve is going too slow (but growing...).
My guess is that this should be very possible with initial document formatting and cleanup, proper bounding boxes and labeling, OCR, and maybe LVM for specific areas or more complex tables.

My initial goal is to demonstrate that this is possible with a working prototype, where-after further development and testing is possible.
Looking for technical partners to get involved or point me in the right direction.

[NicolasCB-7472]

Hi, I'm just a student but I can provide the following feedback to think about and point you towards something.

• Scope: The amount of the entries is important as it may affect the whole system, for example it's not the same a system for 10 to 100 entries than a system to 10000 or more, storage and processing wise.
• Database model: You may want to use a SQL or a NoSQL database.
• Information storage: Some information is more sensitive than other, like full name, phone, adult or not (childs may need stricter protection parameters), and the pictures/drawings, and other afflictions.
• Information quality & backup: The quality of the data is important too given the physical nature of the documents (some documents may be uncompleted), and you want a secure backup apart from the database.
• Nature of system: The method in which you input the old information (physical documents/scans) and the new information into the system. Maybe you want to convert all entries while preserving the current system until it's fully operational.
• Frontend: The delivery method of the application, email, SMS, website notification, mobile app notification.

[NicolasCB-7472]

Depending on the case you may want an automated notification system too if enough cases are found in certain area, or a method to highlight the patients that are critical

[Dr-Zorg]

Dear NicolasCB-7472, Thanks for your contributions.
Yes, those are all relevant aspects. We hope to keep things simple and set it up as an agent that only supports to extract the data, and then sends it on to the specific use group. So not messing around with storing patient data on some server ourselves.
If one could extract the data with something like docling to a markdown format, how would you suggest we get the key value pairs from the data ?

[NicolasCB-7472]

Well I'm not familiar with docling or any other text extracting from image tool software, but with the information provided, if you already have a clinical text file or files .txt, .csv, or any other. You can make in any programming language a parsing tool, that in short would read the document with the information you are interested on (the docling output with the clinical data of the picture in text format).

I've to mention there might be some fields that might be empty (that are null), but I imagine the NIL is always completed you can use that, although you would have to verify that the NIL is unique and not repeated. Even then you can create the primary key to identify the block of data without using the NIL as primary key

[production23]

Hey @Dr-Zorg, this is right up our alley at EvoLearns. We’ve built similar pipelines for scanned documents combining OCR, layout parsing, and sketch segmentation.

We can help build a lightweight prototype that:

Processes scanned clinic cards locally (no patient data stored)

Extracts structured fields (diagnosis, treatment, etc.)

Segments lesion sketches as separate image files

Outputs everything in JSON or CSV for downstream use

If you can share a few redacted samples, we’d be happy to put together a proof-of-concept. Let’s connect, happy to support this important work.

• EvoLearns Team
  [email protected]
  www.evolearns.info

[Dr-Zorg]

Many thanks EvoLearns Team,
Would be great to have a conversation on what you think is possible.
I sent some more info to your email,
A