Model Pipeline

[mkassm]

Data preparation

• Loading data
• Understanding data
• Data augmentation

Image preprocessing

• Image enhancement like noise removal, morphological operation .. etc
• Image binarization
• Image resize or normalization

Image Segmentation

• Line segmentation
• Word segmentation
• Character segmentation

Layout analysis (low priority)

• Margins cropping or region classification

Touching components (low priority)

• Touching letter segmentation

Word spotting

• Word spotting by example
• Text generation

[mkassm]

Word spotting

• Code example using Siamese network on VML-HD dataset
• Code example using Siamese network on VML-HD dataset
• Code example for general word spotting using CNN on IAM and WG (George Washington) dataset

[gemydev]

Data preparation

Understand the available datasets

• IBN SINA
  Used in Subword Recognition in Historical Arabic Documents using C-GRUs with CNN-GRU 96.10% and in Arabic Word Spotting Based on Key-Points Features and in Arabic Word Recognition System for Historical Documents using Multiscale Representation Method
  Taken from a philosophical book by Ibn Sina.
  Allows images in binary and color formats.
  The dataset consists of 51 folios which correspond to 20722 CCs (almost 500 CC on each folio).

• VML-HD
  Used in Subword Recognition in Historical Arabic Documents using C-GRUs with CNN-GRU 98.60% and in Word Spotting Using Convolutional Siamese Network
  based on five books written by different writers in the years 1088 - 1451
  The 668 pages are fully annotated on the subword level.
  For each page, we manually applied bounding boxes on the different subwords and annotated the sequence of characters.
  It consists of 159,149 sub-word appearances consisting of 326,289 characters out of a vocabulary of 5,509 forms of sub-words.

• George Washington
  Used in Word Spotting Using Convolutional Siamese Network
  collection of scanned pages of handwritten letters from George Washington and his affiliates in the English language.

• [HADARA80P]
  An Historical Handwritten Arabic Dataset for Segmentation-Free Word Spotting.
  Used in Enabling Indexing and Retrieval of Historical Arabic Manuscripts through Template Matching Based Word Spotting
  a dataset comprising 80 pages of an historical handwritten Arabic document.

• KERTAS
  Used in ICFHR_2018_Competition_on_Recognition_of_.pdf
  The dataset consists of over 2000 images taken from various handwritten Arabic manuscripts spanning fourteen centuries.

• IFN/ENIT

Loading data

After studying most of the available data sets that we were exposed to while reading scientific papers in the stage of understanding the challenge and preparing for work, we found that the most appropriate data set is VML-HD.
Link of notebook

Understanding data

based on five books written by different writers in the years 1088 - 1451
The 668 pages are fully annotated on the subword level.
For each page, we manually applied bounding boxes on the different subwords and annotated the sequence of characters.
It consists of 159,149 sub-word appearances consisting of 326,289 characters out of a vocabulary of 5,509 forms of sub-words.
Link of paper
Link of notebook

Data augmentation (if required)

• https://github.com/beratkurar/word-spotting-using-unsupervised-siamese-network/blob/master/homography.py

[OlaAbdallahM]

Image pre-processing

Image Binarization

• document-image-binarization on Persian-docs dataset
• https://github.com/masyagin1998/robin

Image Enhancement

• Morphological Text Processing by cv2
• Denoise-Noisy-Docs by cv2 and ML

[mkassm]

Reference
https://github.com/awslabs/handwritten-text-recognition-for-apache-mxnet