Handwritten Malayalam word recognition using hybrid deep neural network models

Abstract

Handwritten document analysis remains a significant challenge in patternrecognition, particularly for scripts with complex morphology such as Malayalam.Even today, handwritten Malayalam documents are widely used in government,police, and legal settings, yet there are no reliable automated methods for reading orretrieving information from them. The lack of publicly available handwrittenMalayalam word datasets further restricts progress, limiting the use of deep learningmodels. This study addresses these challenges through a comprehensive researcheffort involving the construction of a new handwritten word dataset, thedevelopment of a hybrid deep learning–based handwritten word recognitionframework, and the design of a practical document retrieval system based onrecognized word content.The first component of this work focuses on constructing the MalayalamHandwritten Word Dataset (MHWD). Handwritten material was sourced frompolice First Information Statements (FIS) and further enriched with samplesprovided by writers from different age groups. All document images underwent auniform preprocessing sequence that included input image enhancement,binarization, contour detection, and word-level segmentation using bounding boxes.To define the vocabulary, the most frequently occurring words and domain specificwords in the collected FIS documents were manually shortlisted, yielding 321 wordcategories. The number of samples from real documents was insufficient andimbalanced. So, additional handwritten instances were collected from individualsaged 10 to 65, yielding 150 samples per class. This balanced dataset serves as adependable base for training and evaluating Malayalam handwritten wordrecognition methods.The second component of this research introduces the proposed handwrittenMalayalam word recognition model, termed TrA-MHWR. The model integrates aDeiT-based vision transformer for feature extraction with an Attention-basedFeedforward Neural Network (AFFNN) as the recognition module. The transformerencoder provides a rich, context-aware representation of word images, while theAFFNN enhances the discriminative capability of these features through multi-headattention and residual learning. This combination allows the model to effectivelyhandle handwriting variability and visually similar word forms commonlyencountered in real-world documents.The third contribution of the study is the development of a practical documentretrieval system built upon the outputs of the word recognition model. For thisapplication-oriented stage, a lightweight EfficientNetB0-based recognition modelwas adopted due to its low parameter count. The same preprocessing andsegmentation pipeline was applied to each page of the collected handwrittendocuments. The EfficientNet recognizer was used to identify words belonging to the321-word lexicon, enabling partial textual representation of each document. Therecognized subset of words provides sufficient information for effective retrieval. ATF–IDF (Term Frequency-Inverse Document Frequency) representation wasconstructed from the recognized words, and Latent Dirichlet Allocation (LDA) wasused to model semantic relationships between documents. Each document is rankedaccording to its similarity to the query, and the top-5 documents are returned as theretrieval output.Overall, this thesis presents a complete framework, from dataset construction andrecognition model development to real-world document retrieval. The developedmethodologies provide a foundation for further research on low-resource scripts andopen the way for scalable deployment in government, legal, and administrativecontexts where handwritten records continue to play a significant role.