Pattern primitive based malayalam handwritten character recognition studies for real-time applications

Abstract

The communication between people in society is partially replaced with machines as one part of the process for a few decades. As technology dwelt faster changes, more and more devices came out for simplifying the task of human-machine interaction, majorly computers and mobile devices. Nowadays, ways for human-machine coalescence are extensive in speech and handwriting termed under Natural Language Processing. For handwriting recognition, off-line and online are the two methods to simplify the task of data input in various styles followed in different scripts. Online Handwritten Character Recognition (OHCR) involves the automatic conversion of handwriting consisting of temporal information that are converted into letter codes usable in computers. The thesis analyses existing techniques in Online Handwritten Character Recognition (OHCR) and investigates the potential use of a pattern primitive based scheme, for real-time applications in Malayalam. The study starts with an analysis of Malayalam characters and their linguistic formulations. The representation of the sounds of a language by written symbols is also a part of the study. All the vowels, consonants, and other units in the Malayalam language are detailed with the various rules, concerning the formation of conjuncts within the orthographical perspective. The study contributes a database of online handwritten characters constructed using two devices. The first device records handwritten strokes in a traditional way, where paper and digital pen with a sensor is used for data acquisition. In the second device, the touch screen capability of current technology is utilized in acquiring data with a touch screen laptop. As a part of the study, a native database named Calicut University Online Handwriting Data Base (CU-OHDB) is created containing 30800 samples of online Malayalam handwritten characters written by twenty writers, where 13200 samples are obtained using a pen and paper method, and the other 17600 samples recorded using a touchscreen-enabled laptop. The preprocessing of raw data is necessary due to the variations in writing style and writing imperfection caused by the writer and also the device. The methods involved in the preprocessing phases are detailed in the thesis with the corresponding algorithms. For normalization, min-max normalization is used. Smoothing is done using moving average filter and resampling is carried out using equidistant resampling with spline interpolation for missing points. The thesis addresses handwriting recognition as a pattern recognition problem. Since online handwritings are temporal sequences, they are also considered as patterns. Studies tuned in such directions reveal so many aspects of Malayalam characters. An extensive study is conducted to identify the structural and directional properties of every character. Segmentation (Lexical analysis) of Malayalam characters is conducted to identify unique patterns in them. From the analysis, it is found that, certain pattern segments, namely pattern primitives, are frequent in Malayalam characters. It is also found that, there are 26 pattern primitives to represent all the 44 single stroke vowel and consonant characters in Malayalam. These points are manually marked on every character to form a character reference set. As a part of the work, a label to every primitive is assigned, and every character is arranged as a sequence of pattern primitive labels. The segmentation of character samples is performed in the work using three algorithms, namely, Ramer Douglas Peucker (RDP) algorithm, Eight Direction Freeman Code (EDFC) and, a combined approach of these two algorithms. The Ramer Douglas Peucker (RDP) algorithm, which is used for polygonal approximations of curves in numerous pattern recognition problems to reduce a character pattern into a finite number of points, is used for segmentation of the online handwritten character samples. Even with the variability in the writing style of the samples, the technique performed well with an average segmentation rate of 75.07%. The role of direction in the recognition process of a character is dealt within the segmentation part of the thesis using the well-known Eight Direction Freeman Code (EDFC). The segmentation scheme using Eight Direction Freeman Code (EDFC) proposed in the study also reported an accuracy of 74.68%. A combined approach of RDP and EDFC with certain filtering over directions and redundant points achieved a segmentation rate of 91.27% through visual comparisons. An automated scheme for measuring segme/ntation accuracy is also implemented in the study and obtained 79.13% accuracy for the combined approach. The comparison of the three segmentation schemes suggested that the combined approach is the best method, and the segmentation of pattern primitives is performed using the combined approach. The pattern primitives of the characters obtained from segmentation using the combined approach are extensively studied in the remaining sections of the thesis. In the next phase, an in-depth study of various features of every pattern primitive is deployed in visual and structural formulations. The research focuses on features, mainly on the direction aspects and morphology of the characters. Preprocessing of pattern primitives is also described in the thesis for linear types. Pattern primitives with curvature values near zero are redrawn as a smooth line using Bresenham’s line drawing algorithm, and angle corrections are made by measuring slope values. The study shows that the maximum number of pattern primitives in the selected single stroke character set of Malayalam is eight and the minimum number of pattern primitives is two. Recognition experiments based on pattern primitives are conducted in a class-modular approach, where clusters are formed based on the number of pattern primitives present in the characters. For every cluster, the discriminating features like cusps, linearity, reduced direction code, pattern primitive length, the direction of a pattern primitive, and intersections in various pattern primitives are identified. An intra-class recognition using the extracted features displayed higher accuracies in the classification experiments using Support Vector Machine (SVM) classifier for the 44 single stroke online handwritten samples of Malayalam characters. The accuracy thus obtained ranges from 81.67% to 100% in various clusters that emphasizes the suitability of pattern primitive approach for the recognition of online handwritten Malayalam characters. The predictive model for real-time recognition of Malayalam handwritten characters, based on pattern primitives, is presented in the thesis with prime focus. Two approaches are proposed in the model. In the first approach, Deterministic Finite Automata (DFA) based OHCR, using pattern primitive string representation of characters, is described.In this approach, the character will be recognized only after finishing the writing of the entire character. Even though the experiments showed an average accuracy of 65.75%, the method uses a holistic approach, which causes considerable delay in recognition. Hence an alternate approach is also proposed. In this approach, a real-time prediction model for OHCR is used. The transition sequences of pattern primitives that constitute each character are analyzed in the predictive model. These pattern primitive transition information, obtained from the characters,are utilized for constructing a tree structure for the possible predictions of every character in a real-time environment. The average Reduction in Writing Time (RWT) is the significant advantage of the predictive model to be incorporated in Malayalam OHCR. The study shows that most of the Malayalam characters start writing in the upward direction. It can also be seen that fifty percentage of the characters start with a unique pattern primitive. The most frequent pattern primitive transition is also identified in the study . Experiments are also conducted to verify the effectiveness of the model using online handwritten character samples taken from CU-OHDB dataset and obtained an average prediction accuracy of 77.63%. The prediction part of the system is quick and straightforward to redefine existing systems that recognize the handwritten characters, only after completing the writing process. The model is useful in building a fast handwritten character recognition system to be realized in real-time environments. A Keyword Spotting (KWS) technique is implemented as an application of handwriting recognition in the final section of the thesis. The well-known pattern matching technique, Dynamic Time Warping (DTW) is used to compare the words in the experiments. An average performance score of 78.31% over a regular DTW search obtained in xithe experiment suggests that, the method can effectively be used for Keyword Spotting (KWS) for Malayalam online handwritten documents in a real-time environment.