Search

Abstract With over 55 million people globally affected by dementia and nearly 10 million new cases reported annually, Alzheimer’s disease is a prevalent and challenging neurodegenerative disorder. Despite significant advancements in machine learning techniques for Alzheimer’s disease detection, the widespread adoption of deep learning models raises concerns about their explainability. The lack of explainability in deep learning models for online handwriting analysis is a critical gap in the literature in the context of Alzheimer’s disease detection. This paper addresses this challenge by interpreting predictions from a Convolutional Neural Network applied to multivariate time series data, generated by online handwriting data associated with continuous loop series handwritten on a graphical tablet. Our explainability methods reveal distinct motor behavior characteristics for healthy individuals and those diagnosed with Alzheimer’s. Healthy subjects exhibited consistent, smooth movements, while Alzheimer’s patients demonstrated erratic patterns marked by abrupt stops and direction changes. This emphasizes the critical role of explainability in translating complex models into clinically relevant insights. Our research contributes to the enhancement of early diagnosis, providing significant and reliable insights to stakeholders involved in patient care and intervention strategies. Our work bridges the gap between machine learning predictions and clinical insights, fostering a more effective and understandable application of advanced models for Alzheimer’s disease assessment.

Online handwritten word recognition (OHR) in low-resource languages such as Bangla is still an open problem. Although the need and importance of OHR are increasing nowadays, research works on word-level recognition are few (specifically for Bangla script), and there is a lot of room for improving recognition performance. In the current work, we employed different Recurrent Neural Network (RNN) architectures such as Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (BLSTM), Gated Recurrent Unit (GRU), and Bidirectional Gated Recurrent Unit (BGRU) for the recognition of online handwritten Bangla words written in an unconstrained domain. One of the challenges includes the variable number of strokes used to write words. This study aims to develop a segmentation-free recognition module where the features from constituent strokes of the word sample are fed to the developed RNN architectures. Sequential and dynamic information obtained from the strokes is considered as the features for the current experiment. The customized architecture of BLSTM known as BWordDeepNet (Bangla Word Deep-learning Network) provides the best performance with 98.35% correct recognition accuracy on the dataset having 7992 online handwritten Bangla word samples. Additionally, the model achieves a numerical gain of 8.08% compared to the Bangla word recognition work mentioned in [38] that was performed on the same word dataset containing 5550 word samples. We have also compared the performance of our proposed model with state-of-the-art techniques used for the same purpose. [ABSTRACT FROM AUTHOR]

The increasing age of our society is connected to a rising number of people suffering from disorders. One such disorder is Parkinson’s disease (PD). Predictions indicate that the number of individuals affected by PD will more, than double in the future. Neurologists and data scientists consider handwriting as one of the motor symptoms of PD and recognize it as a valuable resource for detecting this disorder. Within this framework, we introduce an innovative system for Parkinson’s disease detection, which encompasses several key stages. The process commences with data augmentation and preprocessing, subsequently leading to the segmentation of online handwriting into Beta strokes. Following that, feature extraction is carried out utilizing the Beta-elliptical approach and the fuzzy perceptual detector. Finally, we employ bidirectional long short-term memory (BLSTM) for the classification task. To assess the performance of our system, we created a new online Arabic handwriting dataset designed for detecting Parkinson’s disease. The results we obtained affirm the efficacy of our proposed system. Through comprehensive evaluations conducted on the PaHaW dataset, we achieved good accuracy, thereby highlighting that our system surpasses the performance of existing systems.

Building upon the recent advances and successes in the application of deep learning to the medical field, we propose in this work a new approach to detect and classify early-stage Alzheimer patients using online handwriting (HW) loop patterns. To cope with the lack of training data prevalent in the tasks of classification of neuro-degenerative diseases from behavioral data, we investigate several data augmentation techniques. In this respect, compared to the traditional data augmentation techniques proposed for HW-based Parkinson detection, we investigate a variant of Generative Adversarial Networks (GANs), DoppelGANger, especially tailored for times series and hence suitable for synthesizing realistic online handwriting sequences. Based on a 1D-Convolutional Neural Network (1D-CNN) to perform Alzheimer classification, we show, on a real dataset related to HW and Alzheimer, that our DoppelGANger-based augmentation model allow the CNN to significantly outperform both the current state of the art and the other data augmentation techniques.

Copyright of Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji is the property of Gazi University and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Handwriting is a complex perceptual–motor skill that is mastered around the age of 8. Although its computerized analysis has been utilized in many biometric and digital health applications, the possible effect of gender is frequently neglected. The aim of this paper is to analyze different online handwritten tasks performed by intact subjects and explore gender differences in commonly used temporal, kinematic, and dynamic features. The differences were explored in the BIOSECUR-ID database. We have identified a significant gender difference in on-surface/in-air time of genuine and skilled forgery signatures, on-surface time in cursive letters and numbers, and pressure, speed, and acceleration in text written in capital letters. Our findings accent the need to consider gender as an important confounding factor in studies dealing with online handwriting signal processing. [ABSTRACT FROM AUTHOR]

Calligraphy is an essential part of the Arabic heritage and culture. It has been used in the past for the decoration of houses and mosques. Usually, such calligraphy is designed manually by experts with aesthetic insights. In the past few years, there has been a considerable effort to digitize such type of art by either taking a photograph of decorated buildings or drawing them using digital devices. The latter is considered an online form where the drawing is tracked by recording the apparatus movement, an electronic pen, for instance, on a screen. In the literature, there are many offline datasets with diverse Arabic styles for calligraphy. However, there is no available online dataset for Arabic calligraphy. In this paper, we illustrate our approach for collecting and annotating an online dataset for Arabic calligraphy called Calliar, which consists of 2,500 sentences. Calliar is annotated for stroke, character, word, and sentence-level prediction. We also propose various baseline models for the character classification task. The results we achieved highlight that it is still an open problem. [ABSTRACT FROM AUTHOR]

In this paper, we present a pressure characterization and normalization procedure for online handwritten acquisition. Normalization process has been tested in biometric recognition experiments (identification and verification) using online signature database MCYT, which consists of the signatures from 330 users. The goal is to analyze the real mismatch scenarios where users are enrolled with one stylus and then, later on, they produce some testing samples using a different stylus model with different pressure response. Experimental results show: 1) a saturation behavior in pressure signal 2) different dynamic ranges in the different stylus studied 3) improved biometric recognition accuracy by means of pressure signal normalization as well as a performance degradation in mismatched conditions 4) interoperability between different stylus can be obtained by means of pressure normalization. Normalization produces an improvement in signature identification rates higher than 7% (absolute value) when compared with mismatched scenarios.

Parkinson’s disease (PD) is the second most frequent neurodegenerative disease associated with several motor symptoms, including alterations in handwriting, also known as PD dysgraphia. Several computerized decision support systems for PD dysgraphia have been proposed, however, the associated challenges require new approaches for more accurate diagnosis. Therefore, this work adds spectral and cepstral handwriting features to the already-used temporal, kinematic and statistics handwriting features. First, we calculate temporal and kinematic features using displacement; statistic features $\left ({SF }\right)$ using displacement, and horizontal and vertical displacement; spectral $\left ({SDF }\right)$ and cepstral $\left ({CDF }\right)$ using displacement, horizontal and vertical displacement and pressure. Since the employed dataset (PaHaW) contains only 37 PD patients and 38 healthy control subjects (HC), then as the second step, we augment the percentage of the smaller training set to equal the larger. Next, we augment both classes to increase the training patient’s data and added random Gaussian noise in all augmentations. Third, the most relevant features were selected using the modified fast correlation-based filtering method (mFCBF). Finally, autoML is employed to train and test more than ten plain and ensembled classifiers. Experimental results show that adding spectral and cepstral features to temporal, kinematics and statistics features highly improved classification accuracy to 98.57%. Our proposed model, with lower computational complexities, outperforms conventional state-of-the-art models for all tasks, which is 97.62%.

Advantageous property of behavioural signals (e.g. handwriting), in contrast to morphological ones (e.g. iris, fingerprint, hand geometry), is the possibility to ask a user to perform many different tasks. This article summarises recent findings and applications of different handwriting/drawing tasks in the field of security and health. More specifically, it is focused on on-line handwriting and hand-based interaction, i.e. signals that utilise a digitizing device (specific devoted or general-purpose tablet/smartphone) during the realization of the tasks. Such devices permit the acquisition of on-surface dynamics as well as in-air movements in time, thus providing complex and richer information when compared to the conventional "pen and paper" method. Although the scientific literature reports a wide range of tasks and applications, in this paper, we summarize only those providing competitive results (e.g. in terms of discrimination power) and having a significant impact in the field. [ABSTRACT FROM AUTHOR]

Online handwriting recognition (OHR) has gained major research interest not just due to the enormous technological advancement in recent years, but also the easy availability of the various electronic devices. This digital revolution is opening up a new dimension in every passing day to the regional and low resource languages with these languages get noticed by the researchers. In this paper, we have targeted a low resource language, Assamese, which is mainly spoken in the eastern region of India. We have proposed a novel and efficient feature vector for recognition of online handwritten Assamese numeral images. Our feature vector has been conceptualized based on the properties of light rays emerging out from a point source. Here we consider that there are multiple hypothetical light emerging sources in a sample numeral image. The amount of light fenced by the image is quantified and considered as a feature. The idea of using point light source to estimate the shape of online handwritten numerals is completely new and efficient. Impressive recognition accuracy is obtained on application of the feature vector on a standard online handwritten Assamese numeral database and it outnumbers some popular and standard feature descriptors, available in the literature. The source code of this work can be found in the following github link: https://github.com/ghoshsoulib/CTRL-Assamese-Digit-Recognition. [ABSTRACT FROM AUTHOR]

Graphomotor disabilities (GD) are present in up to 30% of school-aged children and are associated with several symptoms in the field of kinematics. Although the basic kinematic features such as velocity, acceleration, and jerk were proved to effectively quantify these symptoms, a recent body of research identified that the theory of fractional calculus can be used to even improve the objective GD assessment. The goal of this study is to extend the current knowledge in this field and explore the abilities of several fractional order derivatives (FD) approximations to estimate the severity of GD in the children population. We enrolled 85 children attending the 3rd and 4th grade of primary school, who performed a combined loop task on a digitizing tablet. Their performance was rated by psychologists and the online handwriting signals were parametrised by kinematic features utilising three FD approximations: Grünwald-Letnikov's, Riemann-Liouville's, and Caputo's. In this study, we showed the differences across the employed FD approaches for the same kinematic handwriting features and their potential in GD analysis. The results suggest that the Riemann-Liouville's approximation in the field of quantitative GD analysis outperforms the other ones. Using this approach, we were able to estimate the overall score with a low error of 0.65 points, while the scale range is 4. In fact, the psychologists tend to make the error even higher.

School-aged children spend 31–60% of their time at school performing handwriting, which is a complex perceptual-motor skill composed of a coordinated combination of fine graphomotor movements. As up to 30% of them experience graphomotor difficulties (GD), timely diagnosis of these difficulties and therapeutic intervention are of great importance. At present, an objective, computerized decision support system for the identification and assessment of GD in school-aged children is still missing. In this study, we propose three novel advanced handwriting parametrization techniques based on modulation spectra, fractional order derivatives, and tunable Q-factor wavelet transform to improve the identification of GD using online handwriting. For this purpose, we analyzed signals acquired from 7 basic graphomotor tasks performed by 53 children attending 3rd and 4th grade at several primary schools around the Czech Republic. Combining the newly proposed features with the conventionally used ones, we were able to identify GD with 84% accuracy. In this study, we showed that using advanced parametrization of basic graphomotor movements can be potentially used to improve our capabilities of quantifying problems with the development of legible, fast-paced handwriting, and help with the early diagnosis of handwriting difficulties frequently manifested in developmental dysgraphia.

Online handwritten analysis presents many applications in e-security, signature biometrics being the most popular but not the only one. Handwriting analysis also has an important set of applications in e-health. Both kinds of applications (e-security and e-health) have some unsolved questions and relations among them that should be addressed in the next years. We summarize the state of the art and applications based on handwriting signals. Later on, we focus on the main achievements and challenges that should be addressed by the scientific community, providing a guide for future research. Among all the points discussed in this article, we remark the importance of considering security, health, and metadata from a joint perspective. This is especially critical due to the risks inherent when using these behavioral signals. [ABSTRACT FROM AUTHOR]

This paper proposes a model which is used to recognize online handwritten Bangla words. This word recognition module comprises of different modules: preprocessing of the word samples, segmentation of words into basic strokes, recognizing the basic strokes using multilayer perceptron, followed by recognition of words using Hidden Markov Model (HMM) aided by Language Model (LM). For stroke recognition, two different feature extraction techniques (point-based and curvature-based procedures) are used using late fusion technique. Top 5 stroke recognition choices are used to construct HMM for the prediction of word sample. An N-gram LM is applied as a post-processing step to rectify the HMM outcomes if required. A total of 50 different word samples with 110 instances each are used to evaluate the proposed model. The overall stroke-level and word-level recognition accuracies obtained by this model are 95.4% and 90.3%, respectively. The proposed model can be extended to recognize online handwritten words written in other script like Devanagari, Assamese, and Gurumukhi, etc. The methodologies described in the manuscript can also be applied for offline word recognition purpose. [ABSTRACT FROM AUTHOR]

Long short-term memory networks (LSTM) achieve great success in temporal dependency modeling for chain-structured data, such as texts and speeches. An extension toward more complex data structures as encountered in 2D graphic languages is proposed in this work. Specifically, we address the problem of handwritten mathematical expression recognition, using a tree-based BLSTM architecture allowing the direct labeling of nodes (symbol) and edges (relationship) from a graph modeling the input strokes. One major difference with the traditional approaches is that there is no explicit segmentation, recognition and layout extraction steps but a unique trainable system that produces directly a stroke label graph describing a mathematical expression. The proposed system, considering no grammar, achieves competitive results in online math expression recognition domain. [ABSTRACT FROM AUTHOR]

Highlights • This article proposes a novel approach for online handwritten cursive and non-cursive word recognition in two of the most popular Indian scripts— Devanagari and Bengali , based on two recently developed versions of Recurrent Neural Network (RNN), named as Long–Short Term Memory (LSTM) and Bidirectional Long–Short Term Memory (BLSTM). • The proposed approach divides each word horizontally into three zones— upper, middle , and lower , before carrying out training of basic strokes using LSTM and BLSTM versions of RNN. This type of zone division is done to reduce the variations in temporal orders of basic strokes within a word. • The major strength of the proposed system is unlike most of the existing wordrecognition systems in these two scripts, it can recognize those words also which are not present in the trainingdataset as it considers basic stroke based class labelling scheme to train the classifier. The proposed system also overcomes various drawbacks of HMM that are common in existing HMM based word recognition systems. • The experiments have been carried out in HMM based platform also to show the comparative performance analysis of the present system in both HMM and RNN based platforms. • Experimental results show that the proposed zone segmentation technique and adopting LSTM–BLSTM based learning outperform existing word recognition systems including HMM based ones in these two Indian scripts. Abstract Devanagari and Bengali scripts are two of the most popular scripts in India. Most of the existing word recognition studies in these two scripts have relied upon the widely used Hidden Markov Model (HMM), in spite of its familiar shortcomings. The existing works were evaluated against and performed well in their chosen metrics. But, the existing word recognition systems in these two scripts could not achieve more than 90% recognition accuracy. This article proposes a novel approach for online handwritten cursive and non-cursive word recognition in Devanagari and Bengali scripts based on two recently developed models of Recurrent Neural Network (RNN)—Long–Short Term Memory (LSTM) and Bidirectional Long–Short Term Memory (BLSTM). The proposed approach divides each word horizontally into three zones—upper, middle, and lower, to reduce the variations in basic stroke order within a word. Next, the word portions from middle zone are re-segmented into its basic strokes. Various structural and directional features are then extracted from each basic stroke of the word separately for each zone. These zone wise basic stroke features are then studied using both LSTM and BLSTM versions of RNN. Most of the existing word recognition systems in these two scripts have followed word based class labelling approach, whereas proposed system has followed the basic stroke based class labelling approach. An exhaustive experiment on large datasets has been performed to evaluate the performance of the proposed approach using both RNN and HMM to make a comparative performance analysis. Experimental results show that the proposed RNN based system is superior over HMM achieving 99.50% and 95.24% accuracies in Devanagari and Bengali scripts respectively and outperforms existing HMM based systems in the literature as well. [ABSTRACT FROM AUTHOR]

Recently, several researches were carried on handwritten document analysis field thanks to the evolution of data capture technologies. For a given document, multiple components could be treated as text, signatures and graphics. In this study, we present a new framework for a Multilanguage online handwritten text analysis where both script identification and recognition are made. The proposed system proceeds by segmenting the script into continuous trajectories delimited between two successive pen-down and pen-up moments. These segments are clustered and trained using Time Delay Neural Network (TDNN) according to their beta-elliptic parameters. In script identification process, the segments belonging to the same script are gathered and brought to a Recurrent Neural Network with Long Short Term Memory (RNN-LSTM) in order to identify its language. For script recognition stage, the samples from the already selected language database are trained and tested using the fuzzy output description obtained by the TDNN coupled to a Support Vector Machines (SVM). The Experiments were made on a large multi-language database containing 45686 online handwriting words from Latin, Arabic and digit scripts and shows very promising results that exceed the recognition rate of 99%. [ABSTRACT FROM AUTHOR]

Delayed strokes, such as i-dots and t-crosses, cause a challenge in online handwriting recognition by introducing an extra source of variation in the sequence order of the handwritten input. The problem is especially relevant for languages where delayed strokes are abundant and training data are limited. Studies for handling delayed strokes have mainly focused on Arabic and Farsi scripts where the problem is most severe, with less attention devoted for scripts based on the Latin alphabet. This study aims to investigate the effectiveness of the delayed stroke handling methods proposed in the literature. Evaluated methods include the removal of delayed strokes and embedding delayed strokes in the correct writing order, together with their variations. Starting with new definitions of a delayed stroke, we tested each method using both hidden Markov model classifiers separately for English and Turkish and bidirectional long short-term memory networks for English. For both the UNIPEN and Turkish datasets, the best results are obtained with hidden Markov model recognizers by removing all delayed strokes, with up to 2.13% and 2.03% points accuracy increases over the respective baselines. In case of the bidirectional long short-term memory networks, stroke order correction of the delayed strokes by embedding performs the best, with 1.81% (raw) and 1.72% (post-processed) points improvements above the baseline. [ABSTRACT FROM AUTHOR]

Highlights • We propose a paradigm unveiling handwriting changes due to aging and Alzheimer's. • Our new semi-supervised learning and sequential representation learning are key. • Semi-supervised learning brings to light handwriting multimodal behavioral trends. • Our sequential representation learning uncovers temporal feature representations. • Classification based on temporal representations outperforms the state of the art. Abstract We present, in this paper, a novel paradigm for assessing Alzheimer 's disease and aging by analyzing impairment of handwriting (HW) on tablets, a challenging problem that is still in its infancy. The state of the art is dominated by methods that assume a unique behavioral trend for each cognitive profile or age group, and that extract global kinematic parameters, assessed by standard statistical tests or classification models, for discriminating the neuropathological disorders (Alzheimer 's (AD), Mild Cognitive Impairment (MCI)) from Healthy Controls (HC), or HC age groups from each other. Our work tackles these two major limitations as follows. First, instead of considering a unique behavioral pattern for each cognitive profile or age group, we relax this heavy constraint by allowing the emergence of multimodal behavioral patterns. We achieve this by performing semi or unsupervised learning to uncover homogeneous clusters of subjects, and then we analyze how much information these clusters carry on the cognitive profiles (or age groups). Second, instead of relying on global kinematic parameters, mostly consisting of their average, we refine the encoding either by a semi-global parameterization, or by modeling the full dynamics of each parameter, harnessing thereby the rich temporal information inherently characterizing online HW. To illustrate the power of our paradigm, we present three studies, one regarding age, and two regarding Alzheimer 's. Thanks to our modeling, we obtain new findings that are the first of their kind on this research field. On aging, unlike previous works reporting only one pattern of HW change with age, our study, based on a semiglobal parametrization scheme, uncovers three major aging HW styles, one specific to aged subjects and two shared with other age groups. On Alzheimer 's, a striking finding is revealed: two major clusters are unveiled, one dominated by HC and MCI subjects, and one by MCI and ES-AD , thus revealing that MCI patients have fine motor skills leaning towards either HC 's or ES-AD 's. Our paper introduces also a new temporal representation learning from HW trajectories that uncovers a rich set of features simultaneously like the full velocity profile, size and slant, fluidity, and shakiness, and reveals, in a naturally explainable way, how these HW features conjointly characterize, with fine and subtle details, the cognitive profiles. [ABSTRACT FROM AUTHOR]