Search

The research community considers handwritten word recognition (HWR) as an open research problem to date. The reasons behind this are variations in intra-/interpersonal writing style, overlapping and/or touching characters in a word, degraded scanned document images, etc. Two major approaches, namely holistic and analytical, are followed by the researchers while designing an HWR system. In this work, we have followed the holistic approach as it works well on limited and pre-defined lexicon as compared to the analytical approach. As observed in the literature related to handwritten word recognition, irrespective of the approaches, researchers generally extract various local features from hypothetically partitioned segments of a word image while dealing with the said problem. However, no such work has been found which has considered inter-segment similarity that might carry some distinct information about different patterns (here, word segments). To this end, in the present work, we have used Hausdorff and Fréchet distances to quantize the similarity among all possible word segments taking two at a time. Along with this, conventional chain code histogram (a shape-based feature descriptor) and modified negative refraction-based shape transformation features have been used. Finally, a majority voting schema is used to combine outputs from six different classifiers. The model has been evaluated on two standard databases, namely IAM and IFN/ENIT, and the results obtained are promising in comparison with state-of-the-art holistic word recognition methods. Moreover, a performance comparison of the present method with some deep learning models confirms the usefulness of the proposed method. [ABSTRACT FROM AUTHOR]

Recognition of unconstrained handwritten word images is an interesting research problem which gets more challenging when lexicon-free words are considered. Prerequisite for developing a lexicon-free handwritten word recognition technique is the segmentation of a word image into its constituent character set. Therefore, a competent character segmentation technique is required to design a comprehensive word recognition module. However, the literature study reveals that there is no standard word image database with ground truth information. As a result, most character segmentation algorithms found in the literature rely on self-made databases with manual evaluation. To fill the research need, in the present scope of the work, a comprehensive database consisting of handwritten Bangla word images is prepared primarily for evaluating any character segmentation algorithms. Additionally, the present work also provides two types of ground truth images related to segmented character shapes of the word images. Besides, an evaluation tool is developed for assessing the performance of any character segmentation algorithm on the developed benchmark database. The benchmark result, as found here, is 0.9212 (F-score) which outperforms some state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Handwritten word recognition, a classical pattern recognition problem, converts a word image into its machine editable form. Mainly two basic approaches are followed to solve this problem, one is segmentation-based and the other is holistic. A number of research attempts have shown that the holistic approach performs better than its counterpart when the lexicon is predefined, fixed and small in size. Relying on this, initial benchmark recognition accuracy on CMATERdb2.1.2, a publicly available database consists of handwritten city names in Bangla, was reported following a holistic word recognition protocol. In the present work, we have followed the same trend to recognize the word samples of the said database and set a new benchmark recognition accuracy. A sparse convolutional neural network (CNN)-based model which is a low-cost trainable model has been developed for this. We have relied on a recently proposed hypothesis, known as lottery ticket hypothesis for pruning the layers of CNN model methodically, and derived a low-resource model having much less number of training parameters. This model competently surpasses the previously reported recognition accuracy on the said database by a significant margin with an axed training cost. [ABSTRACT FROM AUTHOR]

Feature selection plays a key role in reducing the dimensionality of a feature vector by discarding redundant and irrelevant ones. In this paper, a Genetic Algorithm-based hierarchical feature selection (HFS) model has been designed to optimize the local and global features extracted from each of the handwritten word images under consideration. In this context, two recently developed feature descriptors based on shape and texture of the word images have been taken into account. Experimentation is conducted on an in-house dataset of 12,000 handwritten word samples written in Bangla script. This database comprises names of 80 popular cities of West Bengal, a state of India. Proposed model not only reduces the feature dimension by nearly 28%, but also enhances the performance of the handwritten word recognition (HWR) technique by 1.28% over the recognition performance obtained with unreduced feature set. Moreover, the proposed HFS-based HWR system performs better in comparison with some recently developed methods on the present dataset. [ABSTRACT FROM AUTHOR]

Due to the cursive nature, segmentation of handwritten Bangla words into characters and also recognition of the same sometimes become a very challenging problem to the researchers. Presence of comparatively large character set along with modifiers, ascendants, descendants, and compound characters makes the segmentation task more complex. As holistic method avoids such character-level segmentation, it is generally useful for the recognition of words written in any such complex scripts. In the present work, a holistic handwritten word recognition method is developed using a feature descriptor, designed by combining different Elliptical, Tetragonal and Vertical pixel density histogram-based features. Recognition process is carried out separately using two classifiers, namely multi-layer perceptron (MLP) and support vector machine (SVM). For evaluation of the proposed method, a database of 18,000 handwritten Bangla word images, having 120 word classes, is prepared. The proposed system performs comparatively better with SVM than MLP for the prepared dataset. It has achieved 83.64% accuracy at best case and 79.38% accuracy on an average using fivefold cross-validation. The current method has also outperformed some recently reported holistic word recognition technique tested on the developed dataset. In addition to that the database, prepared in this work, is made freely available to fill the absence of a publicly available standard database for holistic Bangla word recognition. [ABSTRACT FROM AUTHOR]

Contrast enhancement is an important pre-processing task for several image and video processing applications. The objective of a contrast enhancement method is to improve the quality of the visual information contained in the images for further processing. Due to the enormous challenges, it is still considered as an open research problem. Several approaches have been proposed in the literature based on spatial and fr uency domain techniques. Among them, the Incomplete Beta Function (IBF) is a popular scheme used for image contrast enhancement. In the IBF based image contrast enhancement technique, quality of an image is improved by two controlling parameters. Under such conditions, these parameters need to be tuned for obtaining better outcomes. In this paper, a new gray-scale contrast enhancement algorithm is introduced where, instead of tuning the controlling parameters of IBF experimentally, their near-optimal values are calculated with the help of a recently published meta-heuristic algorithm called Artificial Electric Field Algorithm (AEFA). The proposed method has been compared with many state-of-the-art techniques in terms of some standard metrics considering three different datasets, namely Kodak, MIT-Adobe FiveK and USC-SIPI. Simulation results demonstrate that the proposed AEFA based image enhancement technique increases the overall image contrast and enriches the information present in the image. [ABSTRACT FROM AUTHOR]

Digital face manipulation has become a concern in the last few years due to its harmful impacts on society. It is especially concerning for high-profile celebrities because their identities can be easily manipulated using mobile or web applications such as FaceSwap and FaceApp. These manipulated faces are so close to real ones that it becomes really hard to detect them, even with bare eyes. Though deep learning-based models are predominantly used by researchers, they hardly check for the presence of irrelevant or redundant features produced by those models. To this end, we have proposed a hierarchical feature selection (HFS)-based method to detect deepfake images or videos. First, we have extracted both handcrafted and deep learning features from the inputs. Next, the HFS is applied to select a near optimal set of features. In each stage of it, a hybrid feature selection method is employed that integrates a population-based meta-heuristic model, called Grey Wolf Optimization, and a single solution-based meta-heuristic model, called the Vortex Search algorithm. We have evaluated our model on three publicly available datasets, namely Celeb-DF (V2), FaceForensics++, and Deepfake Detection Challenge (DFDC). The model provides 99.35%, 99.16%, and 85.67% AUC scores on the Celeb-DF (V2), FaceForensics++, and DFDC datasets, respectively, while using only 11.50%, 12.65%, and 10.22% of actual features. Besides, our model outperforms most of the state-of-the-art methods found in our literature review and evaluated on these three datasets. [ABSTRACT FROM AUTHOR]

The prime goal of creating synthetic digital data is to generate something very closer to real ones when the original data are scarce. However, the trustworthiness of such digital content is dipping potentially in society owing to malicious users. Deepfake method that uses computer graphics and computer vision techniques to replace the face of one person with the face of a different person is becoming an area of big concern. Such techniques can easily be used to hide the identity of a person. Therefore, a method is needed to verify the originality of such face images/videos. To this end, we design a deep learning model enhanced with visual attention technique to differentiate manipulated videos/images (generated by deepfake methods) from real ones. At first, we extract the face region from video frames and then pass the same through the pre-trained Xception model to obtain the feature maps. Next, with the help of the visual attention mechanism, we mainly try to focus on the deepfake video manipulation leftover artifacts. We evaluate our model on two publicly available datasets, namely FaceForensics++ and Celeb-DF (V2), and our model outperforms many state-of-the-art methods tested on these two datasets. Source code of the proposed method can be found at: https://github.com/tre3x/Deepfake-Video-Forgery-Detection. [ABSTRACT FROM AUTHOR]

Handwritten keyword spotting (KWS) is of great interest to the document image research community. In this work, we propose a learning-free keyword spotting method following query by example (QBE) setting for handwritten documents. It consists of four key processes: pre-processing, vertical zone division, feature extraction, and feature matching. The pre-processing step deals with the noise found in the word images, and the skewness of the handwritings caused by the varied writing styles of the individuals. Next, the vertical zone division splits the word image into several zones. The number of vertical zones is guided by the number of letters in the query word image. To obtain this information (i.e., number of letters in a query word image) during experimentation, we use the text encoding of the query word image. The user provides the information to the system. The feature extraction process involves the use of the Hough transform. The last step is feature matching, which first compares the features extracted from the word images and then generates a similarity score. The performance of this algorithm has been tested on three publicly available datasets: IAM, QUWI, and ICDAR KWS 2015. It is noticed that the proposed method outperforms state-of-the-art learning-free KWS methods considered here for comparison while evaluated on the present datasets. We also evaluate the performance of the present KWS model using state-of-the-art deep features and it is found that the features used in the present work perform better than the deep features extracted using InceptionV3, VGG19, and DenseNet121 models. [ABSTRACT FROM AUTHOR]

The novel coronavirus (COVID-19), has undoubtedly imprinted our lives with its deadly impact. Early testing with isolation of the individual is the best possible way to curb the spread of this deadly virus. Computer aided diagnosis (CAD) provides an alternative and cheap option for screening of the said virus. In this paper, we propose a convolution neural network (CNN)-based CAD method for COVID-19 and pneumonia detection from chest X-ray images. We consider three input types for three identical base classifiers. To capture maximum possible complementary features, we consider the original RGB image, Red channel image and the original image stacked with Robert's edge information. After that we develop an ensemble strategy based on the technique for order preference by similarity to an ideal solution (TOPSIS) to aggregate the outcomes of base classifiers. The overall framework, called TOPCONet, is very light in comparison with standard CNN models in terms of the number of trainable parameters required. TOPCONet achieves state-of-the-art results when evaluated on the three publicly available datasets: (1) IEEE COVID-19 dataset + Kaggle Pneumonia Dataset, (2) Kaggle Radiography dataset and (3) COVIDx. [ABSTRACT FROM AUTHOR]

In ancient times, there was no system to record or document music. A basic notation system to write European music was formulated around 14th century in the Baroque period which slowly evolved into the standard notation system that we have today. Later, the musical pieces from the classical and post-classical period of European music were documented as scores using this standard European staff notations. These notations are used by most of the modern genres of music due to their versatility. Hence, it is very important to develop a method that can store such music sheets containing handwritten music scores digitally. Optical music recognition (OMR) is a system that automatically interprets the scanned handwritten music scores. In this work, we have proposed a classifier ensemble of deep transfer learning models with support vector machine (SVM) as the aggregator for handwritten music symbol recognition. We have applied three pre-trained deep learning models, namely ResNet50, GoogleNet and DenseNet161 (each trained on ImageNet), and fine-tuned on our target datasets i.e., music symbol image datasets. The proposed ensemble technique can capture a more complex association of the base classifiers, thus improving the overall performance. We have evaluated the proposed model on five publicly available standard datasets, namely Handwritten Online Music Symbols (HOMUS), Capitan_Score_Uniform, Capitan_Score_Non-uniform, Rebelo_real and Fornés, and achieved state-of-the-art results for all these datasets. Additionally, we have evaluated our model on publicly available two non-music symbols datasets, namely CMATERdb 2.1.2 containing 120 handwritten Bangla city names and CMATERdb 3.1.1 dataset containing handwritten Bangla numerals to validate its effectiveness on diversified datasets. The source code of this present work is available at https://github.com/ashis0013/Music-Symbol-Recognition. [ABSTRACT FROM AUTHOR]

Graph neural networks (GNN) uphold the essence of irregularly structured information embedded in a graph via message passing among the nodes and aggregating the node features at various levels of the graph. In the past, researchers have extensively used the GNN models for several semi-supervised node classification tasks. Existing GNN models do not use nodes' information sufficiently. The use of inter-node feature-level correlational information with the existing GNN models might lead to more powerful learning models. Here, a weighting scheme has been developed for message passing and aggregation functions. This model has been named "Vector GNN", or in short, "VecGNN", due to its relationship with vector space. VecGNN takes into consideration the relative position of a node with respect to its neighboring nodes in the feature space, which influences the weight of features passed to the information aggregation phase. These weights are assigned using two different statistical measures: Jaccard's coefficient and Cosine similarity. The proposed weighting scheme uses a generalized approach that can be easily incorporated into several GNN frameworks. VecGNN is evaluated using three citation datasets: Citeseer, Pubmed, and Cora. On these datasets, three sets of experiments have been conducted with varying numbers of training and testing nodes. We have used training, validation, and test set nodes with ratios of 1:1:8, 2:1:7, and 3:1:6. Experimenting on these, we observe an improvement of 2%–4% over the baseline models: Graph Convolution Network (GCN), Graph Attention Network (GAT), and Jumping Knowledge Networks (JKNets). The source code is available at the link https://github.com/sourodeeproy/VecGNN. [ABSTRACT FROM AUTHOR]

The rapid outbreak of COVID-19 has affected the lives and livelihoods of a large part of the society. Hence, to confine the rapid spread of this virus, early detection of COVID-19 is extremely important. One of the most common ways of detecting COVID-19 is by using chest X-ray images. In the literature, it is found that most of the research activities applied convolutional neural network (CNN) models where the features generated by the last convolutional layer were directly passed to the classification models. In this paper, convolutional long short-term memory (ConvLSTM) layer is used in order to encode the spatial dependency among the feature maps obtained from the last convolutional layer of the CNN and to improve the image representational capability of the model. Additionally, the squeeze-and-excitation (SE) block, a spatial attention mechanism, is used to allocate weights to important local features. These two mechanisms are employed on three popular CNN models – VGG19, InceptionV3, and MobileNet to improve their classification strength. Finally, the Sugeno fuzzy integral based ensemble method is used on these classifiers' outputs to enhance the detection accuracy further. For experiments, three chest X-ray datasets, which are very prevalent for COVID-19 detection, are considered. For all the three datasets, it is found that the results obtained by the proposed method are comparable to state-of-the-art methods. The code, along with the pre-trained models, can be found at https://github.com/colabpro123/CovidConvLSTM. • Proposed a model to classify COVID-19, Pneumonia, and Normal chest X-ray images. • Generated feature maps using three pre-trained standard deep learning models. • Improved the spatial dependency among feature maps prior to use as features. • Used squeeze-and-excitation block as a spatial attention scheme. • Utilized the Sugeno fuzzy integral based classifier ensembling technique. [ABSTRACT FROM AUTHOR]

Though handwriting recognition is a well-explored research area for decades, there are a few sub-areas of this field that have still not obtained much attention from the researchers. Some examples include recognition of hand-drawn graphics components like circuit components and diagrams. Complete digitization of such handwritten documents is not possible without automatic conversion of the said circuit diagrams. Besides, to date, in most of the cases for commercial circuit design purposes, concerned people manually enter the components into the simulating software like Cadence, Spice to analyze the circuit and judge its performance. In this work, it has been tried to move one step towards automating this process by recognizing the hand-drawn circuit components which are considered as the most important step for this automation. The present endeavour is to design a two-stage convolutional neural network (CNN)-based model that recognizes the hand-drawn circuit components. In the first stage, all the similar-looking (i.e., similar shape and structure) circuit components are clustered into a single group using visual perception and input from confusion matrix of single-stage CNN-based classification, and in the later stage, the circuit components belong to the same group are classified into their actual classes. The proposed model has been evaluated on a self-made database where 20 different classes of hand-drawn circuit components are considered. The experimental outcome shows that the proposed two-stage classification model provides an accuracy 97.33% which is much higher than the single-stage method, which provides an accuracy of 86.00%. [ABSTRACT FROM AUTHOR]

Handwritten word recognition is considered as an active research area since long because of its various real life applications. The key obstacle of this research problem is the huge variation of the writing styles of different individuals. In addition to that the complex shapes of alphabet make the recognition process more difficult. A holistic word recognition approach is proposed here in order to classify 80-class handwritten city name images written in Bangla script. Based on the negative refraction property of the light, a novel shape-based feature vector of size 186 is generated from each of the word images. Effectiveness of the feature vector is tested on a database containing total 12000 handwritten word images having equal number of samples from each class. The proposed method achieves a reasonably good recognition accuracy of 87.50% which proves better while comparing with some of the recently published feature vectors used for similar job. The reported result is achieved by combining the classifiers namely Sequential Minimal Optimization (SMO), Simple Logistic and CV Parameter Selection embedded with SMO. To verify the robustness of the present method it is also applied on handwritten word images written in Roman and Devanagari scripts separately and it is found that our method obtains satisfactory result on the both the cases. [ABSTRACT FROM AUTHOR]

With the advent of image generative technologies, there is a huge growth in the development of facial manipulation techniques that allow people to easily modify media data like videos and images by changing the identity or facial expression of the target person with another person's face. Colloquially, these manipulated videos and images are termed "deepfakes". As a result, every piece of content in digital media comes with a question — is this authentic? Hence, there is an unprecedented need for a competent deepfakes detection method. The rapid changes in forging methods make this a very challenging task and thus generalization of the detection methods is also of utmost required. However, the generalization strengths of the prevailing deepfakes detection methods are not satisfactory. In other words, these models perform well when trained and tested on the same dataset but fail to perform satisfactorily when models are trained on one dataset and tested on another. The most modern deep learning aided deepfakes detection techniques looked for a consistent pattern among the leftover artifacts in specific facial regions of the target face rather than the entire face. To this end, we propose a Vision Transformer with Xception Network (ViXNet) to learn the consistency of these almost imperceptible artifacts left by deepfaking methods on the entire facial region. The ViXNet comprises two branches — one tries to learn inconsistencies among local face region specifics by combining patch-wise self-attention module and vision transformer, and the other generates global spatial features using a deep convolutional neural network. To assess the performance of ViXNet, we evaluate it using two different experimental setups — intra-dataset and inter-dataset when using three standard deepfakes video datasets, namely FaceForensics++, and Celeb-DF (V2) and one deepfakes image dataset called Deepfakes. We have attained 98.57% (83.60%), 99.26% (74.78%), and 98.93% (75.13%) AUC scores using intra(inter)-dataset experimental setups on FaceForensics++, Celeb-DF (V2), and Deepfakes datasets respectively. Additionally, we have evaluated ViXNet on the Deepfake Detection Challenge (DFDC) dataset and we have obtained 86.32% AUC score and 79.06% F1-score on the said dataset. Performances of the proposed model are comparable to state-of-the-art methods. Besides, the obtained results ensure the robustness and the generalization ability of the proposed model. • Proposed a deep learning based model for deepfake image/video detection. • It has a patch-wise self-attention module which learns local image artifacts. • It consists of a vision transformer which learns correlation among masked patches. • Xception based global image features are stacked with patch based local features. • The model achieves good results on some standard video forgery detection datasets. [ABSTRACT FROM AUTHOR]

Among various biometric systems, an offline signature verification system has been widely used in all fields such as in banks, educational institutes, legal procedures and, criminal investigation where authentication and verification are utmost required. Despite the popularity of the online signature verification system, its offline counterpart still has great importance in developing countries, especially in rural areas, where easy availability of smart devices along with fast internet connection is not available. In this work, we have developed a language invariant offline signature verification model which is almost equally applicable for both writer dependent and writer independent scenarios. At first, an offline signature is collected as an image, following which a corresponding signal is generated using singular value decomposition. Then four different kinds of features namely, statistical, shape-based, similarity-based, and frequency-based are extracted from the transformed signal of the signature image. Next, to reduce the feature dimension, we have designed a novel wrapper feature selection method based on Red Deer Algorithm, a recently proposed meta-heuristic method, to keep only the relevant features to be used during signature authentication and verification process. Finally, a confidence score from the Naïve Bayes classifier has been used to perform the authentication and verification process. Our model has been evaluated on CEDAR (English), UTSig (Persian), Sigcomp 2011 Dutch, Sigcomp 2011 Chinese, and SigWIcomp 2015 Bengali signature datasets. Obtained results confirm that the proposed model can outperform many of its predecessors. • Developed a language invariant signature verification framework. • Converted signature images to signals utilizing singular value decomposition. • Extracted features from signals rather than images unlike other methods. • Used Red Deer Algorithm as a meta-heuristic wrapper feature selection method. • Achieved satisfactory performance as compared to some state-of-the-art methods. [ABSTRACT FROM AUTHOR]