Search

• Introduced and formulated the problem of Bangla sign language (BdSL) alphabet recognition in Zero-shot Learning. • Created a new BdSL semantic attribute representation. • Proposed a seen/unseen split of BdSL signs for the Zero-shot learning approach. • Developed a transfer learning based approach to recognize both one-handed and two handed BdSL. • Compiled and evaluated our model on a large-scale dataset and made this publicly available on GitHub. Bangla, being the fifth most spoken language in the world has its own distinct sign language with two methods (one-handed and two-handed) of representation. However, a standard automatic recognition system of Bangla sign language (BdSL) is still to be achieved. Though widely studied and explored by researchers in the past years, certain unaddressed issues like identifying unseen signs and both types of BdSL or lack of evaluation of the models in versatile environmental conditions demarcate the real-world implementation of the automatic recognition of BdSL. To find a probable solution to the shortcomings in the existing works, this paper proposes two approaches based on conventional transfer learning and contemporary Zero-shot learning (ZSL) for automatic BdSL alphabet recognition of both seen and unseen data. The performance of the proposed system is evaluated for both types of Bangla sign representations as well as on a large dataset with 35,149 images from over 350 subjects, varying in terms of backgrounds, camera angle, light contrast, skin tone, hand size, and orientation. For the ZSL approach, a new semantic descriptor dedicated to BdSL is created and a split of the dataset into seen and unseen classes is proposed. Our model achieved 68.21%, 91.57%, and 54.34% of harmonic mean accuracy, seen accuracy, and zero-shot accuracy with six unseen classes respectively. For the transfer learning-based approach, we found pre-trained DenseNet201 architecture to be the best performing feature extractor and Linear Discriminant Analysis as the best classifier with an overall accuracy of 93.68% on the large dataset after conducting quantitative experimentation on 18 CNN architectures and 21 classifiers. The satisfactory result from our models supports its very probative potential to serve extensively for the hearing and speaking impaired community. [ABSTRACT FROM AUTHOR]

Abstract Hearing impaired people have own language called Sign Language but it is difficult for understanding to general people. Sign language is the basic method of communication for deaf people during their everyday of life. Sign digits are also a major part of sign language. So machine translator is necessary to allow them to communicate with general people. For making their language understandable to general people, computer vision based solutions are well known nowadays. In this research work we aims at constructing a model in deep learning approach to recognize Bangla Sign Language (BdSL) digits. In this approach there used Convolutional Neural Network (CNN) to train particular signs with a respective training dataset (Eshara-Lipi) for acquiring our aim. The model trained and tested with respectively 860 training images and 215 (20%) test images of tent classes of digits. Finally, the training model gained about 95% accuracy at recognition of Bangla sign language digits. This model will contribute for moving one step forward to make BdSL machine translator. [ABSTRACT FROM AUTHOR]

Sign language is the comprehensive medium of mass communication for hearing and speaking impaired individuals. As they cannot speak or hear, they are not able to use sound or vocal signals as an information medium for their communication. Rather, they are bound to exchange visual signals to express their feeling in their day-to-day life. For this, they use various body language mainly hand gestures as sign language. Sign language fundamentals can be largely divided into two parts namely digits (numerals) and characters (alphabetical). In this paper, we proposed a hybrid model consisting of a deep transfer learning-based convolutional neural network with a random forest classifier for the automatic recognition of Bangla Sign Language (numerals and alphabets). The overall performance of the presented system is verified on 'Ishara-Bochon' and 'Ishara-Lipi' datasets. 'Ishara-Bochon' and 'Ishara-Lipi' are datasets of isolated numerals and alphabets respectively which are the first complete multipurpose open-access dataset for Bangla Sign Language (BSL). Besides, we also proposed a background elimination algorithm that removes unnecessary features from the sign images. Along with the proposed background elimination technique, the system is able to achieve accuracy, precision, recall, f1-score values of 91.67%, 93.64%, 91.67%, 91.47% for character recognition and 97.33%, 97.89%, 97.33%, 97.37% for digit recognition respectively. The detailed experimental analysis assures the feasibility and effectiveness of the proposed system for BSL recognition. • Demonstration of recent advancements in various Sign Language recognition research. • Employment of proposed background elimination algorithm to remove unwanted features. • Hybridization of transfer learning model with the Random Forest classifier. • Employment of backbone networks pre-trained on ImageNet to handle smaller datasets. • Improvement of evaluation parameters compared to other existing recognition systems. [ABSTRACT FROM AUTHOR]