Search

The Visual Articulators like Teeth, Lips and tongue are correlated to one another and these correlation among those visual features are extracted with visual perceptions. The term visual perceptions indicates the features that are used as a parameter for representation learning and the description of visual information. These visual features are extracted and classified into different classes of Kannada Words. The movements of lips, tongue, and teeth are extracted by analyzing the inner and outer portion of lips along with movement of tongue and teeth. These parts teeth, lips, tongue are together used for feature extraction, as these features are correlated with resonances. These resonance information is extracted from every frames by analyzing and understanding the correlation that exists among them in different sequence of frames of a video. The proposed method of visual perceptions has yielded an accuracy of 82.83% over a dataset having different benchmark challenges. These benchmark challenges include facial tilt as a result of which the correlation may be less among teeth, tongue and lips. Thus, we have erected a new methodology of analyzing and understanding the visual features. The Kannada Words spoken by a person is indicated by assigning labels to the sequence of frames of a video in specific pattern. If these sequence of patterns of data is extracted and visualized from a video, the system recognizes the lip movements into different classes of words spoken.

Lip reading involves the extraction of visual speech information contained in the inner and outer lip contour. Visibility of teeth and tongue during speech provides important speech cues. Particularly for fricatives, the place of articulation can often be determined visually, that is, for labiodentals (upper teeth or lower lip), interdentals (behind tongue or front teeth), and alveolar (tongue touching gun ridge). Other speech information might be contained in the protrusion and wrinkling of lips. Feature extraction is a remarkable process in lip reading as it holds an important role in lip reading classification. In Improved Speeded Up Robust Feature (ISURF), extraction for finding an exact edge is difficult because of more false corner ratio. In case of PSST, exact edges could be obtained with reduced false corner ratio. This paper provides the idea about PSST based on Harris algorithm and gives more precise edge detection in different illumination conditions.

This paper envisage the system provides a statistical based effort to predict lip movements of speaker. The words spoken by a person is identified by analyzing the shape of lip movement in every instance of time. The approach learns the process of prediction of shapes of lips based on recognition of movement. The lip shapes are predicted by annotating and tracking the movement of lips and synchronization of shape recognition with respect to time is achieved by extracting the shape of lips with different statistical information extracted from every frames of a video. Hence, grooved statistical data lends the system with more appropriate shape based in terms of mean, variance, standard deviation and various other statistical features. The proposed system based on statistical features extraction leads to lip movement recognition and mapping of various Kannada words into different classes based on recognition of shape leads the system perform good initiation towards achieving the Lip Reading. The effort has provided overall accuracy of 40.21% with distinct statistical pattern of features extraction and classification.

The Deep Weighted Features play a significant role in extraction of visual features, thus, we present an algorithm to analyze and understand the language of a person with the help of lips movements. The proposed weighted visual features extraction of deep learning makes the system analyze and understand the words used by the person. The weights are calculated from the movement of lips by assigning labels to the shapes of lips in each and every frame of a video with reference to time. The sequence of frames of a video is processed for calculating the weights from the shape of a lip with respect to time in sequence. Thereby we shall train the system to analyze, which character or word is spoken by the person. The deep weighted feature descriptors help the system to analyze and understand the word corresponding to the shape of lips processed in sequence with respect to time. The weighted feature descriptor of deep learning has yielded an accuracy of 84.82% on a dataset provided by the deaf and dumb institution. Further, the methodology works on reading the language of Kannada by analyzing the movement of lips.

Lip tracking exercise is the most important pre-requisite for lip reading system. Most of the lip reading procedures are accessible based on lip contour analysis. Similarly, lip contour extraction is a fundamental footstep. As a result of lip contour extraction, initially the process of lip contour detection in the first frame of an audio-visual image sequence. Subsequently capturing contour in successive frames based on methods of automatic, semiautomatic and hybrid is normally named as lip tracking. Here the effort presents an overview of contemporary works on extracting the face from digital video and classified the face into lip area and non-lip area by categorising the approaches into low and high-level processing techniques. Furthermore, the comparative study of approaches and their effectiveness based on various factors like asymmetric lips, the mouth with visible teeth, tongue and mouth with mustache are offered.