Search

Your search keyword '"Wumaier, Aishan"' showing total 50 results
Start Over Author "Wumaier, Aishan"
50 results on '"Wumaier, Aishan"'

4. Enriching Under-Represented Named-Entities To Improve Speech Recognition Performance

Author

Mao, Tingzhi, Khassanov, Yerbolat, Pham, Van Tung, Xu, Haihua, Huang, Hao, Wumaier, Aishan, and Chng, Eng Siong

Subjects
Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing
Abstract

Automatic speech recognition (ASR) for under-represented named-entity (UR-NE) is challenging due to such named-entities (NE) have insufficient instances and poor contextual coverage in the training data to learn reliable estimates and representations. In this paper, we propose approaches to enriching UR-NEs to improve speech recognition performance. Specifically, our first priority is to ensure those UR-NEs to appear in the word lattice if there is any. To this end, we make exemplar utterances for those UR-NEs according to their categories (e.g. location, person, organization, etc.), ending up with an improved language model (LM) that boosts the UR-NE occurrence in the word lattice. With more UR-NEs appearing in the lattice, we then boost the recognition performance through lattice rescoring methods. We first enrich the representations of UR-NEs in a pre-trained recurrent neural network LM (RNNLM) by borrowing the embedding representations of the rich-represented NEs (RR-NEs), yielding the lattices that statistically favor the UR-NEs. Finally, we directly boost the likelihood scores of the utterances containing UR-NEs and gain further performance improvement.

Published
2020

23. Automatic Speech Disfluency Detection Using wav2vec2.0 for Different Languages with Variable Lengths.

Author

Liu, Jiajun, Wumaier, Aishan, Wei, Dongping, and Guo, Shen

Subjects
SPEECH, CONVOLUTIONAL neural networks, AUTOMATIC speech recognition, TRANSFORMER models, INTERPERSONAL communication, FEATURE extraction
Abstract

Speech is critical for interpersonal communication, but not everyone has fluent communication skills. Speech disfluency, including stuttering and interruptions, affects not only emotional expression but also clarity of expression for people who stutter. Existing methods for detecting speech disfluency rely heavily on annotated data, which can be costly. Additionally, these methods have not considered the issue of variable-length disfluent speech, which limits the scalability of detection methods. To address these limitations, this paper proposes an automated method for detecting speech disfluency that can improve communication skills for individuals and assist therapists in tracking the progress of stuttering patients. The proposed method focuses on detecting four types of disfluency features using single-task detection and utilizes embeddings from the pre-trained wav2vec2.0 model, as well as convolutional neural network (CNN) and Transformer models for feature extraction. The model's scalability is improved by considering the issue of variable-length disfluent speech and modifying the model based on the entropy invariance of attention mechanisms. The proposed automated method for detecting speech disfluency has the potential to assist individuals in overcoming speech disfluency, improve their communication skills, and aid therapists in tracking the progress of stuttering patients. Additionally, the model's scalability across languages and lengths enhances its practical applicability. The experiments demonstrate that the model outperforms baseline models in both English and Chinese datasets, proving its universality and scalability in real-world applications. [ABSTRACT FROM AUTHOR]

Published
2023
Full Text
View/download PDF

49. Automatic Speech Fluency Evaluation Method Based on Multimodality for Putonghua Proficiency Test Propositional Speaking

Author

Liu, Jiajun, Wumaier, Aishan, Zheng, Linna, and Meng, Huazhen

Abstract

The Putonghua Proficiency Test (Putonghua Shuiping Ceshi, PSC) is a speaking test in China. The propositional speaking section in PSC focuses on the ability of speakers to express ideas fluently and accurately without textual reference. However, unlike other sections of the PSC, propositional speaking is still scored manually, which can result in inefficiency, high costs, and subjectivity. To address these issues, an automatic speech fluency evaluation method based on multimodality is proposed. First, different neural networks are used to extract unimodal features. Then, cross-modal attention is applied to achieve multimodal fusion. Finally, fluency evaluation results are obtained using self-attention to reinforce high-contributing information. The proposed method achieves 81.67%accuracy on a self-built dataset, demonstrating that combining textual and acoustic features provides complementary information to improve automatic speech fluency evaluation accuracy.

Published
2022
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results