Search

Speech recognition in noisy environments is one of the long-standing research themes but remains a very important challenge nowadays. Therefore, there is much research into all techniques and approaches to improve the performance of speech recognition systems, even in poor conditions. This paper presents a comparative study under various conditions based on two architectures (GMM-HMM and DNN-HMM), the Hybrid GMM-HMM models using the CMU Sphinx tools and the Hybrid DNN-HMM using the KALDI toolkit in noise environment. In this study, we compare the Hybrid GMM-HMM models and the Hybrid DNN-HMM models to evaluate the performance of the proposed system. The novelty of this paper is to test if the presented tools could be, with good accuracy, recognize the Arabic speech principally in noisy environment. In addition, we adopted the noisy training theory in this paper based on GMM-HMM and DNN-HMM model. We use the public Arabic Speech Corpus for Isolated Words (20 words), three noise levels, and three noise types. The implementation of our system consists of two phases: Features extraction using Mel-frequency Cepstral Coefficient (MFCC) and the classification phase will use separately the previous two models. In order to test the performance of these methods a simulation will presented for different SNR and for different district type of noise.

This paper presents the analysis and classification of speech spectrograms for recognizing emotions in RAVDESS dataset. Feature extraction from speech utterances is performed using Mel-Frequency Cepstrum Coefficient. Thereafter, deep neural networks are employed to classify speech into six emotions (happy, sad, neutral, calm, disgust, and fear). Firstly, this paper presents a comprehensive comparative study on DNNs on prosodic features. The outcomes of all DNNs are presented in the paper. Secondly, the paper puts forward an analysis of Bag of Visual Words that uses speeded-up robust features (SURF) to cluster them using K-means and further classify them using support vector machine (SVM) into aforementioned emotions. Out of the five DNNs deployed, (i) Long Short-Term Memory (LSTM) on MFCC and, (ii) Multi-Layer Perceptron (MLP) classifier on MFCC, outperforms others, giving an accuracy score of 0.70 (in both cases). Further, the BoVW technique performed 53% of correct classification. Therefore, the proposed methodology constructs a Hybrid of Acoustic Features (HAF) and feeds them into an ensemble of bagged multi-layer perceptron classifier imparting an accuracy of 85%. Also, it achieves a precision score between 0.77 and 0.88 for the classification of six emotions.

Automatic speaker verification (ASV) systems are enhanced enough, that industry is attracted to use them practically in security systems. However, vulnerability of these systems to various direct and indirect access attacks weakens the power of ASV authentication mechanism. The increasing research in spoofing and anti-spoofing technologies is contributing to the enhancement of these systems. The objective of this paper is to review and analyze these important advancements proposed by different researchers and scientists. Various classical, autoregressive, cepstral, etc., and modern deep learning based feature extraction techniques that are chosen to design the frontend of these systems are discussed. Extracted features are learned and classified in the backend of an ASV system, which can be classical machine learning or deep learning models that are also the main focus of the presented review. Experimental studies use constantly modified datasets and evaluation measures to develop robust systems since emergence of practical work in this area. This paper analysis most of the contributing spoofed speech datasets and evaluation protocols. Speech synthesis (SS), voice conversion (VC), replay, mimicry and twins are the potential spoofing attacks to ASV systems. This work provides the knowledge of generation techniques of these attacks to empower the defence mechanism of ASV. This survey marks the start of a new era in ASV system development and highlights the start of a new generation (G4) in SS attack development methods. With the increase in advancement of deep learning techniques, the paper makes best efforts to give the complete idea of ASV to new comers to this area and also, puts some light on some of the spoofing attacks that can be targeted during implementation of the future ASV systems.

This paper proposes harmonic vector analysis (HVA) based on a general algorithmic framework of audio blind source separation (BSS) that is also presented in this paper. BSS for a convolutive audio mixture is usually performed by multichannel linear filtering when the numbers of microphones and sources are equal (determined situation). This paper addresses such determined BSS based on batch processing. To estimate the demixing filters, effective modeling of the source signals is important. One successful example is independent vector analysis (IVA) that models the signals via co-occurrence among the frequency components in each source. To give more freedom to the source modeling, a general framework of determined BSS is presented in this paper. It is based on the plug-and-play scheme using a primal-dual splitting algorithm and enables us to model the source signals implicitly through a time-frequency mask. By using the proposed framework, determined BSS algorithms can be developed by designing masks that enhance the source signals. As an example of its application, we propose HVA by defining a time-frequency mask that enhances the harmonic structure of audio signals via sparsity of cepstrum. The experiments showed that HVA outperforms IVA and independent low-rank matrix analysis (ILRMA) for both speech and music signals. A MATLAB code is provided along with the paper for a reference ( https://doi.org/10.24433/CO.9507820.v1 ).

Homo-sapiens suffer from psychogenic pain due to current day lifestyle. According to psychologists, stress is the most destructive form of psychalgia and it is a vicious companion for this species. Immoderate levels of stress may lead to the death of many individuals. Normally, the presence of stress gives rise to certain emotions which can be detected to predict stress levels of a person. This paper proposes the development of mechanized and efficient Speech Emotion Recognition (SER) for stress level analysis. The paper investigates the performance of perceptual based speech features like Revised Perceptual Linear Prediction Coefficients, Bark Frequency Cepstral Coefficients, Perceptual Linear Predictive Cepstrum, Gammatone Frequency Cepstral coefficient, Mel Frequency Cepstral Coefficient, Gammatone Wavelet Cepstral Coefficient and Inverted Mel Frequency Cepstral Coefficients on SER. The novelty of this work involves application of a SemiEager (SemiE) learning algorithm for evaluating auditory cues. SemiE offers advantages over eager and lazy based learning by reducing the computational cost. Stress level recognition being the main objective, the Speech Under Simulated and Actual Stress (SUSAS) benchmark database is used for performance analysis. A comparative analysis is presented to demonstrate the improvement in the SED performance. An overall accuracy of 90.66% recognition of stress related emotions is achieved.

Spotting of keywords in continuous speech signal with the aid of the computer is called a keyword spotting (KWS) system. A variety of strategies have been suggested in the literature to detect keywords from the adult’s speech effectively. However, only a limited number of studies have been reported for KWS in children’s speech. Due to the difference in physiological properties, the pitch and speaking rate of children’s differ from the adult’s. Consequently, KWS system model parameters trained on the speech data from adult’s signal yield poor performance for children speech. In this paper, we have developed a KWS system for spotting keywords from children’s speech using models trained on adults’ speech. The proposed approach uses spectral moment time–frequency distribution augmented by low-order cepstral (SMAC) as the front-end feature. The mismatches due to differences in pitch and speaking rate of children and adult speakers are further mitigated by data-augmented training using explicit pitch and speaking rate modifications. The experimental findings presented in this paper show that the SMAC feature offers significantly better output for both clean and noisy test conditions than the conventional Mel frequency cepstral coefficients.

This paper explores the behavior of different normalization techniques viz. cepstral mean normalization, cepstral variance normalization, cepstral mean subtraction. cepstral mean and variance normalization, wiener filter, and spectral subtraction in noisy conditions. The performance parameters viz. EER (Equal Error Rate) and DCF (Detection Cost Function) has been calculated using NIST 2003 SRE and Aurora 2 with the help of various normalization techniques considered in this paper for different noisy backgrounds at 0, 5 and 10 dB signal-to-noise ratio. The experimental results obtained from these techniques reveal that cepstral mean normalization (CVN) normalization method is found to be better when compared to other normalization techniques used in this paper.

Alignment is a key step before learning a mapping function between a source and a target speaker’s spectral features in various state-of-the-art parallel data Voice Conversion (VC) techniques. After alignment, some corresponding pairs are still inconsistent with the rest of the data and are considered outliers. These outliers shift the parameters of the mapping function from their true value and hence, negatively affect the learning of mapping function during the training phase of the VC task. To the best of the authors’ knowledge, the effect of outliers (and hence, their removal) on quality of the converted voice has not been much explored in the VC literature. Recent research has shown the effectiveness of the outlier removal as a pre-processing step in the VC. In this paper, we extend this study with a detailed theory and analysis. The proposed method uses a score distance that is estimated using Robust Principal Component Analysis (ROBPCA) to detect the outliers. In particular, the outliers are determined using a fixed cut-off on the score distances, based on the degrees of freedom in a chi-squared distribution, which is speaker-pair independent. The fixed cut-off is due to the assumption that the score distances follow the normal (i.e., Gaussian) distribution. However, this is a weak statistical assumption even in the cases where quite many samples are available. Hence, in this paper, we propose to explore speaker-pair dependent cut-offs to detect the outliers. In addition, we have presented our results on two state-of-the-art databases, namely, CMU-ARCTIC and Voice Conversion Challenge (VCC) 2016 by developing various state-of-the-art methods in the VC. In particular, we have presented the effectiveness of the outlier removal on Gaussian Mixture Model (GMM), Artificial Neural Network (ANN), and Deep Neural Network (DNN)-based VC techniques. Furthermore, we have presented subjective and objective evaluations using a 95% confidence interval for the statistical significance of the tests. We obtained an average 0.56% relative reduction in Mel Cepstral Distortion (MCD) with the proposed outlier removal approach as a pre-processing step. In particular, with the proposed speaker-pair dependent cut-off, we have observed relative improvement of 12.24% and 30.51% in the speech quality, and 39.7% and 4.27% absolute improvement in the speaker similarity for the CMU-ARCTIC and the VCC 2016, respectively.

In the field of vibration based condition monitoring a trusted symptom of a defective bearing is the observation of peaks, at characteristic frequencies, in the squared envelope spectrum (SES). If a machine is operating in a varying speed regime the SES is computed on the order tracked signal, i.e. the signal resampled at constant angular increments, and the SES can still be used for diagnostic. Despite its versatility a common problem with the SES is that peaks from other sources of vibrations, as for instance gears, can prevent the diagnosis of a defective bearing. Therefore pre-processing techniques are applied to the vibrational signal before the computation of the SES to enhance the signal from the bearings. Among these techniques cepstral pre-whitening (CPW) has gained much attention offering a remarkable capability of eliminating, in a blind way, both harmonics and modulation side-bands of the unwanted components. In the case of a varying speed regime the usual procedure consists of three steps: order track the signal, calculate the CPW, evaluate the SES. In this paper on the contrary the CPW is applied before the step of order tracking; therefore the proposed approach is: CPW the raw time signal, order tracking, evaluation of the SES. The remarkable observation is that for this approach the cepstrum does not present peaks at characteristic quefrencies, being the raw signal acquired in a varying speed regime. However this paper shows by means of numerical simulations and analysis of experimental data, that with the proposed methodology the masking components coming from the gears are suppressed and the signal from the defective bearing is enhanced.

In this paper, we demonstrate the significance of restoring harmonics of the fundamental frequency (pitch) in deep neural network (DNN) based speech enhancement. We propose a sliding-window attention network to regress the spectral magnitude mask (SMM) from the noisy speech signal. Even though the network parameters can be estimated by minimizing the mask loss, it does not restore the pitch harmonics, especially at higher frequencies. In this paper, we propose to restore the pitch harmonics in the spectral domain by minimizing cepstral loss around the pitch peak. The network parameters are estimated using a combination of the mask loss and cepstral loss. The proposed network architecture functions like an adaptive comb filter on voiced segments, and emphasizes the pitch harmonics in the speech spectrum. The proposed approach achieves comparable performance with the state-of-the-art methods with much lesser computational complexity.

Recent methodologies for audio classification frequently involve cepstral and spectral features, applied to single channel recordings of acoustic scenes and events. Further, the concept of transfer learning has been widely used over the years, and has proven to provide an efficient alternative to training neural networks from scratch. The lower time and resource requirements when using pre-trained models allows for more versatility in developing system classification approaches. However, information on classification performance when using different features for multi-channel recordings is often limited. Furthermore, pre-trained networks are initially trained on bigger databases and are often unnecessarily large. This poses a challenge when developing systems for devices with limited computational resources, such as mobile or embedded devices. This paper presents a detailed study of the most apparent and widely-used cepstral and spectral features for multi-channel audio applications. Accordingly, we propose the use of spectro-temporal features. Additionally, the paper details the development of a compact version of the AlexNet model for computationally-limited platforms through studies of performances against various architectural and parameter modifications of the original network. The aim is to minimize the network size while maintaining the series network architecture and preserving the classification accuracy. Considering that other state-of-the-art compact networks present complex directed acyclic graphs, a series architecture proposes an advantage in customizability. Experimentation was carried out through Matlab, using a database that we have generated for this task, which composes of four-channel synthetic recordings of both sound events and scenes. The top performing methodology resulted in a weighted F1-score of 87.92% for scalogram features classified via the modified AlexNet-33 network, which has a size of 14.33 MB. The AlexNet network returned 86.24% at a size of 222.71 MB.

Accurate rotational speed determination for rotary machines tends to be allocated high priority in technical applications. In some cases, it is not easy to measure shaft speed directly. Vibration diagnostic tools can offer an alternative solution to the problem posed by direct rotational speed measurement. Using calculated spectra and cepstra can help to determine the rotational speed easily and accurately. This paper examines the comparison of the spectrum and cepstrum based methods in terms of their applicable ranges and rotational speed estimation accuracy. Most papers which present similar comparisons state that the speed of rotary machines can be determined better with cepstrum calculation. However, this argument does not entirely stand up to scrutiny. This paper explains how to calculate the possible speed estimation error which arises out of the resolution of the discrete output data produced by the methodologies. The novel hybrid speed estimation logic uses these equations to decide which method results in the most accurate output. Numerous vibration measurements were made to test the usability of this hybrid method. It was successfully tested on electromotive drive trains, as well as spark- and compression-ignition internal combustion engines with different cylinder numbers and arrangements.

This paper presents wavelet-based power normalized spectrum for computing robust cepstral features named WP-PNCC features. The proposed technique computes wavelet packet-based short-time spectrum of speech signal. A nonlinear function is defined as relating power spectrum of clean speech to the power spectrum of speech corrupted with noise. The constants of function are computed from longer-duration speech spectrum, and the short-time spectrum for each frame is weighted with the power function. The weighted speech spectrum is processed with logarithmic and discrete cosine transform operation to compute cepstral coefficients. The cepstral coefficients thus obtained are processed with quantile-based cepstral dynamics normalization technique. The proposed features are examined with hidden Markov model classifier on TIFR database for Hindi phoneme classification task and on TIMIT database for English phoneme classification task along with mel-frequency cepstral coefficients, power normalized cepstral coefficients and 24-band wavelet-based features in clean and noisy environments. Different noises from NOISEX-92 database are used for preparing noisy database with SNR ranging from 20 dB to 0 dB. The results show enhanced performance of proposed features in all the considered cases. The simulations are performed on MATLAB 2015b. The performance of proposed features is also evaluated on hidden Markov model toolkit-based speech recognition system. The comparative results confirm the robustness of proposed features with sufficient improvement over other features examined in this paper.

At present, pathological voice recognition is mainly based on the classification of pathological voice. However, almost all the researches are based on the single vowel \a\ samples, but few on multi-vowels. In addition, the current researches on multi-vowels recognition are mainly for normal voices, which are unsuitable for the speech recognition of normal and pathological multi-vowels simultaneously. This paper concentrates on developing an accurate and robust feature called enhanced-bark line spectrum pair (E-BLSP) to detect and classify normal and pathological multi-vowels. We explore the impact of E-BLSP feature on recognition performance and propose an effective method based on the combination of three features including E-BLSP for pathological and normal multi-vowels. In this paper, first LSP and difference of adjacent LSP (DAL) features of a vowel are extracted. Then LSP feature is warped at bark domain to get bark line spectrum pair (BLSP). In addition, then E-BLSP feature is calculated by adjusting BLSP using DAL feature. Finally, the adjusted E-BLSP feature and other two traditional features, including linear prediction cepstrum coefficient (LPCC) and mel-frequency cepstrum coefficients (MFCC) are applied to support vector machine (SVM) and deep neural network (DNN) classifiers to explore the classification performance of single feature and feature combinations for pathological and normal vowels /a/, /i/ and /u/. The results show that the highest achieved accuracies for DNN and SVM network are 98.6190% and 96.2693%, while the largest achieved area under curves (AUC) are 0.9925 and 0.9868, correspondingly with the combination of three features including LPCC, MFCC, and E-BLSP.

The conventional sound source localization systems require the significant complexity because of multiple synchronized analog-to-digital conversion channels as well as the scalable algorithms. This paper proposes a single-channel sound localization system for transport with multiple receivers. The individual receivers are connected by the single analog microphone network which provides the superimposed signal over simple connectivity based on asynchronized analog circuit. The proposed system consists of two computational stages as homomorphic deconvolution and machine learning stage. A previous study has verified the performance of time-of-flight estimation by utilizing the non-parametric and parametric homomorphic deconvolution algorithms. This paper employs the linear regression with supervised learning for angle-of-arrival prediction. Among the circular configurations of receiver positions, the optimal location is selected for three-receiver structure based on the extensive simulations. The non-parametric method presents the consistent performance and Yule&ndash, Walker parametric algorithm indicates the least accuracy. The Steiglitz&ndash, McBride parametric algorithm delivers the best predictions with reduced model order as well as other parameter values. The experiments in the anechoic chamber demonstrate the accurate predictions in proper ensemble length and model order.

With the support of big data and information technology, various sectors such as sports, health, and medical industry can realize the integration and readjustment of the existing resources, which improve the operation efficiency of the industry and tap its huge potential. With the advancement in big data analysis, voice features, and Internet of Things (IoT), personalized health management is becoming the development trend and breakthrough of sports and health industry. The application of big data will tap out the huge potential of the sports and health industry. In this paper, we have used the Mel-requency cepstrum coefficient as the speech feature processing method. When the linear frequency is transformed to the Mel frequency by Fourier transform, the calculation accuracy will decrease with the increase in the frequency, and the low-frequency signal will be retained to improve the anti-noise ability. With further study of the voice feature processing and IoT model of big data’s sports and health management, a vector addition regression was developed to compare the two real scoring features of the processing results that pave the way for further analysis and result evaluation. Through experimental verification, it is proved that the method in this paper can better learn the speech features. At the same time, with the introduction of noise reduction, the big data of speech recognition in sports health management has a stronger robustness and improves the overall system performance.

The constant Q transform (CQT) has been shown to be one of the most effective speech signal pre-transforms to facilitate synthetic speech detection, followed by either hand-crafted (subband) constant Q cepstral coefficient (CQCC) feature extraction and a back-end binary classifier, or a deep neural network (DNN) directly for further feature extraction and classification. Despite the rich literature on such a pipeline, we show in this paper that the pre-transform and hand-crafted features could simply be replaced by end-to-end DNNs. Specifically, we experimentally verify that by only using standard components, a light-weight neural network could outperform the state-of-the-art methods for the ASVspoof2019 challenge. The proposed model is termed Time-domain Synthetic Speech Detection Net (TSSDNet), having ResNet- or Inception-style structures. We further demonstrate that the proposed models also have attractive generalization capability. Trained on ASVspoof2019, they could achieve promising detection performance when tested on disjoint ASVspoof2015, significantly better than the existing cross-dataset results. This paper reveals the great potential of end-to-end DNNs for synthetic speech detection, without hand-crafted features., Comment: Accepted in IEEE Signal Processing Letters 2021

A new technique of hiding a speech signal clip inside a digital color image is proposed in this paper to improve steganography security and loading capacity. The suggested technique of image steganography is achieved using both spatial and cepstral domains, where the Mel-frequency cepstral coefficients (MFCCs) are adopted, as very efficient features of the speech signal. The presented technique in this paper contributes to improving the image steganography features through two approaches. First is to support the hiding capacity by the usage of the extracted MFCCs features and pitches extracted from the speech signal and embed them inside the cover color image rather than directly hiding the whole samples of the digitized speech signal. Second is to improve the data security by hiding the secret data (MFCCs features) anywhere in the host image rather than directly using the least significant bits substitution of the cover image. At the recovering side, the proposed approach recovers these hidden features and using them to reconstruct the speech waveform again by inverting the steps of MFCCs extraction to recover an approximated vocal tract response and combine it with recovered pitch based excitation signal. The results show a peak signal to noise ratio of 52.4 dB of the stego-image, which reflect a very good quality and a reduction ratio of embedded data to about (6%–25%). In addition, the results show a speech reconstruction degree of about 94.24% correlation with the original speech signal.

In this paper we introduce attention-regression model to demonstrate predicting acoustic features from electroencephalography (EEG) features recorded in parallel with spoken sentences. First we demonstrate predicting acoustic features directly from EEG features using our attention model and then we demonstrate predicting acoustic features from EEG features using a two-step approach where in the first step we use our attention model to predict articulatory features from EEG features and then in second step another attention-regression model is trained to transform the predicted articulatory features to acoustic features. Our proposed attention-regression model demonstrates superior performance compared to the regression model introduced by authors in [1] when tested using their data set for majority of the subjects during test time. The results presented in this paper further advances the work described by authors in [1]., Comment: Under review

In the recent years, along with the development of artificial intelligence (AI) and man-machine interaction technology, speech recognition and production have been asked to adapt to the rapid development of AI and man-machine technology, which need to improve recognition accuracy through adding novel features, fusing the feature, and improving recognition methods. Aiming at developing novel recognition feature and application to speech recognition, this paper presents a new method for articulatory-to-acoustic conversion. In the study, we have converted articulatory features (i.e., velocities of tongue and motion of lips) into acoustic features (i.e., the second formant and Mel-Cepstra). By considering the graphical representation of the articulators’ motion, this study combined Bidirectional Long Short-Term Memory (BiLSTM) with convolution neural network (CNN) and adopted the idea of word attention in Mandarin to extract semantic features. In this paper, we used the electromagnetic articulography (EMA) database designed by Taiyuan University of Technology, which contains ten speakers’ 299 disyllables and sentences of Mandarin, and extracted 8-dimensional articulatory features and 1-dimensional semantic feature relying on the word-attention layer; we then trained 200 samples and tested 99 samples for the articulatory-to-acoustic conversion. Finally, Root Mean Square Error (RMSE), Mean Mel-Cepstral Distortion (MMCD), and correlation coefficient have been used to evaluate the conversion effect and for comparison with Gaussian Mixture Model (GMM) and BiLSTM of recurrent neural network (BiLSTM-RNN). The results illustrated that the MMCD of Mel-Frequency Cepstrum Coefficient (MFCC) was 1.467 dB, and the RMSE of F2 was 22.10 Hz. The research results of this study can be used in the features fusion and speech recognition to improve the accuracy of recognition.

Although the Arabic language is the fourth most popular language in the world, it has not received sufficient attention in artificial intelligence research, especially in automatic speech recognition (ASR). The key feature of the Arabic language is that its words are pronounced exactly as they are written. Above all, taking into account the diacritics, 2 there are no words with similar pronunciation and writing. This motivates us to think of building an Arabic ASR system by recognizing its alphabet phonetics. Therefore, the Arabic alphabet phonemes classification must be studied, this is what the paper aims to achieve. In this paper, we create a new dataset, called Arabic alphabet phonetics dataset (AAPD). AAPD was collected by taking sound recordings of 1420 persons. We build several Arabic alphabet phonemes classification systems using three feature extraction techniques and four deep neural networks. Based on AAPD, we designed numerous experiments to compare the performance of feature extraction and classification methods, which can be used as a benchmark. Experimental results showed that Mel-frequency Cepstral Coefficient (MFCC) is considered most effective to feature extraction due to its highest accuracy, particularly when using 20 for Mel-bands number the training time is the least. Additionally, the appropriate model that achieved the highest accuracy with the least computational load is the proposed model VGG–based, where acquired an accuracy of 95.68%.

This paper extends the concept of scalar cepstrum coefficients from single-input single-output linear time invariant dynamical systems to multiple-input multiple-output models, making use of the Smith-McMillan form of the transfer function. These coefficients are interpreted in terms of poles and transmission zeros of the underlying dynamical system. We present a method to compute the MIMO cepstrum based on input/output signal data for systems with square transfer function matrices (i.e. systems with as many inputs as outputs). This allows us to do a model-free analysis. Two examples to illustrate these results are included: a simple MIMO system with 3 inputs and 3 outputs, of which the poles and zeros are known exactly, that allows us to directly verify the equivalences derived in the paper, and a case study on realistic data. This case study analyses data coming from a (model of) a non-isothermal continuous stirred tank reactor, which experiences linear fouling. We analyse normal and faulty operating behaviour, both with and without a controller present. We show that the cepstrum detects faulty behaviour, even when hidden by controller compensation. The code for the numerical analysis is available online., Comment: 6 pages, 4 figures, submitted for review to L-CSS with CDC option