Search

A large amount of parallel training corpus is necessary for robust, high-quality voice conversion. However, such parallel data may not always be available. This letter presents a new voice conversion method that needs no parallel speech corpus, and adopts a restricted Boltzmann machine (RBM) to represent the distribution of the spectral features derived from a target speaker. A linear transformation was employed to convert the spectral and delta features. A conversion function was obtained by maximizing the conditional probability density function with respect to the target RBM. A feasibility test was carried out on the OGI VOICES corpus. Results from the subjective listening tests and the objective results both showed that the proposed method outperforms the conventional GMM-based method.

With the development of portable ultrasound imaging systems, prosthetic hands using ultrasound images (UIs) of the human forearm has been studied. Since the features for the estimation of finger positions were heuristically determined from the corresponding UIs in the previous studies, optimal performance was not guaranteed. In this paper, we propose a method to ensure optimal performance for the estimation of finger movements using convolutional neural networks (CNN). The experimental results showed that the proposed method revealed the improved performance over previous methods. The best performance of the proposed method was RMSE of 0.0571. A validation test was also carried out to verify the usefulness of the UI features resulted from the trained CNN, using Gradient-weighted Class Activation Mapping (Grad-CAM).

Continuous recognition of ingested foods without user intervention is very useful for the pre-screening of obesity and diet-related disease. An automatic food recognition method that combines the two modalities of audio and ultrasonic signals (US) is proposed in this study. Under a noise-free environment, classification accuracy of an audio-only recognizer is generally higher than that of US-only recognizers, but the performance of US recognizers is unaffected by acoustic noise levels. In the recognition system presented herein, the likelihood score of the audio-US feature was given by a linear combination of class-conditional observation log-likelihoods for two classifiers, using the appropriate weights. We developed a weighting process adaptive to signal-to-noise ratios (SNRs). The main objective here involves determining the optimal SNR classification boundaries and constructing a set of optimum stream weights for each SNR class. A feasibility test was conducted to verify the usefulness of the proposed method by conducting recognition experiments on seven types of food. The performance was compared with conventional methods that use in-ear and throat microphones. The proposed method yielded remarkable levels of recognition performance of 90.13% for artificially added noise and 89.67% under actual noisy environments, when the SNR ranged from 0 to 20 dB.

A voice transformation (VT) method that can make the utterance of a source speaker mimic that of a target speaker is described. Speaker individuality transformation is achieved by altering four feature parameters, which include the linear prediction coefficients cepstrum (LPCC), Δ LPCC, LP-residual and pitch period. The main objective of this study involves construction of an optimal sequence of features selected from a target speaker’s database, to maximize both the correlation probabilities between the transformed and the source features and the likelihood of the transformed features with respect to the target model. A set of two-pass conversion rules is proposed, where the feature parameters are first selected from a database then the optimal sequence of the feature parameters is then constructed in the second pass. The conversion rules were developed using a statistical approach that employed a maximum likelihood criterion. In constructing an optimal sequence of the features, a hidden Markov model (HMM) with global control variables (GCV) was employed to find the most likely combination of the features with respect to the target speaker’s model. The effectiveness of the proposed transformation method was evaluated using objective tests and formal listening tests. We confirmed that the proposed method leads to perceptually more preferred results, compared with the conventional methods.

A voice transformation (VT) method that can make the utterance of a source speaker mimic that of a target speaker is described. Speaker individuality transformation is achieved by altering three feature parameters, which include the LPC cepstrum, pitch period and gain. The main objective of this study involves construction of an optimal sequence of features selected from a target speaker’s database, to maximize both the correlation probabilities between the transformed and the source features and the likelihood of the transformed features with respect to the target model. A set of two-pass conversion rules is proposed, where the feature parameters are first selected from a database then the optimal sequence of the feature parameters is then constructed in the second pass. The conversion rules were developed using a statistical approach that employed a maximum likelihood criterion. In constructing an optimal sequence of the features, a hidden Markov model (HMM) was employed to find the most likely combination of the features with respect to the target speaker’s model. The effectiveness of the proposed transformation method was evaluated using objective tests and informal listening tests. We confirmed that the proposed method leads to perceptually more preferred results, compared with the conventional methods.

Myoelectric signals (MESs) from the speaker's mouth region have been successfully shown to improve the noise robustness of automatic speech recognizers (ASRs), thus promising to extend their usability in implementing noise-robust ASR. In the recognition system presented herein, extracted audio and facial MES features were integrated by a decision fusion method, where the likelihood score of the audio-MES observation vector was given by a linear combination of class-conditional observation log-likelihoods of two classifiers, using appropriate weights. We developed a weighting process adaptive to SNRs. The main objective of the paper involves determining the optimal SNR classification boundaries and constructing a set of optimum stream weights for each SNR class. These two parameters were determined by a method based on a maximum mutual information criterion. Acoustic and facial MES data were collected from five subjects, using a 60-word vocabulary. Four types of acoustic noise including babble, car, aircraft, and white noise were acoustically added to clean speech signals with SNR ranging from -14 to 31 dB. The classification accuracy of the audio ASR was as low as 25.5%. Whereas, the classification accuracy of the MES ASR was 85.2%. The classification accuracy could be further improved by employing the proposed audio-MES weighting method, which was as high as 89.4% in the case of babble noise. A similar result was also found for the other types of noise.

A voice transformation method which changes the source speaker's utterances so as to sound similar to those of a target speaker is described. Speaker individuality transformation is achieved by altering the LPC cepstrum, average pitch period and average speaking rate. The main objective of the work involves building a nonlinear relationship between the parameters for the acoustical features of two speakers, based on a probabilistic model. The conversion rules involve the probabilistic classification and a cross correlation probability between the acoustic features of the two speakers. The parameters of the conversion rules are estimated by estimating the maximum likelihood of the training data. To obtain transformed speech signals which are perceptually closer to the target speaker's voice, prosody modification is also involved. Prosody modification is achieved by scaling excitation spectrum and time scale modification with appropriate modification factors. An evaluation by objective tests and informal listening tests clearly indicated the effectiveness of the proposed transformation method. We also confirmed that the proposed method leads to smoothly evolving spectral contours over time, which, from a perceptual standpoint, produced results that were superior to conventional vector quantization (VQ)-based methods

The automatic labeling of a large speech corpus plays an important role in the development of a high-quality Text-To-Speech (TTS) synthesis system. This paper describes a method for the automatic labeling of speech signals, which mainly involves the construction of a large database for a TTS synthesis system. The main objective of the work involves the refinement of an initial estimation of phone boundaries which are provided by an alignment, based on a Hidden Markov Model. A multilayer perceptron (MLP) was employed to refine the phone boundaries. To increase the accuracy of phoneme segmentation, several specialized MLPs were individually trained based on phonetic transition. The optimum partitioning of the entire phonetic transition space and the corresponding MLPs were constructed from the standpoint of minimizing the overall deviation from the hand-labeling position. The experimental results showed that more than 93% of all phone boundaries have a boundary deviation from a reference position smaller than 20 ms. We also confirmed that the database constructed using the proposed method produced results that were perceptually comparable to a hand-labeled database, based on subjective listening tests.

In text-to-speech synthesis, spectral smoothing is often employed to reduce artifacts at unit-joining points. A context-adaptive smoothing method is proposed in this letter, where the amount of smoothing is determined according to context information. Discontinuities at unit boundaries are predicted by a regression tree, and smoothing factors are computed by using predicted discontinuities and real discontinuities at unit boundaries. Experimental results are presented to demonstrate the effectiveness of the proposed method.

This letter addresses a temporal decomposition (TD) technique that is based on a rate-distortion criterion. In the proposed TD scheme, a set of interpolation functions is constructed from a given training corpus, and the optimum target points are found in the sense of minimizing, not only spectral distortion, but also bit rates. The results of the simulation show that an average spectral distortion of about 1.4 dB can be achieved at an average bit rate of about 8 bits/frame.

It is well known that a strong relationship exists between human voices and the movement of articulatory facial muscles. In this paper, we utilize this knowledge to implement an automatic speech recognition scheme which uses solely surface electromyogram (EMG) signals. The sequence of EMG signals for each word is modelled by a hidden Markov model (HMM) framework. The main objective of the work involves building a model for state observation density when multichannel observation sequences are given. The proposed model reflects the dependencies between each of the EMG signals, which are described by introducing a global control variable. We also develop an efficient model training method, based on a maximum likelihood criterion. In a preliminary study, 60 isolated words were used as recognition variables. EMG signals were acquired from three articulatory facial muscles. The findings indicate that such a system may have the capacity to recognize speech signals with an accuracy of up to 87.07%, which is superior to the independent probabilistic model.

A newly developed image enhancement algorithm is described in this contribution. The proposed algorithm makes use of the signal subspace method to enhance images corrupted by uncorrelated additive noise. This enhancement is performed by eliminating the noise subspace and estimating clean image from the remaining signal subspace. We propose the block-adaptive Wiener filtering which engages properties of the human visual system to estimate clean image. This criterion enables one to not only preserve the detailed structure of the given image, but to reduce the level of background noise as well. Subjective evaluation tests show the superiority of the method proposed here. In particular, edge blurring effects are noticeably reduced compared to the conventional methods.

A method for the automatic segmentation of speech signals is described. The method is dedicated to the construction of a large database for a Text-To-Speech (TTS) synthesis system. The main issue of the work involves the refinement of an initial estimation of phone boundaries which are provided by an alignment, based on a Hidden Markov Model (HMM). Multi-layer perceptron (MLP) was used as a phone boundary detector. To increase the performance of segmentation, a technique which individually trains an MLP according to phonetic transition is proposed. The optimum partitioning of the entire phonetic transition space is constructed from the standpoint of minimizing the overall deviation from hand labelling positions. With single speaker stimuli, the experimental results showed that more than 95% of all phone boundaries have a boundary deviation from the reference position smaller than 20 ms, and the refinement of the boundaries reduces the root mean square error by about 25%.

We describe a voice transformation method which changes the source speaker's acoustic features to those of a target speaker. In the method acoustic features are divided into two parts, linear and nonlinear parts. Linear parts are characterized by LPC cepstrum coefficients which are obtained from LP analysis. The nonlinear part, which represents the excitation signal, is modelled by the long-delay nonlinear predictor using a neural net. Conversion rules for the excitation signal are generated by the average pitch ratio and the mapping codebook, and those for LPC cepstrum coefficients are based on the orthogonal vector space conversion. In addition, the spectral envelope compensation is proposed to correct spectral distortion. In the transformed speech a listening test shows that the proposed method makes it possible to convert speaker's individuality while maintaining high quality.