Search

Continuous positive airway pressure (CPAP) is a typical medical treatment for obstructive sleep apnea syndrome. Without using a small CPAP machine which are patented, we aim to design a new technique using microphone array for measuring respiratory sounds. Hence, it is necessary to obtain effective features of respiratory sounds such as, inspiration, expiration, and snoring in order to supply appropriate pressure depending on the respiratory conditions of patients. Furthermore, it is often difficult to measure only respiratory sounds separately from other environmental sounds by the use of CPAP machine operation, for example, blowing sound of subjects should be eliminated for CPAP machine. As the first step, short-time Fourier transform is applied then we can compare the sound features of inspiration and expiration observed by the microphones. Some different features can be observed in the time-frequency magnitude spectrogram. From these results, the feasibility of using inspiration and expiration features independently in a prospective detection technique.

The blind separation problem of analog modulated radio signals and their DOA (Direction Of Arrival) estimation problem are considered. These problems are very important in radio surveillance. Based on the idea of Carlos and Takada [1], the following two methods are proposed for an Independent Component Analysis (ICA) based radio surveillance system. The first method is concerned with the improvement of DOA estimation accuracy after signal separation by ICA. Another method treats separation and DOA estimation in multipath environment. The effectiveness of the proposed methods is proved by computer simulation.

In this report, we propose a tracking algorithm of speaker direction using microphones located at vertices of an equilateral triangle. The method realizes tracking by minimizing a performance index that consists of the cross spectra at three different microphone pairs in the triangular array. We adopt the steepest descent method to minimize it, and for guaranteeing global convergence to the correct direction with high accuracy, we alter the performance index during the adaptation depending on the convergence state. Through some computer simulation and experiments in a real acoustic environment, we show the effectiveness of the proposed method.

Recently, with rapid progress in multimedia technologies, hopes for a speech-based man-machine interface have increased. For high-quality voice collection and acquisition of the position information of the speaker, direction-of-arrival (DOA) estimation technology for voice with a microphone array has been discussed. However, one problem is large-scale implementation of the arrays. In the present research, a new DOA estimation method for a two-channel microphone array based on the harmonic structure of vowels is proposed. The present method has been applied in a real-world environment and its effectiveness has been confirmed. © 2002 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 86(2): 33–42, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.10038

Direction-of-arrival (DOA) estimation of sound sources by using microphone array is an important technique. Its applications are teleconference system and human-robot communicating system and so on. This paper describes a DOA estimation method using array with arbitrary 3-dimentional configuration. The problem discussed in this study is the case in the presence of spatial aliasing. The proposed method is based on sparseness in time-frequency domain of speech signal. The phase uncertainty caused by spatial aliasing can be solved by selecting reliable time-frequency sells whose propagating vector is mostly close to the unit sphere. Through conducted experiments, the proposed method is proved to estimate more accurately than the conventional methods such as histogram and k-means clustering algorithm. Based on the kernel density algorithm stable accurate estimation of DOA can be realized.

For the purpose of automatic speech recognition and speech animation synthesis, speaker verification and so on, there have been studies on `viseme'. Viseme is a visually identifiable unit of utterance or the equivalent unit in the visual domain of the phoneme in audio domain. The classification and the discrimination method of visemes are still important topics. This paper focuses on the number of classification units and a discrimination procedure of Japanese visemes: We extend the number of visemes from 6 to 9 to expanse the word representation by their series, then propose the hierarchical weighted discrimination using multiple discriminative analysis (MDA) to enhance the discriminative ability. In order to verify and discuss the availability of our proposals, visemes discrimination and word recognition experiments were conducted. From these results, the validity of the proposed methods was confirmed.

In recent years, automatic lip reading based on `visemes' have been studied by researchers for realizing human-machine interactive communication system in many applications. However there are a lot of problems such as the definition of the number of viseme classes, discrimination method of visemes, speech recognition method based on visemes, and so on. In this paper, a novel classification of Japanese visemes and hierarchical weighted discrimination method for speech recognition are proposed to address these problems. We augmented the classification number of visemes from 6(conventional) to 9 to represent the words in more detailed by visemes. In addition, considering the difficulty in discriminating with increase of the number of visemes, the hierarchical weighted discrimination method is proposed. For the purpose of comparing with the conventional method, the ATR phonetically balanced word group, which is large vocabulary and includes various visemes, was used and applied to word recognition experiments. From these results, we confirmed the proposed method worked well.

One of the most important issues of signal processing is to identify the model that can describe the generation process for an observed signal through the statistical analysis of the random signal. Recently, as an identification method for such process, a method is considered in which the phase information of the signal using higher-order cumulants is paid close attention. This paper proposes a method in which the observed one-dimensinal signal is modeled as the output signal of a linear time-invariant system generated with the non-Gaussian white process. the generation systm for the signal (not constrained to be minimum phase) is identified using the third-order cumulant of the observed signal. Because the system characteristics are decomposed into their components in the cepstral domain, the proposed method does not require the factorization procedure. In other words, the feature of the method is that it is extendable to the multidimensional system identification, for which the factorization generally is impossible.

Recently, interest on artificial robot audition is growing for developing human-robot interaction. The main purposes of an artificial audio system mounted on mobile robot are localizing sound sources, separating speech signal that is relevant to a particular speaker such as robot’s master, and processing speech sources to extract useful information such as master’s uttering commands. This paper reports a novel proposed method of a speaker’s direction tracking algorithm, and a realization of the real tracking system on a mobile robot. Basic approach of this study belongs to a category of direction finding known as sparseness-based one which employs time-frequency decomposition and disjoint property between different speech signals. The novel points in the proposed source tracking exist on a reliable data selection from time-frequency cells and the application of mean shift tracking to the kernel density estimator derived from these reliable time-frequency components. A wheel-based mobile robot is developed and built-in audio processing system. Experiments are conducted and demonstrate the ability to localize in real environments.

This study proposes a method to track multiple moving sources based on time-frequency analysis. Specific feature of moving source tracking problem is to obtain accurate estimate provided the speakers are stationary within a short time interval, thus stable estimation would be difficult. At first, basic theory for direction finding is developed for arbitrary array configuration and kernel density estimator approach is applied for tracking multiple simultaneously uttered speeches. Expectation maximization approach for the kernel density estimator and new concept of particle groups corresponding to individual sources are proposed. The method alters the number of particle groups dynamically and it can track multiple source DOAs using short period observation. Through conducted experiments, the method is proved to be effective in real environments.

Visual speech recognition or lip reading is an approach for noise robust speech recognition by adding speaker's visual cues to audio information. Basically visual-only speech recognition is applicable to speaker verification and multimedia interface for supporting speaking impaired person. The sequential mouth-shape code method is an effective approach of lip reading for particularly uttered Japanese words by utilizing two kinds of distinctive mouth shapes, known as first and last mouth shapes, appeared intermittently. One advantage of this method is its low computational burden for the learning and word registration processes. This paper proposes a novel word lip recognition system by detecting and determining initial mouth-shape codes to recognize uttering consonants. The proposed method eventually is able to discriminate different words consisting of the same sequential vowel codes though containing different consonant codes. The conducted experiments demonstrate that the proposed system provides higher recognition rate than the conventional ones.

In this paper, a spectrum estimation method is proposed which is based on a two-dimensional autoregressive (AR) model with polar coordinate coefficient arrangement. When the signal is given in terms of an orthogonal coordinate system, such a coordinate system is naturally used for linear prediction in a conventional AR estimation method. The support of the estimation coefficients and the peak shape of the corresponding estimation spectrum are known to be related by a certain rule. By using an AR model in which the coordinate arrangement of the linear prediction is in a polar coordinate, the rotation of the coordinate region keeps an AR model in the same polar coordinate form. Hence, it is possible to determine the relative positional relationship between the estimated spectrum shape and the coefficient mask. It is shown that the present method can estimate the spectrum with a high resolution by using this relationship and avoiding the interference between the spectral peaks.

We are developing two crucial improvements on the time-frequency masking approach to the blind speech separation of underdetermined mixtures when processing anechoic and echoic mixtures. First, the proposed method copes with the usually large amount of delay estimation error that appears in a low frequency band. This step generates a restrictive mask for phase delays on the basis of local and global energy distribution analysis. This mask allows the selected cells to contribute to the orientation histogram. Second, the strong WDO assumption (disjoint orthogonal frequency domain) is relaxed by allowing some frequency bins to be shared by both sources. By detecting fundamental frequencies of speakers at instantaneous time points, mask creation is supported by exploring their harmonic frequencies. The proposed method is proved to be effective and reliable in conducting experiments with both simulated and real-life mixtures. Santrauka Straipsnyje nagrinėjamas aklasis signalų šaltinių išskyrimas apdorojant signalų mišinius su aido efektu ar be jo. Detaliai pristatomi matematiškai bei eksperimentų su dirbtiniais ir realiais šnekos duomenimis rezultatais pagrindžiami du esminiai šio metodo patobulinimai. Pirmasis patobulinimas leidžia sumažinti vėlinimo žemuose dažniuose įtaką šnekos signalo išskyrimo klaidai. Antrasis patobulinimas, paremtas kalbėtojo pagrin­dinio dažnio sekimu, leidžia algoritmui išnaudoti tas pačias dažnių sritis skirtingiems signalų šaltiniams išskirti. Raktiniai žodžiai: šaltinių aklasis atskyrimas; histogramos klasterizavimas; spektrogramos analizė; kalbos rekonstravimas; maskavimas laiko ir dažnių skalėje

Direction Of Arrival (DOA) estimation is an important fundamental technique in the array signal processing field. In this paper, we propose a method, utilizing Hough transform for DOA estimation by a pair of sensors. Our proposed method which is based on time-frequency decomposition, applies Hough transform to the Phase Difference (PD) versus Frequency (PD-F) distribution of received mixture signals, and estimates DOA. By introducing bandwidth in the Hough parameter space, the stability and accuracy of DOA estimation are guaranteed. Through some experiments in a real acoustic circumstance, the effectiveness of the proposed method is shown.

In recent years, to achieve robust speech recognition against noises, Audio-Visual Speech Recognition(AVSR) system utilizing not only audio but also visual information of lip has been studied. This paper proposes a decision method of the weight called stream exponent representing reliability ratio of audio and visual features. The method focuses on the correlation between audio and visual modality in order to estimate the optimal stream exponent. Furthermore, we modified the stream exponent using periodicity of speech, such as pitch, to handle abrupt noises. An audio-visual database is comprised of specific speaker's lip image sequences and audio sequences. The contents of the utterance are Japanese counting numbers and sound-alike words. Using this database we constructed the AVSR system and performed an evaluation experiment. The obtained results verify the availability of the proposed method under a variety of noisy environment.

Blind source separation (BSS) is an approach to extract source signals using observed their mixtures. It is applicable in realizing speech-based human-machine interface systems, such as hands-free phones and robot audition system. This paper proposes the Time-Frequency (T-F) masking method for unilateral triangular microphone array. It is based on the condition of three pairs of Phase Difference versus Frequency (PD-F) distributions in frame-by-frame manner. By integrating these PD-F data, we can estimate more reliable T-F cell feature and will obtain better separation result. The constraints on delays between three pairs of triangular microphone array are applied to the PD-F distributions. Some experiments are performed to verify our method. Section 5 is the conclusion.

Many modern systems rely on transparent human-machine interfaces that allow them to fulfill their purpose in a more efficient and unobtrusive way. In order to build an efficient and reliable speech based human-machine interface, being able to determine when to process the incoming signals even in unfavorable environments is a definite requisite. Our Voice Activity Detection (VAD) method proposes a novel way of mixing monaural and microphone array techniques; monaural techniques are mainly focused on providing robusticity, while microphone array techniques complete the system with the capability of detecting source direction from background noise. This is implemented by first applying a cochlear filtering and channel selection to remove noise, and then a series of strict conditions are applied in order to be able to obtain the fundamental frequencies of the sources which is finally used to obtain the VAD masks.

Speech recognition and speaker detection technique from audio visual fusion information attract much attention. In the visual side information, namely lip reading area, most of recent studies are based on analyzing shape of mouth, whereas few studies are based on analyzing lip motion. However, analysis associated with mouth motion gives essential cues for obtaining utterance mechanics. Thus, as a tool to analyze mouth motion, we focus attention on space-time interest points (STIP) that have been effectively applied for analyzing gait and for recognizing human action. This study stems from the idea that the STIP must be also useful for mouth motion analysis. The proposed mouth motion analysis system using STIP needs neither contour estimation nor feature tracking. Then, several image processings are proposed in order to appropriately apply STIP to mouth motion. Additionally, to evaluate the detected STIP as a tool for mouth motion analysis, we classified Japanese vowels utterances into three motion types by using detected STIPs.

Localization of sound sources by microphone array is one of the important problems. This work proposes the method for sound localization using tetrahedraly-coordinated microphone arrays. This is based on the method provided by Berdugo et al. which estimates the time delays between each pair of micro-phones for an individual frequency. We achieved efficient three-dimensional DOA estimation with no bias, including following two improvements: Spherical pseudo-histogram defining Source-Density Function (SDF), and using the weighted addition of the past histograms. The performance has been evaluated by performing the simulation experiments for static speakers and the experiments for moving speakers. In the simulation experiments, we assumed one static speaker and additional white noise. We evaluated the performance by calculating the success estimation rate as the direction of the speaker varies. The experiments for moving speakers were performed in typical office. The results in both simulations and experiments show the effectiveness of our proposal.

Recently, time-frequency masking algorithm has been interested as a method of BSS (blind source separation) of speech mixtures. It is based on the property of speech signals called W-disjoint orthogonality. This paper aims to realize the omnidirectional BSS system with small computational cost and to provide high separation performance even for closely located sound sources. The time-frequency masking is applied to equilateral triangular array where the three delay estimates from each microphone pairs are obtained. These data are integrated to separate speech signals. To prove the efficiency of the method, some experimental results in real environment are shown.

We propose a method for DOA (direction of arrival) estimation of speech signals in both azimuth and elevation directions. Our previous DOA estimation method achieves high precision and uniform spatial resolution by integrating the frequency array data (Hioka, Y. et al., J. Sig. Process., vol.7, no.1, p.105-9, 2003) generated from microphone pairs in an equilateral-triangular microphone array (Hioka and Hamada, N., IEICE Technical Report, EA2003-44, p.9-16, 2003). We now extend the method using four microphones located at the apices of a regular tetrahedron to enable the method to estimate the elevation angle from the array plane as well. Furthermore, we introduce an idea for separate estimation of azimuth and elevation to reduce the computational loads.

This paper presents the constrained MUSIC (multiple signal classification) method with circular array. It is well known that the MUSIC method with a circular array is the high resolution technique to estimate the azimuthal and elevational directions-of-arrival (DOA). However, in general, the MUSIC method cannot distinguish multiple signals when the unknown source signal is highly coherent with the known source signal. By incorporating known source directions, the estimation of unknown source directions can be significantly improved. We propose the constrained MUSIC method with circular array and perform computer simulations to demonstrate the usefulness of the new approach.

In this paper, we will propose a recurrent RBF equalizer for non-linear channel with time-varying characteristics. In the conventional equalization method for that, its channel characteristic is estimated with FIR model, and the noise-free received signals are estimated with this estimate. Then the recurrent RBF equalizer's parameters are updated with this result. However, this method is not available lo nonlinear channel. In this paper, we firstly introduce MSB of the difference between the received signal and the estimate of noise-free received signal as the cost function to estimate all noise-free received signals, because this cost function is minimized when the noise-free received signal is equal to this estimate. Then, we estimate all estimates of noise-free received signal by minimizing this cost function using LMS algorithm, and the recurrent RBF equalizer is updated with these values.