Search

Speech processing algorithms, especially sound source localization (SSL), speech enhancement, and speaker tracking are considered to be the main fields in this application. Most speech processing algorithms require knowing the number of speakers for real implementation. In this article, a novel method for estimating the number of speakers is proposed based on the hive shaped nested microphone array (HNMA) by wavelet packet transform (WPT) and 2D sub-band adaptive steered response power (SB-2DASRP) with phase transform (PHAT) and maximum likelihood (ML) filters, and, finally, the agglomerative classification and elbow criteria for obtaining the number of speakers in near-field scenarios. The proposed HNMA is presented for aliasing and imaging elimination and preparing the proper signals for the speaker counting method. In the following, the Blackman–Tukey spectral estimation method is selected for detecting the proper frequency components of the recorded signal. The WPT is considered for smart sub-band processing by focusing on the frequency bins of the speech signal. In addition, the SRP method is implemented in 2D format and adaptively by ML and PHAT filters on the sub-band signals. The SB-2DASRP peak positions are extracted on various time frames based on the standard deviation (SD) criteria, and the final number of speakers is estimated by unsupervised agglomerative clustering and elbow criteria. The proposed HNMA-SB-2DASRP method is compared with the frequency-domain magnitude squared coherence (FD-MSC), i-vector probabilistic linear discriminant analysis (i-vector PLDA), ambisonics features of the correlational recurrent neural network (AF-CRNN), and speaker counting by density-based classification and clustering decision (SC-DCCD) algorithms on noisy and reverberant environments, which represents the superiority of the proposed method for real implementation.

Background Habitat structure strongly influences niche differentiation, facilitates predator avoidance, and drives species-specific foraging strategies of bats. Vegetation structure is also a strong driver of echolocation call characteristics. The fine-scale assessment of how bats utilise such structures in their natural habitat is instrumental in understanding how habitat composition shapes flight- and acoustic behaviour. However, it is notoriously difficult to study their species-habitat relationship in situ. Methods Here, we describe a methodology combining Light Detection and Ranging (LiDAR) to characterise three-dimensional vegetation structure and acoustic tracking to map bat behaviour. This makes it possible to study fine-scale use of habitat by bats, which is essential to understand spatial niche segregation in bats. Bats were acoustically tracked with microphone arrays and bat calls were classified to bat guild using automated identification. We did this in multiple LiDAR scanned vegetation plots in forest edge habitat. The datasets were spatially aligned to calculate the distance between bats’ positions and vegetation structures. Results Our results are a proof of concept of combining LiDAR with acoustic tracking. Although it entails challenges with combining mass-volumes of fine-scale bat movements and vegetation information, we show the feasibility and potential of combining those two methods through two case studies. The first one shows stereotyped flight patterns of pipistrelles around tree trunks, while the second one presents the distance that bats keep to the vegetation in the presence of artificial light. Conclusion By combining bat guild specific spatial behaviour with precise information on vegetation structure, the bat guild specific response to habitat characteristics can be studied in great detail. This opens up the possibility to address yet unanswered questions on bat behaviour, such as niche segregation or response to abiotic factors in interaction with natural vegetation. This combination of techniques can also pave the way for other applications linking movement patterns of other vocalizing animals and 3D space reconstruction.

Compact acoustic sensor arrays are nowadays popular devices for audio sensing due to minimum size and lightness characteristics that make it attractive for numerous applications such as human-computer interaction, acoustic scene analysis, teleconferencing systems, hearing aid, robotics, and bioacoustics. A compact array has a small inter-microphone distance that allows to reduce the spatial aliasing effect in the acoustic source localization problem, degrading however the spatial resolution. Thus, as a consequence, the capability of this class of sensors in the localization of multiple acoustic sources is rather limited. In this paper, we present an iterative diagonal unloading (IDU) beamforming suitable for the direction of arrivals (DOAs) estimation of concurrent multiple sources using compact microphone arrays. The method is based on the identification of the dominant signal, which corresponds to the largest peak in the pseudo-spectrum (i.e., the acoustic response map) computed by the DU beamformer, and on an iterative cancellation of the dominant signal from the covariance matrix to compute a new spatial pseudo-spectrum. All the pseudo-spectrum responses computed with the iterative procedure are summed to obtain a high resolution map of the acoustic scene. We analyze the DOAs estimation performance using compact uniform linear arrays (ULAs) and spherical microphone arrays (SMAs) with simulations and real acoustic data. The results show that the proposed IDU beamforming increases the spatial resolution allowing an effective localization of concurrent multiple sources.

Sound propagation along vertical and slanted paths through the near-ground atmosphere impacts detection and localization of low-altitude sound sources, such as small unmanned aerial vehicles, from ground-based microphone arrays. This article experimentally investigates the amplitude and phase fluctuations of acoustic signals propagating along such paths. The experiment involved nine microphones on three horizontal booms mounted at different heights to a 135-m meteorological tower at the National Wind Technology Center (Boulder, CO). A ground-based loudspeaker was placed at the base of the tower for vertical propagation or 56 m from the base of the tower for slanted propagation. Phasor scatter plots qualitatively characterize the amplitude and phase fluctuations of the received signals during different meteorological regimes. The measurements are also compared to a theory describing the log-amplitude and phase variances based on the spectrum of shear and buoyancy driven turbulence near the ground. Generally, the theory correctly predicts the measured log-amplitude variances, which are affected primarily by small-scale, isotropic turbulent eddies. However, the theory overpredicts the measured phase variances, which are affected primarily by large-scale, anisotropic, buoyantly driven eddies. Ground blocking of these large eddies likely explains the overprediction Refereed/Peer-reviewed

This article brings a breakthrough in understanding the computation of direction of arrival (DOA) of sound waves and represents an important step in the further introduction of spatial domain into environmental noise and acoustics measurements in general. Instead of improving the accuracy of beamforming, it makes more sense to increase the number of DOA calculations in the time domain and to use the statistical properties of environmental noise to localise the noise sources. We know from experience that two noise sources never have the same frequency characteristics, operating times, and spatial locations. This is the foundation on which the presented research stands and is validated with experiments. The high frequency sampling of 192 kHz allows a subwindow approach to time domain beamforming and the subdivision of microphone signals into time intervals of less than one millisecond. This allows statistical analysis of the immission directivity during the typically used time constant of 125 ms. The calculation of DOA in the subwindows shortens the time needed to evaluate the contribution of each noise source to the total sound pressure level. This leads to more data points (immission vectors) and lower statistical error in environmental noise measurements. In this paper, we propose associating the temporal dominant direction with the temporal sound pressure level within the subwindow and integrating the results to determine the immission directivity. Numerical simulations, experiments and practical application of the sub-windowing differential beamforming algorithm (SubW-DBA) method provided promising results.

Recently we presented a modulation-domain multichannel Kalman filtering (MKF) algorithm for speech enhancement, which jointly exploits the inter-frame modulation-domain temporal evolution of speech and the inter-channel spatial correlation to estimate the clean speech signal. The goal of speech enhancement is to suppress noise while keeping the speech undistorted, and a key problem is to achieve the best trade-off between speech distortion and noise reduction. In this paper, we extend the MKF by presenting a modulation-domain parametric MKF (PMKF) which includes a parameter that enables flexible control of the speech enhancement behaviour in each time-frequency (TF) bin. Based on the decomposition of the MKF cost function, a new cost function for PMKF is proposed, which uses the controlling parameter to weight the noise reduction and speech distortion terms. An optimal PMKF gain is derived using a minimum mean squared error (MMSE) criterion. We analyse the performance of the proposed MKF, and show its relationship to the speech distortion weighted multichannel Wiener filter (SDW-MWF). To evaluate the impact of the controlling parameter on speech enhancement performance, we further propose PMKF speech enhancement systems in which the controlling parameter is adaptively chosen in each TF bin. Experiments on a publicly available head-related impulse response (HRIR) database in different noisy and reverberant conditions demonstrate the effectiveness of the proposed method.

An analysis of the relationship between the bandwidth of acoustic signals and the required resolution of steered-response power phase transform (SRP-PHAT) maps used for sound source localization is presented. This relationship does not rely on the far-field assumption, nor does it depend on any specific array topology. The proposed analysis considers the computation of a SRP map as a process of sampling a set of generalized cross-correlation (GCC) functions, each one corresponding to a different microphone pair. From this approach, we derive a rule that relates GCC bandwidth with inter-microphone distance, resolution of the SRP map, and the potential position of the sound source relative to the array position. This rule is a sufficient condition for an aliasing-free calculation of the specified SRP-PHAT map. Simulation results show that limiting the bandwidth of the GCC according to such rule leads to significant reductions in sound source localization errors when sources are not in the immediate vicinity of the microphone array. These error reductions are more relevant for coarser resolutions of the SRP map, and they happen in both anechoic and reverberant environments.

In this article, we propose a new robust, stable, accurate, and efficient time-delay estimation scheme for the angle-of-arrival (AoA) ultrasonic indoor single-source localization systems. The AoA technique employing microphone arrays and ultrasonic signal emitting sources has turned out to be a promising alternative to the conventional distance-based ultrawideband systems for indoor localization. The crux of AoA estimation lies in the derivation of time-delay estimates, which is typically handled by advanced digital signal processing approaches based on cross-correlation and matching algorithms. The calculation of the channel response for each received signal and the conversion from the time to the time-delay domain consent our echo suppression delay estimator (ESDE) to apply echo suppression, in order to overcome the typical challenges introduced by the channel in indoor environments. Moreover, the optimization of the cross-correlation algorithm through oversampling the cross power spectrum and the curve-fitting algorithm allows to ensure subsample precision results. The performances of our developed AoA estimation system are evaluated by carrying out real-world experiments and compared with the state-of-the-art generalized cross correlators. Our ESDE algorithm is shown to outperform several generalized solutions.

In this paper, a study addressing the task of tracking multiple concurrent speakers in reverberant conditions is presented. Since both past and future observations can contribute to the current location estimate, we propose a forward-backward approach, which improves tracking accuracy by introducing near-future data to the estimator, in the cost of an additional short latency. Unlike classical target tracking, we apply a non-Bayesian approach, which does not make assumptions with respect to the target trajectories, except for assuming a realistic change in the parameters due to natural behaviour. The proposed method is based on the recursive expectation-maximization (REM) approach. The new method is dubbed forward-backward recursive expectation-maximization (FB-REM). The performance is demonstrated using an experimental study, where the tested scenarios involve both simulated and recorded signals, with typical reverberation levels and multiple moving sources. It is shown that the proposed algorithm outperforms the regular common causal (REM).

Convolutional Neural Networks (CNNs) are a popular choice for estimating Direction of Arrival (DoA) without explicitly estimating delays between multiple microphones. The CNN method first optimises unknown filter weights (of a CNN) by using observations and ground-truth directional information. This trained CNN is then used to predict incident directions given test observations. Most existing methods train using spectrally-flat random signals and test using speech. In this paper, which focuses on single source DoA estimation, we find that training with speech or music signals produces a relative improvement in DoA accuracy for a variety of audio classes across 16 acoustic conditions and 9 DoAs, amounting to an average improvement of around 17% and 19% respectively when compared to training with spectrally flat random signals. This improvement is also observed in scenarios in which the speech and music signals are synthesised using, for example, a Generative Adversarial Network (GAN). When the acoustic environments during test and training are similar and reverberant, training a CNN with speech outperforms Generalized Cross Correlation (GCC) methods by about 125%. When the test conditions are different, a CNN performs comparably. This paper takes a step towards answering open questions in the literature regarding the nature of the signals used during training, as well as the amount of data required for estimating DoA using CNNs.

Humanoid robots require to use microphone arrays to acquire speech signals from the human communication partner while suppressing noise, reverberation, and interferences. Unlike many other applications, microphone arrays in humanoid robots have to face the restrictions in size and geometry. To address these challenges, this paper presents an approach to differential beamforming with arbitrary planar array geometries. The major contributions of this work are as follows: (1) a method is presented to design differential beamformers, which works for regular geometries such as linear, circular, and concentric circular ones, as well as irregular geometries, as long as the sensors’ positions are given or can be measured; (2) fundamental requirements for the design of different orders of linear differential microphone arrays (DMAs), partially steerable DMAs, fully steerable DMAs, and robust DMAs are discussed; (3) the validity and limitations of the Jacobi-Anger expansion approximation is analyzed, where we discuss how to achieve an optimal approximation by properly choosing the reference point; and (4) we show how to design an Nth-order DMA with 2N microphones using the Jacobi-Anger expansion.

Acoustic beamforming is routinely used to improve the SNR of the received signal in applications such as hearing aids, robot audition, augmented reality, teleconferencing, source localisation and source tracking. The beamformer can be made adaptive by using an estimate of the time-varying noise covariance matrix in the spectral domain to determine an optimised beam pattern in each frequency bin that is specific to the acoustic environment and that can respond to temporal changes in it. However, robust estimation of the noise covariance matrix remains a challenging task especially in non-stationary acoustic environments. This paper presents a compact model of the signal covariance matrix that is defined by a small number of parameters whose values can be reliably estimated. The model leads to a robust estimate of the noise covariance matrix which can, in turn, be used to construct a beamformer. The performance of beamformers designed using this approach is evaluated for a spherical microphone array under a range of conditions using both simulated and measured room impulse responses. The proposed approach demonstrates consistent gains in intelligibility and perceptual quality metrics compared to the static and adaptive beamformers used as baselines.

Sound samples (.wav), CAD models (.stl), COMSOL model (.mph) and array response simulations (.mat) used to evaluate methods presented in the paper:Parametric Ambisonic Encoding of Arbitrary Microphone Arrays Abstract This article proposes a parametric signal-dependent method for the task of encoding microphone array signals into Ambisonic signals. The proposed method is presented and evaluated in the context of encoding a simulated seven-sensor microphone array, which is mounted on an augmented reality headset device. Given the inherent flexibility of the Ambisonics format, and its popularity within the context of such devices, this array configuration represents a potential future use case for Ambisonic recording. However, due to its irregular geometry and non-uniform sensor placement, conventional linear Ambisonic encoding is particularly limited. The primary aims of the proposed method are to obtain Ambisonic signals over a wider frequency band-width, and at a higher spatial resolution, than would otherwise be possible through conventional linear encoding. The proposed method is based on a multi-source sound-field model and employs spatial filtering to divide the captured sound-field into its individual source and directional ambient components, which are subsequently encoded into the Ambisonics format at an arbitrary order. It is demonstrated through both objective and subjective evaluations that the proposed parametric method outperforms conventional linear encoding in the majority of cases.

Devices for speech communication operated in handsfree mode offer a very natural way of human communication, because the user can move freely in relation to the device. However, the signal-to-noise ratio (SNR) at the microphones of the device is typically low due to propagation loss, reverberation and interfering sounds such as echo or environmental noise. This requires appropriate front-end signal processing (FESP) to enhance the desired speech signal. Nowadays more than one communication device is typically present in a smart home environment or in a conference meeting room. A beamformer (BF) can use the microphones of multiple devices to compensate for the low initial SNR, if all microphone positions are known. For estimating these positions the novel orthogonal geometric projection (OGP) approach is proposed. OGP needs only two acoustic events like speech or hand claps for estimation and thus puts very low effort on the user. For allowing a full-duplex speech communication, one acoustic echo canceller (AEC) per microphone channel is usually employed prior to the BF, which results in a high complexity. Therefore, change prediction (ChaP) is proposed that enables the use of a single AEC after the BF. By collecting information on the acoustic system over time, ChaP can facilitate the adaptation of the AEC such that this low-complexity single-AEC configuration can approach the performance of the high-complexity multi-AEC variant. Conventional linear AEC is actually insufficient for mobile consumer devices, because their low-cost loudspeakers and amplifiers turned up to a high volume show a significant nonlinear behavior. The novel dual-stage multi-channel Kalman (DualStage-MCK) algorithm also compensates for these nonlinear effects and does not suffer from limited modelling capabilities, slow tracking or high computational complexity, which are typical drawbacks of state-of-the-art solutions. The performance of the proposed solutions is evaluated in typical use cases and on realistic test data that includes device-specific acoustic shadowing and nonlinear effects acquired from specifically manufactured tablet, smart speaker and smartphone mockups.

This paper presents a new dataset of measured multichannel room impulse responses (RIRs) named dEchorate. It includes annotations of early echo timings and 3D positions of microphones, real sources, and image sources under different wall configurations in a cuboid room. These data provide a tool for benchmarking recent methods in echo-aware speech enhancement, room geometry estimation, RIR estimation, acoustic echo retrieval, microphone calibration, echo labeling, and reflector position estimation. The dataset is provided with software utilities to easily access, manipulate, and visualize the data as well as baseline methods for echo-related tasks.

More and more devices we use in our daily life are embedded with one or more microphones so that they can be voice controlled. Put together, these devices can form a so-called ad-hoc microphone array (AHMA). A speech enhancement step is often applied on the recorded signals to optimise the execution of the voice commands. To this effect, AHMAs are of high interest because of their flexible usage, their wide spatial coverage and the diversity of their recordings. However, it is challenging to exploit the potential of mbox{AHMAs} because devices that compose them may move and have a limited power and bandwidth capacity. Because of these limits, the speech enhancement solutions deployed in ``classic'' microphone arrays, relying on a fusion center and high processing loads, cannot be afforded.This thesis combines the modelling power of deep neural networks (DNNs) with the flexibility of use of AHMAs. To this end, we introduce a distributed speech enhancement system, which does not rely on a fusion center. So-called compressed signals are sent among the nodes and convey the spatial information recorded by the whole AHMA, while reducing the bandwidth requirements. DNNs are used to estimate the coefficients of a multichannel Wiener filter. We conduct an empirical analysis of this sytem, both on synthesized and real data, in order to validate its efficiency and to highlight the benefits of jointly using DNNs and distributed speech enhancement algorithms. We show that our system performs comparatively well compared with a state-of-the-art solution, while being more flexible and significantly reducing the computation cost.Besides, we develop our solution to adapt it to the typical usage conditions of mbox{AHMAs}. We study its behaviour when the number of devices in the AHMA varies. We introduce and compare a spatial attention mechanism and a self-attention mechanism. Both mechanisms make our system robust to a varying number of devices. We show that the weights of the self-attention mechanism reveal the utility of the information carried by each signal.We also analyse our system when the signals recorded by different devices are not synchronised. We propose a solution to improve its performance in such conditions by introducing a temporal attention mechanism. We show that this mechanism can help estimating the sampling time offset between the several devices of the AHMA.Lastly, we show that our system is also efficient for source separation. It can efficiently process the spatial information recorded by the whole AHMA in a typical meeting scenario and alleviate the needs of a complex DNN architecture., Un grand nombre d’appareils que nous utilisons au quotidien embarque un ou plusieurs microphones afin de rendre possible leur utilisation par commande vocale. Le réseau de microphones que l’on peut former avec ces appareils est ce qu’on appelle une antenne acoustique ad-hoc (AAAH). Une étape de rehaussement de la parole est souvent appliquée afin d’optimiser l’exécution des commandes vocales. Pour cela, les AAAH, de par leur flexibilité d’utilisation, leur large étendue spatiale et la diversité de leurs enregistrements, offrent un grand potentiel. Ce potentiel est néanmoins difficilement exploitable à cause de la mobilité des appareils, leur faible puissance et les contraintes en bande passante. Ceslimites empêchent d’utiliser les algorithmes de rehaussement de la parole « classiques » qui reposent sur un nœud de fusion et requièrent de fortes puissances de calcul.Cette thèse propose de rallier le domaine de l’apprentissage profond à celui des AAAH, en conciliant la puissance de modélisation des réseaux de neurones (RN) à la flexibilité d’utilisation des AAAH. Pour cela, nous présentons un système distribué de rehaussement de la parole. Il est distribué en cela que la contrainte d’un centre de fusion est levée. Des signaux dits compressés, échangés entre les nœuds, permettent de véhiculer l’information spatiale tout en réduisant la consommation en bande passante. Des RN sont utilisés afin d’estimer les coefficients d’un filtre de Wiener multicanal. Une analyse empirique détaillée de ce système est conduite à la fois sur données synthétiques et sur données réelles afin de valider son efficacité et de mettre en évidence l’intérêt d’utiliser conjointement des RN et des algorithmes distribués classiques de rehaussement de la parole. Nous montrons ainsi que notre système obtient des performances équivalentes à celles de l’état de l’art, tout en étant plus flexible et en réduisant significativement la complexité algorithmique.Par ailleurs, nous développons notre solution pour l’adapter à des conditions d’utilisation propres aux AAAH. Nous étudions son comportement lorsque le nombre d’appareils de l’AAAH varie, et nous comparons l’influence de deux mécanismes d’attention, l’un d’attention spatiale et l’autre d’auto-attention. Les deux mécanismes d’attention rendent notre système résilient à un nombre variable d’appareils et les poids du mécanisme d’auto-attention révèlent l’utilité de l’information convoyée par chaque signal. Nous analysons également le comportement de notre système lorsque les signaux des différents appareils sont désynchronisés. Nous proposons une solution pour améliorer les performances de notre système en conditions asynchrones, en présentant un autre mécanisme d’attention. Nous montrons que ce mécanisme d’attention permet de retrouver un ordre de grandeur du décalage d’horloge entre les appareils d’une AAAH. Enfin, nous montrons que notre système est une solution viable pour la séparation de sources de parole. Même avec des RN d’architecture simple, il est capable d’exploiter efficacement l’information spatiale enregistrée par tous les appareils d’une AAAH dans une configuration typique de réunion.

The superdirective beamformer, while attractive for processing broadband acoustic signals, often suffers from the problem of white noise amplification. So, its application requires well-designed acoustic arrays with sensors of extremely low self-noise level, which is difficult if not impossible to attain. In this paper, a new binaural superdirective beamformer is proposed, which is divided into two sub-beamformers. Based on studies and facts in psychoacoustics, these two filters are designed in such a way that they are orthogonal to each other to make the white noise components in the binaural beamforming outputs incoherent while maximizing the output interaural coherence of the diffuse noise, which is important for the brain to localize the sound source of interest. As a result, the signal of interest in the binaural superdirective beamformer&rsquo, s outputs is in phase but the white noise components in the outputs are random phase, so the human auditory system can better separate the acoustic signal of interest from white noise by listening to the outputs of the proposed approach. Experimental results show that the derived binaural superdirective beamformer is superior to its conventional monaural counterpart.

Noise pollution is one of the major health risks in urban life. The approach to measurement and identification of noise sources needs to be improved and enhanced to reduce high costs. Long measurement times and the need for expensive equipment and trained personnel must be automated. Simplifying the identification of main noise sources and excluding residual and background noise allows more effective measures. By spatially filtering the acoustic scene and combining unsupervised learning with psychoacoustic features, this paper presents a prototype system capable of automated calculation of the contribution of individual noise sources to the total noise level. Pilot measurements were performed at three different locations in the city of Ljubljana, Slovenia. Equivalent sound pressure levels obtained with the device were compared to the results obtained by manually marking individual parts of each of the three measurements. The proposed approach correctly identified the main noise sources in the vicinity of the measurement points.

Direction of arrival (DoA) estimation for sound source localization is increasingly prevalent in modern devices. In this paper, we explore a polynomial extension to the multiple signal classification (MUSIC) algorithm, spatio-spectral polynomial (SSP)-MUSIC, and evaluate its performance when using speech sound sources. In addition, we also propose three essential enhancements for SSP-MUSIC to work with noisy reverberant audio data. This paper includes an analysis of SSP-MUSIC using speech signals in a simulated room for different noise and reverberation conditions and the first task of the LOCATA challenge. We show that SSP-MUSIC is more robust to noise and reverberation compared to independent frequency bin (IFB) approaches and improvements can be seen for single sound source localization at signal-to-noise ratios (SNRs) below 5 dB and reverberation times (T60s) larger than 0.7 s.

Many applications rely on MEMS microphone arrays for locating sound sources prior to their execution. Those applications not only are executed under real-time constraints but also are often embedded on low-power devices. These environments become challenging when increasing the number of microphones or requiring dynamic responses. Field-Programmable Gate Arrays (FPGAs) are usually chosen due to their flexibility and computational power. This work intends to guide the design of reconfigurable acoustic beamforming architectures, which are not only able to accurately determine the sound Direction-Of-Arrival (DoA) but also capable to satisfy the most demanding applications in terms of power efficiency. Design considerations of the required operations performing the sound location are discussed and analysed in order to facilitate the elaboration of reconfigurable acoustic beamforming architectures. Performance strategies are proposed and evaluated based on the characteristics of the presented architecture. This power-efficient architecture is compared to a different architecture prioritizing performance in order to reveal the unavoidable design trade-offs.

Audio source separation is essential for many applications such as hearing aids, telecommunications, and robot audition. Subspace decomposition approaches using polynomial matrix eigenvalue decomposition (PEVD) algorithms applied to the microphone signals, or lower-dimension eigenbeams for spherical microphone arrays, are effective for speech enhancement. In this work, we extend the work from speech enhancement and propose a PEVD subspace algorithm that uses eigenbeams for source separation. The proposed PEVD-based source separation approach performs comparably with state-of-the-art algorithms, such as those based on independent component analysis (ICA) and multi-channel non-negative matrix factorization (MNMF). Informal listening examples also indicate that our method does not introduce any audible artifacts.

В даній роботі було проведено аналіз наявної інформації на тему використання детектору голосової активності для знаходження дефектів в масиві мікрофонів. На основі цього було зроблено висновок, що технології з використанням голосових помічників доволі розповсюджена і продовжує поширюватися, так як це дозволяє користувачам зручно взаємодіяти з системами без використанням прямого вводу даних. Проте через кількість даних пристроїв з голосовим способом вводу з’являється проблема їх обслуговування і перевірки. Для дослідження були вивчені існуючі кодери, які використовуються в мобільному зв’язку, їх властивості, методи обробки, принципи роботи і недоліки. Для відтворення роботи детектора використовувався програмний пакет MATLAB. Була представлена робота із програмним доповненням для системи розумного будинку Amazon тм Dot, як приклад використання програмного забезпечення для роботи із одним елементом масиву мікрофонів. In this thesis, an analysis of available information on the use of a voice activity detector to find defects in the array of microphones was performed. Based on this, it was concluded that technology using voice assistants is quite common and continues to spread, as it allows users to easily interact with systems without the use of direct data input. However, due to the number of data devices with voice input, there is a problem with their maintenance and verification. This study examined existing coders used in mobile communications, their properties, processing methods, principles of operation and shortcomings. MATLAB software package was used to reproduce the detector. Work with the software add-on for the Amazon тм Dot smart home system was presented as an example of using software to work with one element of an array of microphones.

An array with MEMS microphones can distinguish individual noise sources in an environment through spatial filtering. Its effectiveness depends on the variations in microphone sensitivity and phase. Quantification of these variations is valuable, because it enables assessment and optimization of array performance. This is particularly important if the measurements are to be used for enforcement of noise regulations.Nominal microphone sensitivity and phase are manufacturer-specified, but the distribution (histogram) around these values is not. Hence, this work demonstrates a free-field comparison method for measuring these variations in a batch of arrays. We also provide the histograms at 1 kHz for a sample population of 8384 Knowles SPH0641LM4H-1 MEMS microphones (131 arrays of 64 microphones). The histograms follow t-distributions, resulting in 95% confidence intervals of ±0.39dB for sensitivity and ±0.82◦ for phase. Finally, we illustrate that delay-and-sum beamforming with these microphones results in a Gumbel-distributed gain with −0.13/+0.10dB 95% confidence interval.

We have been exploring the integration of sparse recovery methods into the ray space transform over the past years and now demonstrate the potential and benefits of beamforming and upscaling signals in the integrated ray space and sparse recovery domain. A primary advantage of the ray space approach derives from its robust ability to integrate information from multiple arrays and viewpoints. Nonetheless, for a given viewpoint, the ray space technique requires a dense array that can be divided into sub-arrays enabling the plenacoustic approach to signal processing. In this work, we explore a method to upscale an array beyond the limits imposed by the inter-microphone distances associated with the array and the concomitant spatial aliasing. In other words, sparse recovery enables one to synthesize or interpolate signals corresponding to an array with a greater number of microphones with a smaller inter-microphone distance. A critical issue is whether or not this interpolative synthesis actually improves array signal processing. This work shows that upscaling signals in the integrated ray space and sparse recovery domain can improve both source localization and separation.

Submitted to ICASSP 2020; Speech separation with several speakers is a challenging task because of the non-stationarity of the speech and the strong signal similarity between interferent sources. Current state-of-the-art solutions can separate well the different sources using sophisticated deep neural networks which are very tedious to train. When several microphones are available, spatial information can be exploited to design much simpler algorithms to discriminate speakers. We propose a distributed algorithm that can process spatial information in a spatially unconstrained microphone array. The algorithm relies on a convolutional recurrent neural network that can exploit the signal diversity from the distributed nodes. In a typical case of a meeting room, this algorithm can capture an estimate of each source in a first step and propagate it over the microphone array in order to increase the separation performance in a second step. We show that this approach performs even better when the number of sources and nodes increases. We also study the influence of a mismatch in the number of sources between the training and testing conditions.

The literature is rich with techniques for the design of small-size Differential Microphone Arrays (DMAs), known for their almost frequency-invariant beampatterns and low computational cost. Few works, instead, discuss the properties of beamformers based on multiple DMA units. In this paper, we consider arbitrarily shaped planar arrays of DMA units. In turn, each DMA unit is a first-order continuously-steerable differential microphone characterized by an arbitrary configuration of omnidirectional sensors and a symmetric beampattern. We present a beamforming technique that, assumed all the DMA units to steer identical beams in the same direction, allows us to approach the behavior of a Delay-And-Sum beamformer or Super-Directive beamformer by solely varying a single scalar parameter. Efficient implementations of the proposed beamformers can be developed by taking into account that, for a wide range of frequencies, the values of such a parameter are practically invariant with respect to the geometry of the array.

Binaural room auralization involves Binaural Room Impulse Responses (BRIRs). Dynamic binaural synthesis (i.e., head-tracked presentation) requires BRIRs for multiple head poses. Artificial heads can be used to measure BRIRs, but BRIR modeling from microphone array room impulse responses (RIRs) is becoming popular since personalized BRIRs can be obtained for any head pose with low extra effort. We present a novel framework for estimating a binaural signal from microphone array signals, using causal Wiener filtering and polynomial matrix formalism. The formulation places no explicit constraints on the geometry of the microphone array and enables directional weighting of the estimation error. A microphone noise model is used for regularization and to balance filter performance and noise gain. A complete procedure for BRIR modeling from microphone array RIRs is also presented, employing the proposed Wiener filtering framework. An application example illustrates the modeling procedure using a 19-channel spherical microphone array. Direct and reflected sound segments are modeled separately. The modeled BRIRs are compared to measured BRIRs and are shown to be waveform-accurate up to at least 1.5 kHz. At higher frequencies, correct statistical properties of diffuse sound field components are aimed for. A listening test indicates small perceptual differences to measured BRIRs. The presented method facilitates fast BRIR data set acquisition for use in dynamic binaural synthesis and is a viable alternative to Ambisonics-based binaural room auralization.

Supervised learning methods are a powerful tool for direction of arrival (DOA) estimation because they can cope with adverse conditions where simplified models fail. In this work, we consider a previously proposed convolutional neural network (CNN) approach that estimates the DOAs for multiple sources from the phase spectra of the microphones. For speech, specifically, the approach was shown to work well even when trained entirely on synthetically generated data. However, as each frame is processed separately, temporal context cannot be taken into account. This prevents the exploitation of interframe signal correlations, and the fact that DOAs do not change arbitrarily over time. We therefore consider two different extensions of the CNN: the integration of a long short-term memory (LSTM) layer, or of a temporal convolutional network (TCN). In order to accommodate the incorporation of temporal context, the training data generation framework needs to be adjusted. To obtain an easily parameterizable model, we propose to employ Markov chains to realize a gradual evolution of the source activity at different times, frequencies, and directions, throughout a training sequence. A thorough evaluation demonstrates that the proposed configuration for generating training data is suitable for the tasks of single-, and multi-talker localization. In particular, we note that with temporal context, it is important to use speech, or realistic signals in general, for the sources. Experiments with recorded impulse responses and noise reveal that the CNN with the LSTM extension outperforms all other considered approaches, including the plain CNN, and the TCN extension.

The acquisition of the spatio-temporal characteristics of a sound field over a large volume of space is experimentally challenging, as a large number of transducers is required to sample the sound field. Sound field reconstruction methods are a resourceful approach, as they enable the interpolation and extrapolation of the sound field from a limited number of observed data. In this study we examine the spatio-temporal and spatio-spectral reconstruction of the sound field in a room from distributed measurements of the sound pressure. Specifically, a variational Gaussian process regression model is formulated, using time-domain anisotropic kernels to reconstruct the direct sound and early reflections, and frequency-domain isotropic kernels for reconstructing the late reverberant field. The proposed methodology is compared experimentally to classical regression models based on plane wave decompositions, which are widely used in sound field reconstruction in enclosures due to their simplicity and accuracy.

This paper presents dEchorate: a new database of measured multichannel Room Impulse Responses (RIRs) including annotations of early echo timings and 3D positions of microphones, real sources and image sources under different wall configurations in a cuboid room. These data provide a tool for benchmarking recent methods in echo-aware speech enhancement, room geometry estimation, RIR estimation, acoustic echo retrieval, microphone calibration, echo labeling and reflectors estimation. The database is accompanied with software utilities to easily access, manipulate and visualize the data as well as baseline methods for echo-related tasks.

This paper proposes a neural network based system for multichannel speech enhancement and dereverberation. Speech recorded indoors by a far field microphone, is invariably degraded by noise and reflections. Recent single channel enhancement systems have improved denoising performance, but do not reduce reverberation, which also reduces speech quality and intelligibility. To address this, we propose a deep complex convolution recurrent network (DCCRN) based multi-channel system, with integrated minimum power distortionless response (MPDR) beamformer and weighted prediction error (WPE) preprocessing. PESQ and STOI performance is evaluated on a test set of room impulse responses and noise samples recorded by the same setup. The proposed system shows a statistically significant improvement (p « 0.05) over competitive systems.

The bearings of rotating machinery often fail, leading to unforeseen downtime of large machines in industrial plants. Therefore, condition monitoring can be a powerful tool to aid in the quick identification of these faults and make it possible to plan maintenance before the fault becomes too drastic, reducing downtime and cost. Predictive maintenance is often based on information gathered from accelerometers. However, these sensors are contact-based, making them less attractive for use in an industrial plant and more prone to breakage. In this paper, condition monitoring based on ultrasound is researched, where non-invasive sensors are used to record the noise originating from different defects of the Machinery Fault Simulator. The acoustic data are recorded using a sparse microphone array in a lab environment. The same array was used to record real spatial noise in a fully operational plant which was later added to the acoustic data containing the different defects with a variety of Signal To Noise ratios. In this paper, we compare the classification results of the noisy acoustic data of only one microphone to the beamformed acoustic data. We do this to investigate how beamforming could improve the classification process in an ultrasound condition-monitoring application in a real industrial plant.

International audience; Spherical microphone arrays are widespread nowadays, especially with the advent of techniques for decomposing acoustic fields on the basis of spherical harmonics (i.e. Ambisonics). However, in these approaches, the corresponding arrays require a quadratically increasing number of microphones as the degree of the components to be estimated increases. However, for some applications, it is unnecessary to represent the whole solid angle of the sound field from the array point of view: for instance as soon as the array is placed near the boundaries of the acoustic field of study (i.e. walls in a room). To tackle this issue, one proposes a geometry which uses the acoustic rigid boundary condition at room walls to decrease the number of required microphones. With one eighth of a sphere array placed in the corner of a room, only certain spherical harmonics are required for the sound pressure field decomposition, and the microphone number can drop up to 8 times lower when compared to the full spherical array configuration. A microphone array prototype having the geometry of one eighth of a sphere is described, built, and evaluated in a rigid bounded eighth of an anechoic space.

Many animal species, including many species of bats, exhibit collective behavior where groups of individuals coordinate their motion. Bats are unique among these animals in that they use the active sensing mechanism of echolocation as their primary means of navigation. Due to their use of echolocation in large groups, bats run the risk of signal interference from sonar jamming. However, several species of bats have developed strategies to prevent interference, which may lead to different behavior when flying with conspecifics than when flying alone. This study seeks to explore the role of this acoustic sensing on the behavior of bat pairs flying together. Field data from a maternity colony of gray bats (Myotis grisescens) were collected using an array of cameras and microphones. These data were analyzed using the information theoretic measure of transfer entropy in order to quantify the interaction between pairs of bats and to determine the effect echolocation calls have on this interaction. This study expands on previous work that only computed information theoretic measures on the 3D position of bats without echolocation calls or that looked at the echolocation calls without using information theoretic analyses. Results show that there is evidence of information transfer between bats flying in pairs when time series for the speed of the bats and their turning behavior are used in the analysis. Unidirectional information transfer was found in some subsets of the data which could be evidence of a leader&ndash, follower interaction.

This paper proposes a novel method to detect and localize disturbances in dynamical systems using non-collocated acoustic arrays by processing their data such that established artificial intelligence techniques can be used. Methods are proposed to compress time-domain based separate microphone data into frequency-domain based images. Furthermore, a data augmentation solution is given to significantly reduce the amount of required training data by using augmentation in the time domain. The use of artificial intelligence in the field of condition monitoring and diagnostics is becoming increasingly popular. However, the vast majority of work is based on sensors collocated in the machines. Therefore, the possibility of using acoustic arrays as a non-intrusive contactless sensor is emerging. The goal of this work is to develop data processing methods that allow for analysis of acoustic array images of vibration patterns. These methods have been demonstrated on a newly created experimental dataset where the a vibrating plate is measured using a 32 by 32 microphone array. On this plate, a disturbance mass placed in different positions to modify the dynamic properties of the system. The experimental results show that the proposed methods are able to determine the position of the disturbance mass even with low amounts of training data. They show to be promising for applications where space-frequency information is of essence.

Multichannel processing is widely used for speech enhancement but several limitations appear when trying to deploy these solutions to the real-world. Distributed sensor arrays that consider several devices with a few microphones is a viable alternative that allows for exploiting the multiple devices equipped with microphones that we are using in our everyday life. In this context, we propose to extend the distributed adaptive node-specific signal estimation approach to a neural networks framework. At each node, a local filtering is performed to send one signal to the other nodes where a mask is estimated by a neural network in order to compute a global multi-channel Wiener filter. In an array of two nodes, we show that this additional signal can be efficiently taken into account to predict the masks and leads to better speech enhancement performances than when the mask estimation relies only on the local signals., Submitted to ICASSP2020

A novel method is proposed for generic target tracking by audio measurements from a microphone array. To cope with noisy environments characterized by persistent and high energy interfering sources, a classification map (CM) based on spectral signatures is calculated by means of a machine learning algorithm. Next, the CM is combined with the acoustic map, describing the spatial distribution of sound energy, in order to obtain a cleaned joint map in which contributions from the disturbing sources are removed. A likelihood function is derived from this map and fed to a particle filter yielding the target location estimation on the acoustic image. The method is tested on two real environments, addressing both speaker and vehicle tracking. The comparison with a couple of trackers, relying on the acoustic map only, shows a sharp improvement in performance, paving the way to the application of audio tracking in real challenging environments.

Multiple simultaneous sound source localization (SSL) is one of the most important applications in the speech signal processing. The one-step algorithms with the advantage of low computational complexity (and low accuracy), and the two-step methods with high accuracy (and high computational complexity) are proposed for multiple SSL. In this article, a combination of one-step-based method based on the generalized eigenvalue decomposition (GEVD), and a two-step-based method based on the adaptive generalized cross-correlation (GCC) by using the phase transform/maximum likelihood (PHAT/ML) filters along with a novel T-shaped circular distributed microphone array (TCDMA) is proposed for 3D multiple simultaneous SSL. In addition, the low computational complexity advantage of the GCC algorithm is considered in combination with the high accuracy of the GEVD method by using the distributed microphone array to eliminate spatial aliasing and thus obtain more appropriate information. The proposed T-shaped circular distributed microphone array-based adaptive GEVD and GCC-PHAT/ML algorithms (TCDMA-AGGPM) is compared with hierarchical grid refinement (HiGRID), temporal extension of multiple response model of sparse Bayesian learning with spherical harmonic (SH) extension (SH-TMSBL), sound field morphological component analysis (SF-MCA), and time-frequency mixture weight Bayesian nonparametric acoustical holography beamforming (TF-MW-BNP-AHB) methods based on the mean absolute estimation error (MAEE) criteria in noisy and reverberant environments on simulated and real data. The superiority of the proposed method is presented by showing the high accuracy and low computational complexity for 3D multiple simultaneous SSL.

This paper explores the advantages of combining asynchronous microphone array measurements for acoustic source mapping in two and three–dimensional applications. Four different approaches are considered, three consisting of the combination of the source maps (arithmetic mean, geometric mean, and minimum value), and a fourth one obtained by combining the cross–spectral matrices of all the asynchronous measurements into a larger matrix and applying beamforming with it. Both synthetic and experimental test cases convey enhanced results concerning the single measurement baseline, especially by reducing the spurious sidelobes. Two aeroacoustic experiments are considered, the first features a distributed sound source over a flat plate model tested in a wind tunnel and the second features a hovering drone with multiple sound sources, representing typical challenges for 2D and 3D source localization, respectively. For the 2D source mapping configuration, the sidelobe level was reduced up to 5 dB with respect to the baseline, while maintaining a similar beamwidth of the main lobe. For the 3D test cases, the approaches enable volumetric source mapping capabilities, even when planar microphone arrays are considered, commonly available in typical test facilities. Overall, the minimum value approach presents the best performance for all cases in terms of reduced main lobe beamwidth and sidelobe levels.

Aeroacoustic measurements performed by flush-mounted microphone arrays on the walls of closed-section wind tunnels are contaminated by the hydrodynamic pressure fluctuations of the wall's boundary layer. This study evaluates three different microphone cavity geometries for mitigating this issue. Their improvement to the signal-to-noise ratio (SNR) and the accuracy of their acoustic imaging results are compared to a flush-mounted microphone array. The four geometries include: (1) an array of flush-mounted microphones as the baseline, (2) a cylindrical hard-plastic cavity with a countersink, (3) a conical cavity made of melamine acoustic absorbing foam, and (4) a conical cavity with star-shaped protrusions, also made of melamine. The three arrays with cavities were covered with a steel-wire cloth to reduce the boundary layer fluctuations at the microphone while the baseline array was uncovered. Two sound sources were tested in an aeroacoustic wind tunnel for assessing the performance of the different cavities: a speaker placed outside the flow and a distributed sound source generated by a flat plate inside of the flow. When using conventional frequency domain beamforming, both cavities made of melamine offer up to a 30 dB increase in SNR with respect to the flush-mounted case, followed by the hard-walled cavity with up to a 20 dB increase. This is a 20 dB improvement when compared to the single microphone cases. The melamine cavities also provide cleaner acoustic source maps and accurate spectral estimations for a wider frequency range. The effect of cavity placement and geometry on the coherence, which affects the beamforming analysis of the acoustic signal was negligible for all cases. Distributed sound source measurements using the three arrays agreed with predictions using the Brooks, Pope, and Marcolini (BPM) model, showing that the cavities could detect vortex shedding that was undetectable by the flush array.

International audience; The problem of localizing and quantifying acoustic sources from a set of acoustic measurements has been addressed, in the last decades, by a huge number of scientists, from different communities (signal processing, mechanics, physics) and in various application fields (underwater, aero, or vibro acoustics). This led to the production of a substantial amount of literature on the subject, together with the development of many methods, specifically adapted and optimized for each configuration and application field, the variety and sophistication of proposed algorithms being sustained by the constant increase in computational and measurement capabilities. The counterpart of this prolific research is that it is quite tricky to get a clear global scheme of the state of the art. The aim of the present work is to make an attempt in this direction, by proposing a unified formalism for different well known imaging techniques, from identification methods (acoustic holography, equivalent sources, Bayesian focusing, Generalized inverse beamforming…) to beamforming deconvolution approaches (DAMAS, CLEAN). The hypothesis, advantages and pitfalls of each approach will be established from a theoretical point of view, with a particular effort in trying to separate differences in the problem definition (a priori information, main assumptions) and in the algorithms used to find the solution. Numerical simulations will be proposed for different source configurations (coherent/incoherent/extended/sparse distributions), and an experimental illustration on a supersonic jet will be finally discussed.

We address the problem of localizing and tracking alternating (moving or stationary) talkers using microphone arrays in a room environment. One of the main challenges is the frequent (and possibly abrupt) change of talker positions which requires the algorithm to capture the active talker rapidly. In addition, the presence of interference, background noise and room reverberation degrades the tracking performance. We propose a new algorithm that jointly exploits the advantages of the particle filter (PF) and particle swarm intelligence. The PF is used as a general tracking framework which incorporates a proposed alternating source-dynamic model for recursive estimation of talker position. Unlike the conventional PF where particles operate independently in the particle sampling stage, the use of swarm intelligence allows particles to interact with each other, thereby improving convergence toward the active talker location. In addition, the memory mechanism in swarm intelligence allows particles to remain at their previous best-fit state estimate when signals are corrupted by interference, noise and/or reverberation. Simulations and experiments were conducted to demonstrate the effectiveness of the proposed algorithm. Accepted version

In this article we propose a methodology for the reconstruction of sound fields in arbitrary locations based on the signals acquired by a spatial distribution of compact microphone arrays (virtual miking). The proposed method is suitable for operating in reverberant environments, thanks to a two-stage analysis process, the former of which aims at separating the direct and the diffuse components of the sound field. The method that we propose is inherently parametric, as the sources of the acoustic scene are characterized by parameters describing location and directivity (spherical harmonics expansion), which are extracted from the exterior model of the direct component of the sound field. Once the parameters of the sources are extracted, the direct sound field at an arbitrary location is reconstructed. The diffuse component is reconstructed from the joint knowledge of the diffuse component at the locations of the distributed microphone arrays, under the assumption of isotropic behavior. Results show that the proposed technique is able to analyze the sound field and reconstruct the parameters of the sources that are active in the scene. In addition, the synthesis of the signals at the virtual microphone locations turns out to accurately match (in terms of spatial cues) the actual sound field, as measured by a microphone places in the desired location.

Multi-microphone speech enhancement methods typically require a reference position with respect to which the target signal is estimated. Often, this reference position is arbitrarily chosen as one of the reference microphones. However, it has been shown that the choice of the reference microphone can have a significant impact on the final noise reduction performance. In this paper, we therefore theoretically analyze the impact of selecting a reference on the noise reduction performance with near-end noise being taken into account. Following the generalized eigenvalue decomposition (GEVD) based optimal variable span filtering framework, we find that for any linear beamformer, the output signal-to-noise ratio (SNR) taking both the near-end and far-end noise into account is reference dependent. Only when the near-end noise is neglected, the output SNR of rank-1 beamformers does not depend on the reference position. However, in general for rank- $r$ beamformers with $r>1$ (e.g., the multichannel Wiener filter) the performance does depend on the reference position. Based on these, we propose an optimal algorithm for microphone reference selection that maximizes the output SNR. In addition, we propose a lower-complexity algorithm that is still optimal for rank-1 beamformers, but sub-optimal for the general $r>1$ rank beamformers. Experiments using a simulated microphone array validate the effectiveness of both proposed methods and show that in terms of quality, several dB can be gained by selecting the proper reference microphone.

International audience; When evaluating or optimising the accuracy of sound field synthesis, it is commonplace to only consider the error in the pressure field that is reproduced. While this is justifiable perceptually ? pressure is what we hear ? it neglects error in particle velocity. Acoustic energy density includes both these quantities, so is less sensitive to locations where one or other quantities is zero and has been shown to be outperform pressure in some adaptive noise control applications. Acoustic cross-energy density was suggested in 2014 as a measure of common energy between waves, generalising spatially the notion of cross-covariance between signals. It?s counterpart quantity, acoustic cross-intensity, is a measure of common energy flux. The two are connected by an energy flux relation, meaning similar information can be computed over a domain or a boundary. Cross-intensity generalises several double-layer microphone array designs that have previously been limited to planar, cylindrical or spherical geometries. Applications of it since published includes a boundary integral equation that is based on wave reflection and satisfies reciprocity, and encoding of a sound field computed by boundary element method to allow auralisation. This paper will study a generalisation of the ordinary coherence function based on acoustic cross-energy, which is computed spatially for waves. It is shown to be a bounded metric that measures wave similarity, and hence can be used to quantity the physical accuracy of sound field synthesis systems.

This paper presents a framework for parametric broadband beamforming that exploits the frequency-domain sparsity of voiced speech to achieve more noise reduction than traditional nonparametric broadband beamforming without introducing additional distortion. In this framework, the harmonic model is used to parametrize the signal of interest by a single parameter, the fundamental frequency, whereby both speech enhancement and derevereration can be performed. This framework thus exploits both the spatial and temporal properties of speech signals simultaneously and includes both fixed and adaptive beamformers, such as (1) delay-and-sum, (2) null forming, (3) Wiener, (4) minimum variance distortionless response (MVDR), and (5) linearly constrained minimum variance beamformers. Moreover, the framework contains standard broadband beamforming as a special case, whereby the proposed beamformers can also handle unvoiced speech. The reported experimental results demonstrate the capabilities of the proposed framework to perform both speech enhancement and dereverberation simultaneously. The proposed beamformers are evaluated in terms of speech distortion and objective measures for speech quality and speech intelligibility, and are compared to nonparametric broadband beamformers. The results show that the proposed beamformers perform well compared to traditional methods, including a state-of-the-art dereverberation method, particularly in adverse conditions with high amounts of noise and reverberation.

Ce mémoire d'habilitation à diriger des recherches fournit une vue d'ensemble sur mes travaux, depuis mon recrutement en tant que Maître de Conférences en Acoustique au Conservatoire National des Arts et Métiers, en 2009. Les différents axes de recherches sur lesquels je me suis concentré depuis cette date ont pour fil conducteur l'utilisation de méthodes temporelles multicanales, pour le traitement des données associées aux réseaux de transducteurs. J'ai développé ces méthodes pour les appliquer à la résolution de problèmes inverses en environnement industriel, à l'élastographie du corps humain pour le diagnostic médical, à la localisation de snipers et de drones, ou encore pour proposer des stratégies d'apprentissage profond appliquées à la localisation de sources sonores, et à la reconnaissance de parole ou de sons environnementaux. Pour chacune de ces applications, je fournis dans ce document une description des méthodes proposées, ainsi qu'une synthèse des résultats les plus importants obtenus. La cohérence et les liens entre les différents projets sont mises en exergue, et les chapitres sont systématiquement illustrés de résultats numériques et expérimentaux. Le développement de dispositifs miniaturisés pour la synthèse de champs sonores et la captation est également mise en avant, puisque c'est l'une des caractéristiques de mes travaux de recherche ces dernières années.En fin de document, je propose également six propositions concrètes de recherches pour les années à venir, permettant ainsi d'exposer ma vision à court et moyen-terme des développements qui pourraient être réalisés sur la base de mes travaux.

Ce mémoire d'habilitation à diriger des recherches fournit une vue d'ensemble sur mes travaux, depuis mon recrutement en tant que Maître de Conférences en Acoustique au Conservatoire National des Arts et Métiers, en 2009. Les différents axes de recherches sur lesquels je me suis concentré depuis cette date ont pour fil conducteur l'utilisation de méthodes temporelles multicanales, pour le traitement des données associées aux réseaux de transducteurs. J'ai développé ces méthodes pour les appliquer à la résolution de problèmes inverses en environnement industriel, à l'élastographie du corps humain pour le diagnostic médical, à la localisation de snipers et de drones, ou encore pour proposer des stratégies d'apprentissage profond appliquées à la localisation de sources sonores, et à la reconnaissance de parole ou de sons environnementaux. Pour chacune de ces applications, je fournis dans ce document une description des méthodes proposées, ainsi qu'une synthèse des résultats les plus importants obtenus. La cohérence et les liens entre les différents projets sont mises en exergue, et les chapitres sont systématiquement illustrés de résultats numériques et expérimentaux. Le développement de dispositifs miniaturisés pour la synthèse de champs sonores et la captation est également mise en avant, puisque c'est l'une des caractéristiques de mes travaux de recherche ces dernières années.En fin de document, je propose également six propositions concrètes de recherches pour les années à venir, permettant ainsi d'exposer ma vision à court et moyen-terme des développements qui pourraient être réalisés sur la base de mes travaux.

International audience; One of the most difficult challenges in developing an Autonomous Vehicle (AV) is the requirement that it will perform no worse than a human. Human drivers use both eyes and ears as visual and audible sensors. However, state-of-the-art AV sensors do not capture acoustic signals, and as a result, decision making algorithms do not use the information that is contained in the acoustic scene. This work is focused on detecting the arrival direction of Emergency Vehicles (EVs). The EV's siren signal is recorded using a microphone array and adjusted Multiple Signal Classification (MUSIC) algorithms are used to estimate the Direction of Arrival (DoA) of the EV. Experiments show DoA estimation results using Microelectromechanical system (MEMS) microphones. Performance using both alternatives of an internal and external microphone array architecture were investigated. In the case of external microphones, free-field model was used to calculate the steering vector with high spatial resolution. However, external microphones require protection against rain, wind, etc. For this reason, the option of using internal microphones was examined. In the case of the internal microphone arrays, the free-field model is no longer valid and therefore estimated Relative Transfer Function (RTF)s are incorporated. The RTFs are estimated using signals that are generated from external source in a finite set of locations, and recorded using the internal microphones in a quite environment. For this reason, in the case of the internal microphones, spatial scan of the MUSIC algorithm is performed using a lower spatial resolution than in the case of the external microphones. The RTF estimation accuracy using Wiener filter is compared to the one using the Generalized Eigenvalue Decomposition (GEVD) method. Experiments show results of the external microphone array mounted on the roof of a car that was parked near a hospital. The EVs were ambulance vehicles recorded arriving to or departing from the hospital. For example, the case where an EV ap­proached from the opposite lane and made its way to the hospital is shown. Basically, at first the DoA comes from the frontal direction, and then switches backwards. This work shows the MUSIC spectrum, the estimated DoA, and the selected frequency for these cases. It is shown that at a certain time, the peak of the MUSIC spectrum shifted from values near 360^\circ to values near 180^\circ. In this case, the decision of an AV should be to continue normal driving and not to yield to the EV. Due to mirroring effect of the porch hardware that is physically placed on the sides of the microphone array board, in cases where the EV was on the left side, the peak values appeared at angles that corresponded to the right side. Nevertheless, it was easy to determine when the EV was in front or behind the car. Additional experiments show results using internal microphones with only front and back RTFs incorporated as steering vectors. The MUSIC spectrum and DoA estimation results were shown for different DoAs of the EV. Two internal microphone array configurations were demonstrated, a broadside array and an end-fire array. The results have shown that using the end-fire array it was impossible to determine whether the EV was behind or in front of the car. The DoA was estimated better using the broadside array. This result was surprising, since one would expect that the symmetry of the broadside array around the driving direction would have caused an ambiguity for waves approaching from the front or from the back. However, the RTFs were estimated using each microphone with less distortion using the broadside array than using the end-fire array. Regardless, the difficulty of estimating the DoA and the lower angular resolution was greater in the case of the internal microphone array than in the case of the external one.

Voice-enabled interaction systems in domestic environments have attracted significant interest recently, being the focus of smart home research projects and commercial voice assistant home devices. Within the multi-module pipelines of such systems, speech activity detection (SAD) constitutes a crucial component, providing input to their activation and speech recognition subsystems. In typical multi-room domestic environments, SAD may also convey spatial intelligence to the interaction, in addition to its traditional temporal segmentation output, by assigning speech activity at the room level. Such room-localized SAD can, for example, disambiguate user command referents, allow localized system feedback, and enable parallel voice interaction sessions by multiple subjects in different rooms. In this paper, we investigate a room-localized SAD system for smart homes equipped with multiple microphones distributed in multiple rooms, significantly extending our earlier work. The system employs a two-stage algorithm, incorporating a set of hand-crafted features specially designed to discriminate room-inside vs. room-outside speech at its second stage, refining SAD hypotheses obtained at its first stage by traditional statistical modeling and acoustic front-end processing. Both algorithmic stages exploit multi-microphone information, combining it at the signal, feature, or decision level. The proposed approach is extensively evaluated on both simulated and real data recorded in a multi-room, multi-microphone smart home, significantly outperforming alternative baselines. Further, it remains robust to reduced microphone setups, while also comparing favorably to deep learning-based alternatives.