Search

Detection and recognition of loud sounds and characteristic noises can significantly increase the level of safety and ensure timely response to various emergency situations. Audio event detection is the first step in recognizing audio signals in a continuous audio input stream. This article presents a number of problems that are associated with the development of sound event detection systems, such as the deviation for each environment and each sound category, overlapping audio events, unreliable training data, etc. Both methods for detecting monophonic impulsive audio event and polyphonic sound event detection methods which are used in the state-of-the-art sound event detection systems are presented. Such systems are presented in Detection and Classification of Acoustic Scenes and Events (DCASE) challenges and workshops, which take place every year. Beside a majority of works focusing on the improving overall performance in terms of accuracy many other aspects have also been studied. Several systems presented at DCASE 2021 task 4 were considered, and based on their analysis, there was a conclusion about possible future for sound event detection systems. Also the actual directions in the development of modern audio analytics systems are presented, including the study and use of various architectures of neural networks, the use of several data augmentation techniques, such as universal sound separation, etc., Виявлення та розпізнавання гучних звуків і характерних шумів дозволяє значно підвищити рівень безпеки та забезпечити своєчасне реагування на різні аварійні ситуації. Детектування аудіоподій – це перший крок у розпізнаванні аудіосигналів з безперервним вхідним аудіопотоком. У даній статті представлено ряд проблем, пов'язаних з розробкою систем виявлення аудіоподій, таких як відхилення для кожного середовища і кожної звукової категорії, звукові події, що перекриваються, недостовірні навчальні дані та ін. Представлені як методи виявлення монофонічних імпульсних звукових подій, так і методи виявлення поліфонічних аудіоподій, які використовуються в сучасних системах виявлення звукових подій. Такі системи представлені у завданнях та семінарах Detection and Classification of Acoustic Scenes and Events (DCASE), які відбуваються щороку. Більшість робіт спрямовані на покращення загальної продуктивності з точки зору точності, хоча також були вивчені багато інших аспектів. Було розглянуто кілька систем, представлених на DCASE 2021 в задачі 4, і на основі їх аналізу був зроблений висновок про можливе майбутнє систем виявлення звукових подій. Також представлені актуальні напрямки розвитку сучасних систем аудіоаналітики, в тому числі вивчення та використання різних архітектур нейронних мереж, використання декількох методів попередньої обробки даних, таких як універсальний розділ звуку та ін.

Detection and recognition of loud sounds and characteristic noises can significantly increase the level of safety and ensure timely response to various emergency situations. Audio event detection is the first step in recognizing audio signals in a continuous audio input stream. This article presents a number of problems that are associated with the development of sound event detection systems, such as the deviation for each environment and each sound category, overlapping audio events, unreliable training data, etc. Both methods for detecting monophonic impulsive audio event and polyphonic sound event detection methods which are used in the state-of-the-art sound event detection systems are presented. Such systems are presented in Detection and Classification of Acoustic Scenes and Events (DCASE) challenges and workshops, which take place every year. Beside a majority of works focusing on the improving overall performance in terms of accuracy many other aspects have also been studied. Several systems presented at DCASE 2021 task 4 were considered, and based on their analysis, there was a conclusion about possible future for sound event detection systems. Also the actual directions in the development of modern audio analytics systems are presented, including the study and use of various architectures of neural networks, the use of several data augmentation techniques, such as universal sound separation, etc. Виявлення та розпізнавання гучних звуків і характерних шумів дозволяє значно підвищити рівень безпеки та забезпечити своєчасне реагування на різні аварійні ситуації. Детектування аудіоподій – це перший крок у розпізнаванні аудіосигналів з безперервним вхідним аудіопотоком. У даній статті представлено ряд проблем, пов'язаних з розробкою систем виявлення аудіоподій таких, як відхилення для кожного середовища і кожної звукової категорії, звукові події, що перекриваються, недостовірні навчальні дані та ін. Представлені, як методи виявлення монофонічних імпульсних звукових подій, так і методи виявлення поліфонічних аудіоподій, які використовуються в сучасних системах виявлення звукових подій. Такі системи представлені у завданнях та семінарах Detection and Classification of Acoustic Scenes and Events (DCASE), які відбуваються щороку. Більшість робіт спрямовані на покращення загальної продуктивності з точки зору точності, хоча також були вивчені багато інших аспектів. Було розглянуто кілька систем, представлених на DCASE 2021 в задачі 4, і на основі їх аналізу був зроблений висновок про можливе майбутнє систем виявлення звукових подій. Також представлені актуальні напрямки розвитку сучасних систем аудіоаналітики, в тому числі вивчення та використання різних архітектур нейронних мереж, використання декількох методів попередньої обробки даних, таких як універсальний розділ звуку та ін.

Monaural music sound separation isolates individual instrument sources from a mono-channel polyphonic mixture. The primary challenge is to separate the source partials overlapped in time-frequency regions, especially for the full overlapping cases that at least one source does not have any nonoverlapping partial. Due to the lack of effective methods to separate the sources with full overlapping partials, this paper put forward a relaxed extended common amplitude modulation (RECAM) approach to deal with the octave sound separation, one of the most difficult cases. Our strategy uses a multi-band co-processing way for each short-time partial wave segment. Extensive experiments are conducted on octave mixture samples drawn from the Iowa University Musical Instrument Database. Results confirm that our RECAM achieves the best separation performance. For nonvibrato and vibrato mixtures, the average improvement of RECAM in each measure exceeds 3dB and 2dB, respectively.

International audience; Performing sound event detection on real-world recordings often implies dealing with overlapping target sound events and non-target sounds, also referred to as interference or noise. Until now these problems were mainly tackled at the classifier level. We propose to use sound separation as a pre-processing for sound event detection. In this paper we start from a sound separation model trained on the Free Universal Sound Separation dataset and the DCASE 2020 task 4 sound event detection baseline. We explore different methods to combine separated sound sources and the original mixture within the sound event detection. Furthermore, we investigate the impact of adapting the sound separation model to the sound event detection data on both the sound separation and the sound event detection.

International audience; We propose a benchmark of state-of-the-art sound event detection systems (SED). We designed synthetic evaluation sets to focus on specific sound event detection challenges. We analyze the performance of the submissions to DCASE 2021 task 4 depending on time related modifications (time position of an event and length of clips) and we study the impact of non-target sound events and reverberation. We show that the localization in time of sound events is still a problem for SED systems. We also show that reverberation and non-target sound events are severely degrading the performance of the SED systems. In the latter case, sound separation seems like a promising solution.

Cochlear implant (CI) users suffer from limitations in music perception and thus prefer music which has a clear rhythm/beat and is played with only a few instruments. Therefore, existing music pre-processing methods aim to enhance music signals for CI users by either emphasizing preferred voices or reducing the spectral complexity of the signals. In this work, a music pre-processing scheme is described which combines these approaches and is applicable to a wider variety of music genres.The proposed method is evaluated and compared to other recently developed methods using instrumental measures and a listening test with vocoded pop/rock music excerpts and normal hearing listeners. Unprocessed popular music pieces as well as different processed versions were rated comparatively in terms of distinctness of drums, distinctness of melody, and the overall impression. The listening test showed significantly better ratings for the proposed method compared to unprocessed music and most of the other processing schemes. As the instrumental measures also indicate improvements, the proposed combined strategy is a promising candidate for music enhancement for CI listeners.

Wheezing reveals important cues that can be useful in alerting about respiratory disorders, such as Chronic Obstructive Pulmonary Disease. Early detection of wheezing through auscultation will allow the physician to be aware of the existence of the respiratory disorder in its early stage, thus minimizing the damage the disorder can cause to the subject, especially in low-income and middle-income countries. The proposed method presents an extended version of Non-negative Matrix Partial Co-Factorization (NMPCF) that eliminates most of the acoustic interference caused by normal respiratory sounds while preserving the wheezing content needed by the physician to make a reliable diagnosis of the subject&rsquo, s airway status. This extension, called Informed Inter-Segment NMPCF (IIS-NMPCF), attempts to overcome the drawback of the conventional NMPCF that treats all segments of the spectrogram equally, adding greater importance for signal reconstruction of repetitive sound events to those segments where wheezing sounds have not been detected. Specifically, IIS-NMPCF is based on a bases sharing process in which inter-segment information, informed by a wheezing detection system, is incorporated into the factorization to reconstruct a more accurate modelling of normal respiratory sounds. Results demonstrate the significant improvement obtained in the wheezing sound quality by IIS-NMPCF compared to the conventional NMPCF for all the Signal-to-Noise Ratio (SNR) scenarios evaluated, specifically, an SDR, SIR and SAR improvement equals 5.8 dB, 4.9 dB and 7.5 dB evaluating a noisy scenario with SNR = &minus, 5 dB.

Videos are used widely as the media platforms for human beings to touch the physical change of the world. However, we always receive the mixed sound from the multiple sound objects, and cannot distinguish and localize the sounds as the separate entities in videos. In order to solve this problem, a model named the Deep Multi-Modal Attention Network (DMMAN), is established to model the unconstrained video datasets for further finishing the sound source separation and event localization tasks in this paper. Based on the multi-modal separator and multi-modal matching classifier module, our model focuses on the sound separation and modal synchronization problems using two stage fusion of the sound and visual features. To link the multi-modal separator and multi-modal matching classifier modules, the regression and classification losses are employed to build the loss function of the DMMAN. The estimated spectrum masks and attention synchronization scores calculated by the DMMAN can be easily generalized to the sound source and event localization tasks. The quantitative experimental results show the DMMAN not only separates the high quality of the sound sources evaluated by Signal-to-Distortion Ratio and Signal-to-Interference Ratio metrics, but also is suitable for the mixed sound scenes that are never heard jointly. Meanwhile, DMMAN achieves better classification accuracy than other contrast baselines for the event localization tasks.

International audience; We introduce the Free Universal Sound Separation (FUSS) dataset, a new corpus for experiments in separating mixtures of an unknown number of sounds from an open domain of sound types. The dataset consists of 23 hours of single-source audio data drawn from 357 classes, which are used to create mixtures of one to four sources. To simulate reverberation, an acoustic room simulator is used to generate impulse responses of box shaped rooms with frequency-dependent reflective walls. Additional open-source data augmentation tools are also provided to produce new mixtures with different combinations of sources and room simulations. Finally, we introduce an open-source baseline separation model, based on an improved time-domain convolutional network (TDCN++), that can separate a variable number of sources in a mixture. This model achieves 9.8 dB of scale-invariant signal-to-noise ratio improvement (SI-SNRi) on mixtures with two to four sources, while reconstructing single-source inputs with 35.5 dB absolute SI-SNR. We hope this dataset will lower the barrier to new research and allow for fast iteration and application of novel techniques from other machine learning domains to the sound separation challenge.

In this dissertation, a research on heart sound and respiratory sound distinction system by using body conduction sound sensor is presented. In order to separate heart sound and respiratory sound effectively, a new and fast method by using the Harmonic/Percussive Sound Separation measurement has been proposed and verified in this research. In this research, a body conduction sound sensor has been used to collect the mixed heart sound signal and respiratory sound signal from the human body. A simple and fast method based on harmonic percussive sound separation (HPSS) is applied on the mixed sound to be separated. The aim of this research is to study the effectiveness of HPSS measurement to separate a human heart sound and respiratory sound. To evaluate the effectiveness of the proposed measuring method, I conducted an experiment using the proposed method on several subjects. The experimental result showed that the proposed method is effective and can be used to monitor our heart sound and respiratory sound abnormalities in the future.

In bioacoustics technology, advances such as automated sound recognition based on animal vocalization help in biological research and environmental monitoring. However, in a noisy acoustic environment, where there will be interferences such as overlapping sounds made by multiple species, may greatly hamper the automated sound recognizer performance to identify the specific species. Hence, it is desirable to extract the sound made by the target species from the interferences as a pre-process prior to the recognition to get more accurate results. This paper exploits two Blind Source Separation (BSS) algorithms namely Info-max and FastICA to obtain the target frog sounds from the mixtures. The comparison of algorithm performances is expressed according to Signal-to-Interfere (SIR). The empirical simulation results show that FastICA outperforms Infomax in terms of separation quality.

Rindik is a traditional music instrument originated from Bali that consists of 11 bamboo rods and played by a person hitting the bamboo rod with a rubber mallet in each player’s hands. Documentation of Rindik songs with automatic music transcription work is easier to do by separating it first. To overcome that challenge, two Rindik rods sound were separated using the spectral subtraction method. The noise spectrum is the spectrum of a single rod sound that needs to be muffled. The resulting audio is the other single rod sound and vice versa to get both single rod sound. The data consisted of single rod hit sound recordings of 11 single rod sound and 55 combinations of two-rods sounds being hit at the same time. The performance of the spectral subtraction method in separating Rindik sound was measured with MSE and SIR and also heard the noise exists in the separated audio signal. The experiment demonstrated a state-of-the-art performance consists of spectral subtraction with squared noise average magnitude with average MSE value 0.0126 and average SIR value 55.68 dB.

Acoustic domain contains signals related to sound. Speech and music though are included in this domain, both the signals differ with various features. Features used for speech separation does not provide sufficient cue for music separation. This paper covers musical sound separation for monaural recordings. A system is proposed to classify singing voice and music from monaural recordings. For classification, time and frequency domain features along with Mel Frequency Cepstral Coefficients (MFCC) applied to input signal. Information carried by these signals permit to establish results Quantitative experimental results shows that the system performs the separation task successfully in monaural environment.

Mixed sounds can be separated from multiple sound sources using microphone array sensor and signal processing. We believe that promotion of interest in this technique can lead to significant future development in science and technology. To investigate this technique, we designed a language game for children called “KIKIWAKE 3D” that uses a sound-source-separation system to arouse children’s interest in this technology. However, the microphone array sensor in a previous research had a limited scope in separating sounds. We developed a spherical microphone array sensor with three-dimensional directivity designed for this game. In this paper, we report the evaluation of this microphone array sensor in adapting to this game by separating the sound level and using questionnaires.

In the case of a flat-screen TV, the position of the sound was properly configured by reproducing the sound directly from the left and right sides of the screen. However, focusing on the image qual...

In order to solve the problems of current hearing aid devices, we make use of environment sensor network, and propose a hearing support system, where individual target and anti-target sound sources in the environment can be selected, and spatial information of the target sound sources is reconstructed. The performance of the selective sound separation module was evaluated for different noise conditions. Results showed that signal-to-noise ratios of around 15dB could be achieved by the proposed system for a 65dB babble noise plus directional music noise condition. In the same noise condition, subjective intelligibility tests were conducted, and an improvement of 65 to 90% word intelligibility rates could be achieved by using the proposed hearing support system.

In this work we assesses the music genre classification using spectrograms taken from the original signal, percussive content signal, and harmonic content signal. The rationale behind this is that classifiers obtained from this three different representation of the signal may present some complementarity to each other. By this way, one can improve the recognition rates already obtained in previous works which has explored only the original signal content. LBP texture features were used to represent the spectrogram content, and the classification step was supported by SVM. The spectrogram images were zoned taking to account a perceptual scale, and a specific classifier was created for each zone, which has led us to combine classifiers outputs to get the final decision. The performance of our approach reaches the recognition rate about 88.56% which, to the best of our knowledge, is the best rate ever obtained on the LMD dataset using artist filter constraint.

The problem of overlapping harmonics is particularly acute in musical sound separation and has not been addressed adequately. We propose a monaural system based on binary time-frequency masking with an emphasis on robust decisions in time-frequency regions, where harmonics from different sources overlap. Our computational auditory scene analysis system exploits the observation that sounds from the same source tend to have similar spectral envelopes. Quantitative results show that utilizing spectral similarity helps binary decision making in overlapped time-frequency regions and significantly improves separation performance.

In this paper, we propose a method to separate the heart sound signals based on the assumption that heart sound signals originating from cardiac structures are statistically independent and have non-Gaussian distribution. This assumption cast the problem under blind signal separation where in this paper independent component analysis method is used. The objective of this work is intended to improve detection of cardiac abnormalities and generate a murmur expert system that aids physician in their diagnosis. Results indicate that four independent components out of six mixed signals are successfully separated. The separated signals can be utilized with other patient information for better diagnosis.

Interference often occurs between the lung sound (LS) and the heart sound (HS). Due to the overlap in their frequency spectrums, it is difficult to separate them. This paper proposes a novel separation method based on the adaptive Fourier decomposition (AFD) to separate the HS and the LS with the minimum energy loss. This AFD-based separation method is validated on the real HS signal from the University of Michigan Heart Sound and Murmur Library as well as the real LS signal from the 3M repository. Simulation results indicate that the proposed method is better than other extraction methods based on the recursive least square (RLS), the standard empirical mode decomposition (EMD) and various extensions of the EMD including the ensemble EMD (EEMD), the multivariate EMD (M-EMD) and the noise assisted M-EMD (NAM-EMD).

In a virtual sound imaging system, crosstalk cancellation filters are used to create an effective sweet spot for 3D sound reproduction via multiple loudspeakers. A new 3-channel system is proposed to improve system performance on sound separation. Based on the robustness analysis of a crosstalk canceller, a modified-inverse filter technique is explored and demonstrated using two different examples of symmetric speaker positions. The simulation results indicate that the present system is robust over a wider bandwidth compared to a conventional 2-channel system.

We developed a sound separation system that uses a microphone array to separate children's voices from background noise in general living environments. Our focus in the development of this system is on “a clear separation of the human voice” so that it will be easy for children to use. Evaluations conducted of sound separation performance in previous research have not always measured how easily listeners hear separate sounds. In this paper, we examine the validity of our system design by evaluating this factor from the perspectives of engineering and psychology.

Acoustic crosstalk cancelation is a signal processing technique that uses two or more loudspeakers reconstruct the acoustic pressures at the listener's ears. The crosstalk arises because each speaker sends sound to the ipsilateral ear, as well as undesired sound to the contra lateral ear. Taking crosstalk cancelation technique can deliver desired signals exactly at the listener's ears that would result from the natural listening situation to be simulated and using crosstalk cancelation technique people can enjoy spatial audio without wearing a headphone. Classical crosstalk cancelation systems (CCS) employ only two loudspeakers, and their performance is usually unsatisfactory in practice. While the idea of using more loudspeakers has been investigated, we design a new 3-channel system to improve the performance on sound separation and robustness to head. The simulation results indicate that multichannel loudspeaker can improve the performance and the robustness of three-dimensional (3D) audio system.

Three new tests were introduced in this study to compare nasal resonance and speaking time of 30 children 8 to 11 years old who were profoundly deaf with that of 30 children with normal hearing in a matched control group. The ANS-P plane was introduced to position the palatometer sound separation plate. This enabled the plate to be brought easily and repeatedly into the desired standard position despite widely varying facial contours of the subjects. The findings from this study showed the group with hearing loss had significantly more nasalance than did the normal group when nasal consonants were absent and significantly less when an utterance was loaded heavily with nasal consonants. These differences were interpreted as evidence of some limitation in the ability of the children with hearing loss to monitor and control nasal versus oral sound emission. Speaking time was longer in the group of children with hearing loss but was not related to the nasalance score.

This chapter contains sections titled: Human Auditory Scene Analysis Computational Auditory Scene Analysis (CASA) Basics of CASA Systems CASA Evaluation Other Sound Separation Approaches A Brief History of CASA (Prior to 2000) Conclusions This chapter contains sections titled: Acknowledgments References ]]>

national Journal of Audiology 41: 100–112. Ricketts T, Henry P, Gnewikow. (2002) Full time directional versus user selectable microphone modes in hearing aids. Ear and Hearing 24(5): 424–439. Schum D, Pogash R (2002) Initial clinical verification of a DSP instrument. Hearing Review 9(9): 48–51. Thompson SC (1999) Dual microphones or directional-plus-omni: which is best? High Performance Hearing Solutions 3: 31–35. Wouters J, Vanden Berghe J (2001) Speech recognition in noise for cochlear implantees with a twomicrophone monaural adaptive noise reduction system. Ear and Hearing 22(5): 420–430.

The fetal phonocardiogram, which is the acoustic recording of mechanical activity of the fetal heart, facilitates the measurement of the instantaneous fetal heart rate, beat-to-beat differences and duration of systolic and diastolic phases. These measures are sensitive indicators of cardiac function, reflecting fetal well-being. This paper provides an algorithm to non-invasively estimate the phonocardiogram of an individual fetus in a multiple fetus pregnancy. A mixture of fetal phonocardiograms is modeled by a generalized pure delayed mixing model. Mutual independence of fetal phonocardiograms is assumed to apply blind source separation based techniques to extract the fetal phonocardiograms from their mixtures. The performance of the algorithm is verified through simulation results and on experimental data obtained from a phantom that is used to simulate a twin pregnancy.

Recently, shift-invariant tensor factorisation algorithms have been proposed for the purposes of sound source separation of pitched musical instruments. However, in practice, existing algorithms require the use of log-frequency spectrograms to allow shift invariance in frequency which causes problems when attempting to resynthesise the separated sources. Further, it is difficult to impose harmonicity constraints on the recovered basis functions. This paper proposes a new additive synthesis-based approach which allows the use of linear-frequency spectrograms as well as imposing strict harmonic constraints, resulting in an improved model. Further, these additional constraints allow the addition of a source filter model to the factorisation framework, and an extended model which is capable of separating mixtures of pitched and percussive instruments simultaneously.

The ideal binary mask is often seen as a goal for time-frequency masking algorithms trying to increase speech intelligibility, but the required availability of the unmixed signals makes it difficult to calculate the ideal binary mask in any real-life applications. In this paper we derive the theory and the requirements to enable calculations of the ideal binary mask using a directional system without the availability of the unmixed signals. The proposed method has a low complexity and is verified using computer simulation in both ideal and non-ideal setups showing promising results.

The term auditory scene analysis (ASA) refers to the ability of human listeners to form perceptual representations of the constituent sources in an acoustic mixture, as in the well-known ‘cocktail party’ effect. Accordingly, computational auditory scene analysis (CASA) is the field of study which attempts to replicate ASA in machines. Some CASA systems are closely modelled on the known stages of auditory processing, whereas others adopt a more functional approach. However, all are broadly based on the principles underlying the perception and organization of sound by human listeners, and in this respect they differ from ICA and other approaches to sound separation. In this chapter, we review the principles underlying ASA and show how they can be implemented in CASA systems. We also consider the link between CASA and automatic speech recognition, and draw distinctions between the CASA and ICA approaches.