Search

MPEG surround is one of the most popular multichannel audio coding standards used today. In producing quality sound, there are several parts that are influential, but those that are directly related to sound reproduction, one of which is the method of downmixing, mixing the right channels will improve the quality of the sound produced. Because this research is closely related to data computing, the Matrix Laboratory (MATLAB) application is used. There are 2 stages carried out in the MATLAB application, namely the encoding and decoding process. In the encoding process in MATLAB, each audio channel is set according to a previously designed scheme. The results of this encoding are then decoded, so that the ODG value of each tested audio channel is obtained. In his presentation, the design and analysis of several domnmixing schemes was carried out. There are 5 schemes tested and to determine the performance of each scheme, these schemes are tested at high bitrates, from the range of 1000 Kbps to 2000 Kbps. Based on the test, it was found that the effect of increasing the bitrate is directly proportional to the resulting audio quality. During testing Scheme-5 has a better level of stability among all downmixing schemes and the average percentage increase in Objective Difference Grade (ODG) for all audio samples is 9.6927 %., MPEG surround merupakan salah satu standard pengkodean audio multikanal yang populer digunakan saat ini. Dalam menghasilkan suara yang berkualitas, ada beberapa bagian yang berpengaruh, namun yang berkaitan langsung dengan reproduksi suara, salah satunya adalah metode dalam melakukan downmix, percampuran kanal yang tepat akan meningkatkan kualitas dari suara yang dihasilkan. Dikarenakan penelitian ini berkaitan erat dengan komputasi data, maka digunakanlah aplikasi Matrix Laboratory (MATLAB). Ada 2 tahap yang dilakukan pada aplikasi MATLAB, yaitu proses encoding dan decoding. Pada proses encoding di MATLAB, disetting setiap kanal audio sesuai dengan skema yang telah dirancang sebelumnya. Hasil dari encoding ini kemudian di decode, sehingga didapatkan nilai ODG dari setiap kanal audio yang diujikan. Pada pemaparannya dilakukan perancangan dan analisis terhadap beberapa skema domnmixing. Terdapat 5 skema yang diujikan dan untuk mengetahui performansi dari tiap skema, skema tersebut diuji pada bitrate tinggi mulai dari range 1000 Kbps sampai dengan 2000 Kbps. Berdasarkan pengujian didapati bahwa pengaruh kenaikan bitrate berbanding lurus terhadap kualitas audio yang dihasilkan. Selama pengujian Skema-5 memiliki tingkat kestabilan yang lebih baik diantara semua skema downmixing dan rata-rata persentase kenaikan Objective Difference Grade (ODG) untuk seluruh sampel audio adalah 9,6927 %.

To assist an investigation process, forensic experts compare and analyze audio recordings. Speech utterances are compared by humans and/or machines for use in court for investigation. Scientific research community insists for specific automatic or human-based approach to identify uniquxy2e audio features from identical twins group. Filters can be employed to enhance an audio recording for improving clarity. This may entail removal of unnecessary noise to enrich the intelligibility of speech. Forensic audio experts can examine a variety of characteristics of the audio recording to decide the possibility of alterations in the collected evidences. This includes confirming the integrity and authenticating that the content is what it purports to be. Thiswork named as FIST(Forensic Investigation for Twin Identification from Speech: Perceptual and Gamma-tone Features and Models) proposes an automated system to identify a twin from identical twin pairs by the use of gamma-tone features and perceptual features.The proposed features are excerpted from the set of training speeches and templates are created for each twin based on vector quantisation (VQ), Fuzzy C means clustering (FCM) and multivariate hidden Markov modelling (MHMM) techniques. For testing, features are extracted from the set of test utterances and worked out to the templates for classification. Based on the type of classifier used, classification of twin is carried out with minimum distance and maximum loglikelihood value. The proposed features are examinedfor sub-optimal and true success rates as key performance metrics to assess the system and also a comparative analysis is made across the proposed features. Among the inspected features, Gammatone energy features expose better performance in comparison to perceptual features by attaining the overall sub-optimal success rate and true success rate as97.8375% and 92.75% for Gammatone energy features with VQ based modelling technique. This work FIST has also been analysed by inducing disturbance in the form of speech interference from their own twin pairs and Gamma-tone energy feature with VQ based modelling technique performs better for twin identification. A high claim of 99.625% and 95.0625% accuracy has been achieved by employing decision level fusion classifier.

Background The prediction violation account of automatic or pre-attentive change detection assumed that the inferior frontal cortex (IFC) is involved in establishing a prediction model for detecting unexpected changes. Evidence supporting the IFC’s contribution to prediction model is mainly based on the Mismatch Negativity (MMN) to deviants violating predictions that are established based on the frequently presented standard events. However, deviant detection involves processes, such as events comparison, other than prediction model establishment. Objective The current study investigated the critical role of the IFC in establishing a prediction model during standards processing for subsequent deviant detection. Methods Transcranial Magnetic Stimulation (TMS) was applied at the IFC to disrupt the processing of the initial 2 or 5 standards of a 3-, 6-, or 9-standard train, while the MMN responses to pitch deviant presented after the standard trains were recorded and compared. Results An abolishment of MMN was only observed when TMS was delivered to the IFC at the initial 2 standards of the 3-standard train, but not at the initial 5 standards, or when TMS at the vertex or TMS sound recording was applied. The MMNs were also preserved when IFC TMS, vertex TMS, or TMS sound recording was applied at the initial 2 or 5 standards of longer trains. Conclusion The IFC plays a critical role in processing the initial standards of a short standard train for subsequent deviant detection. This result is consistent with the prediction violation account that the IFC is important for establishing the prediction model.

Identification of the acoustic environment from an audio recording, also known as acoustic scene classification, is an active area of research. In this paper, we study dynamically-changing background acoustic scenes from the egocentric perspective of an individual in a workplace. In a novel data collection setup, wearable sensors were deployed on individuals to collect audio signals within a built environment, while Bluetooth-based hubs continuously tracked the individual's location which represents the acoustic scene at a certain time. The data of this paper come from 170 hospital workers gathered continuously during work shifts for a 10 week period. In the first part of our study, we investigate temporal patterns in the egocentric sequence of acoustic scenes encountered by an employee, and the association of those patterns with factors such as job-role and daily routine of the individual. Motivated by evidence of multifaceted effects of ambient sounds on human psychology, we also analyze the association of the temporal dynamics of the perceived acoustic scenes with particular behavioral traits of the individual. Experiments reveal rich temporal patterns in the acoustic scenes experienced by the individuals during their work shifts, and a strong association of those patterns with various constructs related to job-roles and behavior of the employees. In the second part of our study, we employ deep learning models to predict the temporal sequence of acoustic scenes from the egocentric audio signal. We propose a two-stage framework where a recurrent neural network is trained on top of the latent acoustic representations learned by a segment-level neural network. The experimental results show the efficacy of the proposed system in predicting sequence of acoustic scenes, highlighting the existence of underlying temporal patterns in the acoustic scenes experienced in workplace.

Voice Activity Detection (VAD) refers to the task of identification of regions of human speech in digital signals such as audio and video. While VAD is a necessary first step in many speech processing systems, it poses challenges when there are high levels of ambient noise during the audio recording. To improve the performance of VAD in such conditions, several methods utilizing the visual information extracted from the region surrounding the mouth/lip region of the speakers' video recording have been proposed. Even though these provide advantages over audio-only methods, they depend on faithful extraction of lip/mouth regions. Motivated by these, a new paradigm for VAD based on the fact that respiration forms the primary source of energy for speech production is proposed. Specifically, an audio-independent VAD technique using the respiration pattern extracted from the speakers' video is developed. The Respiration Pattern is first extracted from the video focusing on the abdominal-thoracic region of a speaker using an optical flow based method. Subsequently, voice activity is detected from the respiration pattern signal using neural sequence-to-sequence prediction models. The efficacy of the proposed method is demonstrated through experiments on a challenging dataset recorded in real acoustic environments and compared with four previous methods based on audio and visual cues., Accepted in IEEE Sensor Letters

In this article, an artificial neural network is proposed to classify short audio sequences of squeak and rattle (S&R) noises. The aim of the classification is to see how accurately the trained classifier can recognize different types of S&R sounds. Having a high accuracy model that can recognize audible S&R noises could help to build an automatic tool able to identify unpleasant vehicle interior sounds in a matter of seconds from a short audio recording of the sounds. In this article, the training method of the classifier is proposed, and the results show that the trained model can identify various classes of S&R noises: simple (binary classification) and complex ones (multi class classification). © 2020 Institute of Noise Control Engineering.

The problem of automated quality control of audio recordings containing voice samples of individuals is considered. It is shown that in solving this problem the most acute impediment is the problem of small samples of observations. To overcome the problem, a new, high-speed method of acoustic measurements is proposed, based on the principle of relative stability of the frequency of the fundamental tone of a speech signal within a voice sample of short duration. An example of practical implementation of the developed method according to a scheme with inter-period signal accumulation is considered. Using proprietary software, a full-scale experiment was conducted in which statistical estimates of the effectiveness of the method in noise conditions were obtained. It is shown that when using the proposed method, if the signal-to-noise ratio is lower than 15 dB, an audio recording is rejected with a probability of 0.95 or more as unsuitable for biometric identification of a person. The results obtained are intended for use in the development of new systems and modernization of existing systems and technologies for collection and automated quality control of biometric personal data. The article is intended for a wide circle of specialists in the field of acoustic measurements and digital processing of speech signals, as well as for practitioners who organize the work of authorized organizations in preparing samples of biometric personal data for registration with the unified biometric system (UBS).

Identifying the model of a cell-phone with which an audio recording is made serves an important forensic purpose. In this paper, we propose a channel attention mechanism based on subband awareness to focus on the most relevant parts of the frequency bands to produce more efficient feature representations for cell-phone recognition task. A multi-stream network is introduced to fully exploit difference among frequency bands that provide critical clues to identify the fingerprints from the built-in microphones, and thus is able to recognize cell-phones from different manufacturers and even different models from the same manufacturer. The effectiveness of the proposed design is validated by a collection of speaker-independent audio recordings from 20 models of cell-phones made by 5 major manufacturers. In particular, the fusion of data augmentation and attention strategy greatly increases the robustness of the scheme when additive noise is present in the recorded audios. Finally, the salient regions infer the critical subbands for the recognition of different cell-phone models.

Automatic Chord Estimation (ACE) is a fundamental task in Music Information Retrieval (MIR) and has applications in both music performance and MIR research. The task consists of segmenting a music recording or score and assigning a chord label to each segment. Although it has been a task in the annual benchmarking evaluation MIREX for over 10 years, ACE is not yet a solved problem, since performance has stagnated and modern systems have started to tune themselves to subjective training data. We propose DECIBEL, a new ACE system that exploits heterogeneous musical representations, specifically MIDI and tab files, to improve audio-based ACE methods. From an audio file and a set of MIDI and tab files corresponding to the same popular music song, DECIBEL first estimates chord sequences. For audio, state-of-the-art audio ACE methods are used. MIDI files are aligned to the audio, followed by a MIDI chord estimation step. Tab files are transformed into untimed chord sequences and then aligned to the audio. Next, DECIBEL uses data fusion to integrate all estimated chord sequences into one final output sequence. DECIBEL improves all tested state-of-the-art ACE methods by 0.5 to 13.6 percentage points. This result shows that the integration of crowd-sourced annotations from heterogeneous symbolic music representations using data fusion is a suitable strategy for addressing challenging MIR tasks such as ACE.

This article describes a method for determining the emotional state of the speaker based on computer analysis of the parameters of the sound wave of an audio recording.

With the widespread use of voice-controlling services and devices, the research for developing robust and fast systems for automatic speaker identification had accelerated. In this paper, we present a Convolutional Neural Network (CNN) architecture for text-independent automatic speaker identification. The primary purpose is to identify a speaker, among many others, using a short speech segment. Most of the current researches focus on deep CNNs, which were initially designed for computer vision tasks. Besides, most of the existing speaker identification methods require audio samples longer than 3 seconds in the query phase for achieving a high accuracy. We created a CNN architecture appropriate for voice and speech-related classification tasks. We propose an optimum model that achieves 99.5% accuracy on LibriSpeech and 90% accuracy on VoxCeleb 1 dataset using only 1-second test utterances in our experiments.

Deep Neural Network (DNN)-based mask estimation approach is an emerging algorithm in monaural speech enhancement. It is used to enhance speech signals from the noisy background by calculating either speech or noise dominant in a particular frame of the noisy speech signal. It can construct complex models for nonlinear processing. However, the limitation of the DNN-based mask algorithm is a generalization of the targeted population. Past research works focused on their target dataset because of time consumption for the audio recording session. Thus, in this work, different recording conditions were used to study the performance of the DNN-based mask estimation approach. The findings revealed that different language test dataset, as well as different conditions, may not give large impact in speech enhancement performance since the algorithm only learn the noise information. But, the performance of speech enhancement is promising when the trained model has been designed properly, especially given the less sample variations in the input dataset involved during the training session.

This paper tackles the problem of the heavy dependence of clean speech data required by deep learning based audio-denoising methods by showing that it is possible to train deep speech denoising networks using only noisy speech samples. Conventional wisdom dictates that in order to achieve good speech denoising performance, there is a requirement for a large quantity of both noisy speech samples and perfectly clean speech samples, resulting in a need for expensive audio recording equipment and extremely controlled soundproof recording studios. These requirements pose significant challenges in data collection, especially in economically disadvantaged regions and for low resource languages. This work shows that speech denoising deep neural networks can be successfully trained utilizing only noisy training audio. Furthermore it is revealed that such training regimes achieve superior denoising performance over conventional training regimes utilizing clean training audio targets, in cases involving complex noise distributions and low Signal-to-Noise ratios (high noise environments). This is demonstrated through experiments studying the efficacy of our proposed approach over both real-world noises and synthetic noises using the 20 layered Deep Complex U-Net architecture., Published in Interspeech 2021 ( See https://www.isca-speech.org/archive/interspeech_2021/kashyap21_interspeech.html ). 5 pages, 2 figures, 1 table

In the last two decades, statistical clustering models have emerged as a dominant model of how infants learn the sounds of their language. However, recent empirical and computational evidence suggests that purely statistical clustering methods may not be sufficient to explain speech sound acquisition. To model early development of speech perception, the present study used a two-layer network trained with Rescorla-Wagner learning equations, an implementation of discriminative, error-driven learning. The model contained no a priori linguistic units, such as phonemes or phonetic features. Instead, expectations about the upcoming acoustic speech signal were learned from the surrounding speech signal, with spectral components extracted from an audio recording of child- directed speech as both inputs and outputs of the model. To evaluate model performance, we simulated infant responses in the high-amplitude sucking paradigm using vowel and fricative pairs and continua. The simulations were able to discriminate vowel and consonant pairs and predicted the infant speech perception data. The model also showed the greatest amount of discrimination in the expected spectral frequencies. These results suggest that discriminative error-driven learning may provide a viable approach to modelling early infant speech sound acquisition.

Speech recognition technology has been very popular and reached an advanced stage in computational linguistics that enables the recognition and transcription of spoken language into text by computers. However, the research on automatic speech recognition (ASR) for Dzongkha is the first of its kind and none has carried out any research and development on Dzongkha language; whereby making the data resource preparation information very limited. More than 11000 Dzongkha utterances of raw data are collected from Dzongkha Development Commission. The data wrangling is carried out on the limited data and the Dzongkha ASR corpus has been developed. In corpus, a total of 10566 usable utterances is prepared which includes Dzongkha text, audio recordings, and transcriptions. The pronunciation dictionary (lexicon) is generated which contains a total of 12605 unique Dzongkha words; and also, a phone list is recorded using Dzongkha phonemic inventory table for the representation of the smallest unit of the sound at the phoneme level. Such speech corpus is the primary requirement for developing the new speech recognition system for Dzongkha language.

Sound event detection (SED) takes on the task of identifying presence of specific sound events in a complex audio recording. SED has tremendous implications in video analytics, smart speaker algorithms and audio tagging. Recent advances in deep learning have afforded remarkable advances in performance of SED systems; albeit at the cost of extensive labeling efforts to train supervised methods using fully described sound class labels and timestamps. In order to address limitations in availability of training data, this work proposes a self-training technique to leverage unlabeled datasets in supervised learning using pseudo label estimation. This approach proposes a dual-term objective function: a classification loss for the original labels and expectation loss for pseudo labels. The proposed self training technique is applied to sound event detection in the context of the DCASE 2020 challenge, and reports a notable improvement over the baseline system for this task. The self-training approach is particularly effective in extending the labeled database with concurrent sound events.

Lyrics alignment in long music recordings can be memory exhaustive when performed in a single pass. In this study, we present a novel method that performs audio-to-lyrics alignment with a low memory consumption footprint regardless of the duration of the music recording. The proposed system first spots the anchoring words within the audio signal. With respect to these anchors, the recording is then segmented and a second-pass alignment is performed to obtain the word timings. We show that our audio-to-lyrics alignment system performs competitively with the state-of-the-art, while requiring much less computational resources. In addition, we utilise our lyrics alignment system to segment the music recordings into sentence-level chunks. Notably on the segmented recordings, we report the lyrics transcription scores on a number of benchmark test sets. Finally, our experiments highlight the importance of the source separation step for good performance on the transcription and alignment tasks. For reproducibility, we publicly share our code with the research community.

An experiment has been carried out for determination of source of digital audio recording using audio latency feature of mobile phones. The legal requirement of the source correspondence with the audio recordings in question has driven the necessity of such study. The evaluation is based on 300 audio recordings recorded with ten mobile handsets of MOTOROLA make but different model in three different format “3gp”, “wav” and “m4a” format (ten audio recording in each format) recorded in stereo signal at 16 kHz and in different recording sessions using EASY VOICE RECORDER application. Adobe audition 3.0 software is used for analysis. The analysis of audio latency has been carried out in a set of ten mobile handsets having multimedia capability. The result of this study shows that the audio latency of mobile handsets of same make but different model carry the specific feature of latency and such audio latency is robust for establishing the source of audio recordings.

This paper assesses the perceived listening quality of customer support waiting loops using both a subjective experiment as well as an objective perceptual assessment with POLQA (ITU-T Rec. P.863) and VISQOL. A modified version of the methodology defined in the ITU-T Rec. P.835 was derived and used in this paper to subjectively assess the listening speech quality, the listening audio quality as well as the overall listening quality of the waiting loop perceived by the user. It is expected that the perceived listening quality of the customer support waiting loop can influence the overall quality of the customer support communication service. The waiting loop consisted of a speech announcement and a music fragment. Both are assessed separately as well as overall. The results show that the degradations introduced by the investigated codecs and packet loss, representing typical degradations commonly seen in voice communication over telecommunication networks, seriously impact the subjectively perceived listening quality of the speech announcement and music fragment as well as the overall subjectively perceived listening quality of the waiting loop. As expected the listening quality of the music fragment is, in most of the cases, significantly lower than the listening speech quality of the announcement. The POLQA results show a high correlation between subjective and objective measurements for the listening speech and audio quality as well as the overall listening quality of the waiting loop. On the other hand, it is shown that the VISQOL model provides too low correlations between predicted objective and subjective scores for accurate prediction of the listening speech and audio quality as well as the overall listening quality of the waiting loop. © S. Hirzel Verlag · EAA.

The electric network frequency (ENF) signal is an environmental signature that can be captured in audiovisual recordings made in locations where there is electrical activity. This signal is influenced by the power grid in which the recording is made, and recent work has shown that it can be useful toward a number of forensics and security applications. An under-studied area of ENF research is the factors that can affect the capture of ENF traces in media recordings. Not all recordings made in the areas of electrical activity will carry prominent ENF traces, and the strengths by which the ENF traces are captured can vary from one recording to another. A thorough understanding of the factors that can affect the capture of ENF traces in recordings is essential to understanding the applicability of ENF-based approaches and can help inform related studies in the future. This paper carried out a study on such factors, with a focus on audio signals. The impact of the characteristics of an audio recorder and the environment and manner of recording on the intrinsically captured ENF are shown and analyzed.

Recently, end-to-end (E2E) neural text-to-speech systems, such as Tacotron2, have begun to surpass the traditional multi-stage hand-engineered systems, with both simplified system building pipelines and high-quality speech. With a unique encoder-decoder neural structure, the Tacotron2 system no longer needs separately learned text analysis front-end, duration model, acoustic model, and audio synthesis module. The key of such a system lies in the attention mechanism, which learns an alignment between the encoder and the decoder, serving as an implicit duration model bridging the text sequence and the acoustic sequence. However, attention learning suffers from low training efficiency and model instability problems, which hinder the E2E approaches from wide deployment. In this paper, we address the problems and propose a novel pre-alignment guided attention learning approach. Specifically, we inject handy prior knowledge-accurate phoneme durations-in the neural network loss function to bias the attention learning to the desired direction more accurately. The explicit time alignment between an audio recording and its corresponding phoneme sequence can be achieved by forced-alignment from an automatic speech recognizer (ASR). The experiments show that the proposed pre-alignment guided (PAG) attention approach can significantly improve training efficiency and model stability. More specifically, the PAG updated version of Tacotron2 can quickly obtain the attention alignment using only 500 (text, audio) pairs, which is apparently not possible for the original Tacotron2. A series of subjective experiments also show that the PAG-Tacotron2 approach can synthesize more stable and natural speech.

Stereophonic audio, especially binaural audio, plays an essential role in immersive viewing environments. Recent research has explored generating visually guided stereophonic audios supervised by multi-channel audio collections. However, due to the requirement of professional recording devices, existing datasets are limited in scale and variety, which impedes the generalization of supervised methods in real-world scenarios. In this work, we propose PseudoBinaural, an effective pipeline that is free of binaural recordings. The key insight is to carefully build pseudo visual-stereo pairs with mono data for training. Specifically, we leverage spherical harmonic decomposition and head-related impulse response (HRIR) to identify the relationship between spatial locations and received binaural audios. Then in the visual modality, corresponding visual cues of the mono data are manually placed at sound source positions to form the pairs. Compared to fully-supervised paradigms, our binaural-recording-free pipeline shows great stability in cross-dataset evaluation and achieves comparable performance under subjective preference. Moreover, combined with binaural recordings, our method is able to further boost the performance of binaural audio generation under supervised settings., Accepted by CVPR 2021. Code, models, and demo video are available on our webpage: \

Several methods for synthetic audio speech generation have been developed in the literature through the years. With the great technological advances brought by deep learning, many novel synthetic speech techniques achieving incredible realistic results have been recently proposed. As these methods generate convincing fake human voices, they can be used in a malicious way to negatively impact on today’s society (e.g., people impersonation, fake news spreading, opinion formation). For this reason, the ability of detecting whether a speech recording is synthetic or pristine is becoming an urgent necessity. In this work, we develop a synthetic speech detector. This takes as input an audio recording, extracts a series of hand-crafted features motivated by the speech-processing literature, and classify them in either closed-set or open-set. The proposed detector is validated on a publicly available dataset consisting of 17 synthetic speech generation algorithms ranging from old fashioned vocoders to modern deep learning solutions. Results show that the proposed method outperforms recently proposed detectors in the forensics literature.

A hearable device is a promising Life-Tech device realizing hand-free and eye-free information retrieval. Because its output comprises voices and sounds, we need to clarify the performance of simultaneously listening to multiple voices to develop an informative device. In this study, a word intelligibility test was conducted to examine the effects of sound reproduction methods and word familiarity on the ability to listen to competing words. The experimental results show that both factors affect the performance significantly; a reproduction method with high spatial separation and usage of high-familiarity words are keys to achieve high performance.

International audience; Auralizations can be computed in a variety of ways as well as be rendered over different sound reproduction systems. They are used as a design tool in architectural projects and for fundamental studies on spatial perception and cognition, hence requiring reliability and confidence in the obtained results. This study assessed this reliability through auditory perception stability by comparing the perceived differences between two rendering systems for a given set of second-order Ambisonic auralizations: virtual loudspeaker binaural rendering over head-tracked headphones versus 32-loudspeaker rendering. Anechoic extracts of jazz pieces have been recorded and presented in various acoustic conditions over these two systems, evaluated on the following criteria: Readability, distance, listener envelopment (LEV), apparent source width (ASW), reverberance, and loudness. Results show that consistent significant differences between scene conditions are comparably perceived across the two systems. However, significant effects of the sound reproduction system were observed for ASW, LEV, and reverberance in some configurations. V

The article proves the concept that an automatic diagnosis of depression can be achieved using audio recordings of the individuals’ voices. DAIC-WOZ database was used as a data source. Audio and textual data were preprocessed and converted to a set of optimized parameters for two models. Appropriate Deep Learning models to detect depression in the transcripts of the audio recordings and voice quality features, were utilized. We created a text analysis model on a word-level using Natural Language Processing (NLP) techniques, and a voice quality analysis model on tense to breathy dimension. The text analysis model made its best performance with an F1-score equal to 0.8 (0.42) for non-depressed (depressed) individuals, while the voice quality model scored 0.76 (0.38). As a result, we had two models that would be implemented in a system for the diagnosis of depression.

Speaker counting is the task of estimating the number of people that are simultaneously speaking in an audio recording. For several audio processing tasks such as speaker diarization, separation, localization and tracking, knowing the number of speakers at each timestep is a prerequisite, or at least it can be a strong advantage, in addition to enabling a low latency processing. For that purpose, we address the speaker counting problem with a multichannel convolutional recurrent neural network which produces an estimation at a short-term frame resolution. We trained the network to predict up to 5 concurrent speakers in a multichannel mixture, with simulated data including many different conditions in terms of source and microphone positions, reverberation, and noise. The network can predict the number of speakers with good accuracy at frame resolution., 5 pages, 1 figure

Automated cough detection may provide valuable clinical information for monitoring a patient’s health condition. In this paper, we present a cough detection system that utilises an acoustic onset detector as a pre-processing step, aiming to detect impulsive patterns in the audio stream. In a subsequent step, discrimination of coughing events from other impulsive sounds is handled as a binary classification task. In contrast to existing works, the proposed cough discrimination models are trained and tested with heterogeneous data uploaded by different users to online audio repositories. In that way, our system achieves robust performance to a wide range of audio recording devices and to varying noise and/or reverberation conditions. Our evaluation results showed that a sensitivity in the order of 90% and a specificity in the order of 99% can be achieved in a domestic environment with the utilization of Long-Short-Term-Memory deep neural network architecture.

We propose a CNN-based approach to classify ten genres of ballroom dances given audio recordings, five latin and five standard, namely Cha Cha Cha, Jive, Paso Doble, Rumba, Samba, Quickstep, Slow Foxtrot, Slow Waltz, Tango and Viennese Waltz. We utilize a spectrogram of an audio signal and we treat it as an image that is an input of the CNN. The classification is performed independently by 5-seconds spectrogram segments in sliding window fashion and the results are then aggregated. The method was tested on following datasets: Publicly available Extended Ballroom dataset collected by Marchand and Peeters, 2016 and two YouTube datasets collected by us, one in studio quality and the other, more challenging, recorded on mobile phones. The method achieved accuracy 93.9%, 96.7% and 89.8% respectively. The method runs in real-time. We implemented a web application to demonstrate the proposed method.

Voice Activity Detection (VAD) is not easy task when the input audio signal is noisy, and it is even more complicated when the input is not even an audio recording. This is the case with Silent Speech Interfaces (SSI) where we record the movement of the articulatory organs during speech, and we aim to reconstruct the speech signal from this recording. Our SSI system synthesizes speech from ultrasonic videos of the tongue movement, and the quality of the resulting speech signals are evaluated by metrics such as the mean squared error loss function of the underlying neural network and the Mel-Cepstral Distortion (MCD) of the reconstructed speech compared to the original. Here, we first demonstrate that the amount of silence in the training data can have an influence both on the MCD evaluation metric and on the performance of the neural network model. Then, we train a convolutional neural network classifier to separate silent and speech-containing ultrasound tongue images, using a conventional VAD algorithm to create the training labels from the corresponding speech signal. In the experiments our ultrasound-based speech/silence separator achieved a classification accuracy of about 85% and an AUC score around 86%.

Speech Activity Detection (SAD), locating speech segments within an audio recording, is a main part of most speech technology applications. Robust SAD is usually more difficult in noisy conditions with varying signal-to-noise ratios (SNR). The Fearless Steps challenge has recently provided such data from the NASA Apollo-11 mission for different speech processing tasks including SAD. Most audio recordings are degraded by different kinds and levels of noise varying within and between channels. This paper describes the EML online algorithm for the most recent phase of this challenge. The proposed algorithm can be trained both in a supervised and unsupervised manner and assigns speech and non-speech labels at runtime approximately every 0.1 sec. The experimental results show a competitive accuracy on both development and evaluation datasets with a real-time factor of about 0.002 using a single CPU machine.

We present a novel multi-modal unspoken punctuation prediction system for the English language which combines acoustic and text features. We demonstrate for the first time, that by relying exclusively on synthetic data generated using a prosody-aware text-to-speech system, we can outperform a model trained with expensive human audio recordings on the unspoken punctuation prediction problem. Our model architecture is well suited for on-device use. This is achieved by leveraging hash-based embeddings of automatic speech recognition text output in conjunction with acoustic features as input to a quasi-recurrent neural network, keeping the model size small and latency low., Accepted to IEEE ICASSP 2021

In this study, the Improved Variable Mode Decomposition Method (IVMD) is proposed for the estimation of the audio melody in heterophonic works that constitute the general texture of Turkish maqam music. In our study, the fundamental frequencies of the records belonging to huzzam, kurdilihicazkar, ussak, and rast maqams were estimated by using the IVMD method. Since the basis of the heterophonic texture is that the same melody is performed by more than one instrument, the estimated fundamental frequencies are more than one for each time window. After the multiple frequency estimation, in order to obtain the audio melody of the music recording and therefore a single frequency line, the selection of the frequencies belonging to the audio melody line from the fundamental frequencies was made. The study has been compared with the methods widely used in the analysis of polyphonic music works such as YIN and MELODIA. When the comparisons were evaluated on the basis of maqam and mixture according to the MIREX criteria, successful results were obtained with the IVMD method.

Human bowel sounds (BSs) deliver much useful information about gastrointestinal health status. In recent years, the utilization of advanced bio-acoustic sensors, such as the wired electronic stethoscopes and the wireless wearable sound recording patches, has enabled researchers to record, store and analyze the BSs in digitized manners, i.e., computerized bowel sound analysis. However, for collected BS recordings, how to effectively pick up the segments that contain BS events while ignoring those segments that only contain background noises remains difficult. Moreover, the BS segment recognition algorithms that are meant to be applied in the wearable healthcare scenes are further required to retain a low computation complexity. In this work, a light weighted BS recognizer based on convolutional neural networks (CNNs) is proposed for wearable systems. Specifically, the proposed recognizer firstly converts each one-dimensional segment into a two-dimensional spectrogram by calculating the Mel-frequency cepstrum coefficients (MFCCs) frame by frame and then passes the spectrogram through a CNN to infer the category of the segment. To validate the CNN-based BS recognizer, a 28 minutes of BS dataset that contains 955 BS-present segments and 725 BS-absent segments are made. Experimental results on the dataset show that the recognizer attains the 91.25% and 90.83% mean accuracies for the ‘not-across-subjects’ and ‘across-subjects’ validation, respectively. Moreover, compared with the state-of-the-art LSTM approach, the CNN-based BS recognizer is light-weighted with only 20.35k parameters, which is a quarter of that of the LSTM. Due to the lower model complexity, the CNN-based BS recognizer has the potential to be integrated into the gateways in a wearable system to secure the function that only the BS-present segments are allowed to be relayed to the remote. User privacy can be better protected in this way.

In the previous work, the flute robot could only play music that was programmed beforehand. Therefore, in this paper, firstly an algorithm based on fast Fourier transform for extracting rhythm and fundamental frequency of noiseless music was proposed. What’s more, the music recording usually contains negative segment at the beginning and there is always low frequency noise existing in the recording, which will disturb the analysis of rhythm and fundamental frequency. Hence, an algorithm for eliminating the negative segment and filtering noise was given. The results show that the algorithm can extract the rhythm and fundamental frequency precisely and eliminate the negative segment and filtering noise efficiently.

Passive health monitoring has been introduced as a solution for continuous diagnosis and tracking of subjects' condition with minimal effort. This is partially achieved by the technology of passive audio recording although it poses major audio privacy issues for subjects. Existing methods are limited to controlled recording environments and their prediction is significantly influenced by background noises. Meanwhile, they are too compute-intensive to be continuously running on smart phones. In this paper, we implement an efficient and robust audio privacy preserving method that profiles the background audio to focus only on audio activities detected during recording for performance improvement, and to adapt to the noise for more accurate speech segmentation. We analyze the performance of our method using audio data collected by a smart watch in lab noisy settings. Our obfuscation results show a low false positive rate of 20% with a 92% true positive rate by adapting to the recording noise level. We also reduced model memory footprint and execution time of the method on a smart phone by 75% and 62% to enable continuous speech obfuscation.

The world around us changes and digital is entering in all aspects of our lives. One aspect that is on the verge of changing is the ‘analog’ signed paper contracts. A lot of contracts are made over telephone nowadays and the audio recording of the discussion about the agreement it can be seen as a ‘digital’ contract. One problem arises, the authenticity of the recording. One side of the contract may rule later that the recording has been tampered and a ‘no’ or a ‘yes’ has been deleted or replaced. In this paper a watermarking algorithm is proposed for obtaining audio recordings which can be 100% proven as authentic or fake. This way no side of an over phone agreement can argue that the recording is altered. A simple check will prove 100% the authenticity of the recording. This way, cut and paste edits of more than 5 ms are easily detected.

The goal of acoustic matching is to transform an audio recording made in one acoustic environment to sound as if it had been recorded in a different environment, based on reference audio from the target environment. This paper introduces a deep learning solution for two parts of the acoustic matching problem. First, we characterize acoustic environments by mapping audio into a low-dimensional embedding invariant to speech content and speaker identity. Next, a waveform-to-waveform neural network conditioned on this embedding learns to transform an input waveform to match the acoustic qualities encoded in the target embedding. Listening tests on both simulated and real environments show that the proposed approach improves on state-of-the-art baseline methods.

Binaural reproduction can be used in listening experiments under real-life conditions to achieve a high realism and good reproducibility. In recent years a clear trend to more individual reproduction can be observed as the ability to measure individual head-related-transfer-functions (HRTFs) is becoming more widespread. The question of the accuracy and reproduction methods needed for a realistic playback however has not been sufficiently answered. To evaluate an appropriate approach for binaural reproduction via headphones different head-related-transfer-functions (HRTFs) and reproduction methods were compared in this paper. In a listening test eleven explicitly trained participants were asked to localize eleven sound sources positioned in the right hemisphere using the proximal pointing method. Binaural stimuli based on individually measured HRTFs were compared to those of an artificial head in a static reproduction of stimuli and in three dynamic reproduction methods of different resolutions (5°, 2.5° and 1°). Unsigned errors in azimuth and elevation as well as front-back-confusions and in-head-localization were observed. Dynamic reproduction of any resolution applied turned out fundamental for a reduction of undesired front-back-confusions and in-head-localization. Individually measured HRTFs showed a smaller effect on localization accuracy compared to the influence of dynamic sound reproduction. They were mainly observed to reduce the front-back-confusion rate.

Manipulating speech audio recordings through splicing is a task within everyone’s reach. Indeed, it is very easy to collect through social media multiple audio recordings from well-known public figures (e.g., actors, politicians, etc.). These can be cut into smaller excerpts that can be concatenated in order to generate new audio content. As a fake speech from a famous person can be used for fake news spreading and negatively impact on the society, the ability of detecting whether a speech recording has been manipulated is a task of great interest in the forensics community. In this work, we focus on speech audio splicing detection and localization. We leverage the idea that distinct recordings may be acquired in different environments, which are typically characterized by distinctive reverberation cues. Exploiting this property, our method estimates inconsistencies in the reverberation time throughout a speech recording. If reverberation inconsistencies are detected, the audio track is tagged as manipulated and the splicing point time instant is estimated.

Today, the Internet is the only medium through which songs are accessed by the majority, resulting in myriad of songs and artists on the Internet. The only effective way to search music on the Internet still requires at least some of the details about a song or its artist. But, what if we don’t know any such detail about the music? What if we heard it somewhere and know only how to sing or hum? Humming-based song recognition (HBSR) addresses this conundrum. It is a system which takes the audio file or audio recording as input and predicts the song present in the database based on the humming sound analysed. The fundamental ideas of HBSR, its implementation and some techniques to improve its performance are introduced in this paper.

Singing voice detection, also referred to as vocal detection (VD), aims at automatically identifying the regions in a music recording where at least one person sings. It is highly challenging due to the timbral and expressive richness of the human singing voice, as well as the practically endless variety of interfering instrumental accompaniment. Additionally, certain instruments have an inherent risk of being misclassified as vocals due to similarities of the sound production system. In this paper, we present a machine learning approach that is based on our previous work for VD, which is specifically designed to deal with those challenging conditions. The contribution of this paper is threefold: First, we present a new method for VD that passes a compact set of features to a long short-term memory recurrent neural network classifier that obtains state-of-the-art results. Second, we thoroughly evaluate the proposed method along with related approaches to really probe the weaknesses of the methods. In order to allow for such a thorough evaluation, we make a curated collection of datasets available to the research community. Finally, we focus on a specific problem that was not obvious and had not been discussed in the literature so far. The reason for this is precisely because limited evaluations had not revealed this as a problem: the lack of loudness invariance. We will discuss the implications of utilizing loudness-related features and show that our method successfully deals with this problem due to the specific set of features it uses.

In music, repetition is a fundamental concept to establish structure and create temporal relationships. Previous approaches to detecting repetition in music recordings have mainly focused on discovering repeated patterns of variable length and instrumentation at arbitrary locations. In this paper, we present a novel method for the discovery of repeated sung phrases in folk music recordings and, in particular, in oral music traditions, where written scores are usually unavailable. At a first stage, a segmentation algorithm partitions automatically generated note-level transcriptions of the singing melody into sections that correspond to the structural unit of a phrase. A clustering algorithm is then used to form clusters of phrases, where each cluster contains instances of the same melodic content. The clustering algorithm operates on the basis of a distance measure between melodic sequences and, to this end, various melodic distance measures are investigated. A detailed evaluation procedure is used to assess the performance of the algorithm on three different European music traditions and the influence of transcription and segmentation errors is investigated. The proposed system is shown to outperform the state-of-the-art in audio-based approaches to repeated phrase discovery for this task.

The aim of this study was to find an alternative method to prevent the conflict between the orangutan and human with the utilization of audio recording as biofence. The location of this study was in Leuser National Park on Bukit Lawang, Langkat, North Sumatra using 16 orangutans as the subject of this study. The method used in this study was a simulation and playback experiment which were using long call, spontaneous call, predator voice, and siren sound in the audio recording of non-alpha orangutans. The respond was observed using binocular and recorded using camera recorder. The data obtained were analyzed using Krustal Wallist test. The result showed that 87% of orangutan showed an interest in the attractor, on the other hand, 13% showed disinterest to the attractor. The facial expression shown by orangutan was fear, submission, aggression, and worry. The statistical analyzing result showed that long call and the spontaneous call was very significant (P0.01), while the predator and siren voice were insignificant (P0.05). The result can be concluded that long call and spontaneous calls showing the effective result to the alteration of territorial behavior on orangutan and have a potential as a biofence that is used as an alternative method to prevent the conflict between orangutan and human.

The computation of fast Fourier transform (FFT) plays a vital role in all fields of science and engineering, especially in the medical domain. Spectral results of the data convey more information to the clinicians to take handy decisions during the diagnosis. If a problem happens during the computation of FFT, the incorrect results misinterpret the information and hence the suggestions and decisions by practitioners would be the useless one moreover, it may yield harmful result to the patients. This paper deals with how to recover the spectrum from the faulty one with the heart sound case reports. The heart sound recording is the phonocardiogram (PCG) signal. Stethoscopes are commonly used to hear the heart sound for diagnosis. The two PCG signals of normal and abnormal subjects in each were analyzed in this paper. The normal subject produce clear lub and dub sounds, where the abnormal subject produce a kind of whistling or swishing sound in middle which may be due to the problem of septal defect in a heart. This paper analyzes four PCG signals spectrally with fault and the recovered ones along with the execution time results in each using Matlab ${\mathrm {R2016b}}$ tool.

Here is demonstrated Rivener , a mask-less oral and nasal audio recorder and air flow estimation system. This system records audio and low-frequency pseudo-sound from the nares and mouth. The system does not interfere with speech intelligibility, and minimally interferes with visual observation of the speaker. From these recordings, nasalance (a ratio of oral and nasal sound energy), oral air flow, and nasal air flow patterns may be estimated, all while allowing effective clinical observation. The first demonstration is a case-study comparison of the difference between hearing-impaired (HI) speech and non-impaired (NI) speech. Rivener records standard features of HI speech such as: (i) Atypically high or low speech amplitude; (ii) fundamental frequency (pitch) making individual words into intonational phrases; (iii) speech segment substitution; (iv) hypernasalance; (v) atypical air flow in HI speech, including low air flow during plosive release. The second demonstration is a comparison of Rivener and the Rothenberg NAS-1's ability to record nasalance among 26 New Zealand English speakers. The NAS-1 can differentiate low, medium, and high nasalance passages, whereas Rivener differentiates only medium and high nasalance passages consistently.

The current paper examines how native and non-native listeners of New Zealand English differ in terms of speech intelligibility in noise in a number of room acoustics reproduced by a first-order Ambisonics-based sound reproduction system. Speech intelligibility test was conducted under three room acoustics environments (living room, lecture theatre and church) using the sound reproduction system, where a pink noise masker was played from one of five azimuthal angles (0, 45, 90, 135, 180°) while the target speech was always played from 0°. We found significant two-way interactions between language nativeness and speech-noise separation, language nativeness and room acoustics, as well as between room acoustics and speech-noise separation. This suggests that native and non-native listeners respond differently to the virtually reproduced acoustic environments and they benefit from spatial release from masking in a different manner. Post-hoc results showed the native listeners performing significantly better than their non-native counterparts for all the angles of speech-noise separation and the room acoustics.

Heart sound analysis plays an important role in the auscultative diagnosis process to detect the presence of cardiovascular diseases. In this paper we propose a novel parametric heart sound model that accurately represents normal and pathological cardiac audio signals, also known as phonocardiograms (PCG). The proposed model considers that the PCG signal is formed by the sum of two parts: one of them is deterministic and the other one is stochastic. The first part contains most of the acoustic energy. This part is modeled by the Matching Pursuit (MP) algorithm, which performs an analysis-synthesis procedure to represent the PCG signal as a linear combination of elementary waveforms. The second part, also called residual, is obtained after subtracting the deterministic signal from the original heart sound recording and can be accurately represented as an autoregressive process using the Linear Predictive Coding (LPC) technique. We evaluate the proposed heart sound model by performing subjective and objective tests using signals corresponding to different pathological cardiac sounds. The results of the objective evaluation show an average Percentage of Root-Mean-Square Difference of approximately 5% between the original heart sound and the reconstructed signal. For the subjective test we conducted a formal methodology for perceptual evaluation of audio quality with the assistance of medical experts. Statistical results of the subjective evaluation show that our model provides a highly accurate approximation of real heart sound signals. We are not aware of any previous heart sound model rigorously evaluated as our proposal.