Search

Abstract In this paper, we undertake a novel two-pronged investigation into the human recognition of deepfake speech, addressing critical gaps in existing research. First, we pioneer an evaluation of the impact of prior information on deepfake recognition, setting our work apart by simulating real-world attack scenarios where individuals are not informed in advance of deepfake exposure. This approach simulates the unpredictability of real-world deepfake attacks, providing unprecedented insights into human vulnerability under realistic conditions. Second, we introduce a novel metric to evaluate the quality of deepfake audio. This metric facilitates a deeper exploration into how the quality of deepfake speech influences human detection accuracy. By examining both the effect of prior knowledge about deepfakes and the role of deepfake speech quality, our research reveals the importance of these factors, contributes to understanding human vulnerability to deepfakes, and suggests measures to enhance human detection skills.

The development of digital radio, observed in recent years, and the advantages offered by this medium are resulting in the expansion of the audience. In order to ensure the proper quality of broadcasting, it is necessary to monitor this quality. The assessment based on listening tests is very expensive and organizationally inconvenient. The development of methods for objective evaluation of signals makes it possible to monitor the quality of transmitted content, without the need for troublesome procedures associated with subjective evaluation. The good correlation of subjective evaluation and objective measurements in the transmission of perceptually encoded signals, reported in the literature, may allow for random quality monitoring. The authors decided to test the feasibility of using two objective evaluation methods to assess the quality of the speech signal transmitted on digital radio by comparing the results obtained with the subjective evaluation. The paper presents the results of objective measurements of speech quality transmitted via Digital Audio Broadcasting+ in Wroclaw agglomeration. Measurements have been done in various city sites in order to determine the influence of the location on speech quality. The obtained results of the assessment performed by the use of two methods of testing: PESQ and POLQA, allowed to find a correlation between the objective and subjective results of quality evaluations. The method of objective testing may be used for monitoring the quality of signals in Digital Audio Broadcasting networks especially spoken broadcasts.

In this paper, we undertake a novel two-pronged investigation into the human recognition of deepfake speech, addressing critical gaps in existing research. First, we pioneer an evaluation of the impact of prior information on deepfake recognition, setting our work apart by simulating real-world attack scenarios where individuals are not informed in advance of deepfake exposure. This approach simulates the unpredictability of real-world deepfake attacks, providing unprecedented insights into human vulnerability under realistic conditions. Second, we introduce a novel metric to evaluate the quality of deepfake audio. This metric facilitates a deeper exploration into how the quality of deepfake speech influences human detection accuracy. By examining both the effect of prior knowledge about deepfakes and the role of deepfake speech quality, our research reveals the importance of these factors, contributes to understanding human vulnerability to deepfakes, and suggests measures to enhance human detection skills. [ABSTRACT FROM AUTHOR]

The development of digital radio, observed in recent years, and the advantages offered by this medium are resulting in the expansion of the audience. In order to ensure the proper quality of broadcasting, it is necessary to monitor this quality. The assessment based on listening tests is very expensive and organizationally inconvenient. The development of methods for objective evaluation of signals makes it possible to monitor the quality of transmitted content, without the need for troublesome procedures associated with subjective evaluation. The good correlation of subjective evaluation and objective measurements in the transmission of perceptually encoded signals, reported in the literature, may allow for random quality monitoring. The authors decided to test the feasibility of using two objective evaluation methods to assess the quality of the speech signal transmitted on digital radio by comparing the results obtained with the subjective evaluation. The paper presents the results of objective measurements of speech quality transmitted via Digital Audio Broadcasting+ in Wroclaw agglomeration. Measurements have been done in various city sites in order to determine the influence of the location on speech quality. The obtained results of the assessment performed by the use of two methods of testing: PESQ and POLQA, allowed to find a correlation between the objective and subjective results of quality evaluations. The method of objective testing may be used for monitoring the quality of signals in Digital Audio Broadcasting networks especially spoken broadcasts. [ABSTRACT FROM AUTHOR]

Traditional recurrent neural networks (RNNs) encounter difficulty in capturing long-term temporal dependencies. However, lightweight recurrent models for speech enhancement are important to improve noisy speech, while being computationally efficient and able to capture long-term temporal dependencies efficiently. This study proposes a lightweight hourglass-shaped model for speech enhancement (SE) and automatic speech recognition (ASR). Simple recurrent units (SRU) with skip connections are implemented where attention gates are added to the skip connections, highlighting the important features and spectral regions. The model operates without relying on future information that is well-suited for real-time processing. Combined acoustic features and two training objectives are estimated. Experimental evaluations using the short time speech intelligibility (STOI), perceptual evaluation of speech quality (PESQ), and word error rates (WERs) indicate better intelligibility, perceptual quality, and word recognition rates. The composite measures further confirm the performance of residual noise and speech distortion. With the TIMIT database, the proposed model improves the STOI and PESQ by 16.21% and 0.69 (31.1%) whereas with the LibriSpeech database, the model improves STOI by 16.41% and PESQ by 0.71 (32.9%) over the noisy speech. Further, our model outperforms other deep neural networks (DNNs) in seen and unseen conditions. The ASR performance is measured using the Kaldi toolkit and achieves 15.13% WERs in noisy backgrounds.

Traditional recurrent neural networks (RNNs) encounter difficulty in capturing long-term temporal dependencies. However, lightweight recurrent models for speech enhancement are important to improve noisy speech, while being computationally efficient and able to capture long-term temporal dependencies efficiently. This study proposes a lightweight hourglass-shaped model for speech enhancement (SE) and automatic speech recognition (ASR). Simple recurrent units (SRU) with skip connections are implemented where attention gates are added to the skip connections, highlighting the important features and spectral regions. The model operates without relying on future information that is well-suited for real-time processing. Combined acoustic features and two training objectives are estimated. Experimental evaluations using the short time speech intelligibility (STOI), perceptual evaluation of speech quality (PESQ), and word error rates (WERs) indicate better intelligibility, perceptual quality, and word recognition rates. The composite measures further confirm the performance of residual noise and speech distortion. With the TIMIT database, the proposed model improves the STOI and PESQ by 16.21% and 0.69 (31.1%) whereas with the LibriSpeech database, the model improves STOI by 16.41% and PESQ by 0.71 (32.9%) over the noisy speech. Further, our model outperforms other deep neural networks (DNNs) in seen and unseen conditions. The ASR performance is measured using the Kaldi toolkit and achieves 15.13% WERs in noisy backgrounds. [ABSTRACT FROM AUTHOR]

Smartphones and tablets are part of everyday family life. Experts who work with children or deal with various aspects of early childhood development warn of the resulting harmful consequences. It is also feared that research on the impact of the use of smartphones and tablets on children's development cannot keep up with rapid technological development. Bearing these facts in mind, in this research we tried to evaluate the quality of speech, sensorimotor development and socio-emotional behavior in children in the most sensitive period of their growth, preschool age. The research sample consisted of 40 participants, 28 boys and 12 girls, aged 5 to 6 years. Only children whose parents declared that their child spends more than three hours a day in front of the screen were included in the sample. The research used an adapted test for assessing the articulation abilities of children, the Global Articulation Test and the Scale for assessing the psychophysiological abilities of children from 0 to 7 years of age. The results of this research show that there are differences in the quality of speech in preschool children. Statistically significant differences in sensorimotor development and socioemotional behavior in preschool children were obtained by applying one-factor multivariate ANOVA. Knowledge and understanding of the harmful effects of screens on the functioning and development of children in the immediate environment is essential for taking measures to control screen exposure by parents and educators in order to prevent further regressions in the development of speech, language, sensorimotor and socio-emotional maturity and thereby stimulate optimal early development.

In the current field of air traffic control speech, there is a lack of effective objective speech quality evaluation methods. This paper proposes a new network framework based on ResNet–BiLSTM to address this issue. Firstly, the mel-spectrogram of the speech signal is segmented using the sliding window technique. Next, a preceding feature extractor composed of convolutional and pooling layers is employed to extract shallow features from the mel-spectrogram segment. Then, ResNet is utilized to extract spatial features from the shallow features, while BiLSTM is used to extract temporal features, and these features are horizontally concatenated. Finally, based on the concatenated spatiotemporal features, the final speech quality score is computed using fully connected layers. We conduct experiments on the air traffic control speech database and compare the objective scoring results with the subjective scoring results. The experimental results demonstrate that the proposed method has a high correlation with the mean opinion score (MOS) of air traffic control speech. [ABSTRACT FROM AUTHOR]

Speech quality and speech intelligibility can vary dramatically across the wide range of currently available telecommunications systems, devices, and operating environments. This creates a strong demand for efficient real-time measurements of quality and intelligibility. Wideband Audio Waveform Evaluation Networks (WAWEnets) are convolutional neural networks (CNNs) that operate directly on wideband audio waveforms in order to produce evaluations of those waveforms. In the present work these evaluations give qualities of telecommunications speech (e.g., noisiness, intelligibility, overall speech quality). WAWEnets are no-reference networks because they do not require “reference” (original or undistorted) versions of the waveforms they evaluate. Our initial 2020 WAWEnet publication introduces four WAWEnets and each emulates the output of an established full-reference speech quality or intelligibility estimation algorithm. We have updated the WAWEnet architecture to be more efficient and effective. Here we present a single WAWEnet that closely tracks seven different quality and intelligibility values with per-segment correlations in the range of 0.92 to 0.96. We create a second network that additionally tracks four subjective speech quality dimensions. We offer a third network that focuses on just subjective quality scores and achieves a per-segment correlation of 0.97. The performance of our WAWEnet architecture compares favorably to models with orders-of-magnitude more parameters and computational complexity. This work has leveraged 334 hours of speech in 13 languages, more than two million full-reference target values, and more than 93,000 subjective mean opinion scores. We also interpret the operation of WAWEnets and identify the key to their operation using the language of signal processing: ReLUs strategically move spectral information from non-DC components into the DC component. The DC values of 96 output signals define a vector in a 96-D latent space, and this vector is then mapped to a quality or intelligibility value for the input waveform.

In October 2018, local digital radio was launched to cover the agglomeration of Wroclaw. The implementation of this undertaking required many tests, including qualitative ones, that refer to both music and speech. This paper presents the results of subjective tests based on the evaluation of speech quality of signals recorded at various points in Wroclaw. Measurements were carried out in accordance with the recommendations of the International Telecommunication Union as well as in ordinary acoustic conditions in listeners’ flats. The rating was made for male and female voices. The most important conclusion is that for speech signal assessment in meaning of the quality the test conditions do not influence the obtained results. The other fact confirmed in the experiment was that the receiving place of DAB+ signal in the Single-Frequency Network also does not affect the perceived voice quality.

The quality of speech degradeswhile communicating over Voice over Internet Protocol applications, for example, Google Meet, Microsoft Skype, and Apple FaceTime, due to different types of background noise present in the surroundings. It reduces human perceived Quality of Experience (QoE). Along this line, this article proposes a novel speech quality prediction metric that can meet human's desired QoE level. Our motivation is driven by the lack of evidence showing speech quality metrics that can distinguish different noise degradations before predicting the quality of speech. The quality of speech in noisy environments is improved by speech enhancement algorithms, and for measuring and monitoring the quality of speech, objective speech quality metrics are used. With the integration of these components, a novel no-reference context-aware QoE prediction metric (CAQoE) is proposed in this article, which initially identifies the context or noise type or degradation type of the input noisy speech signal and then predicts context-specific speech quality for that input speech signal. It will have of great importance in deciding the speech enhancement algorithms if the types of degradations causing poor speech quality are known along with the quality metric. Results demonstrate that the proposed CAQoE metric outperforms in different contexts as compared to the metric where contexts are not identified before predicting the quality of speech, even in the presence of limited size speech corpus having different contexts available from the NOIZEUS speech database. [ABSTRACT FROM AUTHOR]

Purpose: This study aimed to verify the intelligibility and quality of hearing aid-processed speech across companies through objective measures. The output of hearing aids programmed to different prescriptive formulae (Company fit, NAL NL 1, NAL NL 2, and DSLv5) across companies (Company I, Company II, Company III, Company IV, and Company V) were recorded and stored in the sound level meter connected to the manikin. Materials and Methods: Offline analysis of speech quality of the recorded speech was done using the Hearing Aid Speech Quality Index (HASQI), the intelligibilty was measured using the Hearing Aid Speech Perception Index (HASPI) and Speech Intelligibility Index (SII), and Long-Term Average Speech Spectrum (LTASS) was used to measure the spectral composition of speech. Results: When the companies are compared, the hearing aid of Company V showed significantly higher SII values for all prescriptive formulae indicative of better speech intelligibility provided by Company V hearing aids. In terms of speech quality, all the hearing aids showed similar HASQI values. Conclusions: The reasons for each of these findings are highlighted and recommendations for best practices on the use of objective measures for verifying the hearing aid output in clinical setups are suggested. [ABSTRACT FROM AUTHOR]

Machine learning-based supervised single-channel speech enhancement has achieved considerable research interest over conventional approaches. In this paper, an extended Restricted Boltzmann Machine (RBM) is proposed for the spectral masking-based noisy speech enhancement. In conventional RBM, the acoustic features for the speech enhancement task are layerwise extracted and the feature compression may result in loss of vital information during the network training. In order to exploit the important information in the raw data, an extended RBM is proposed for the acoustic feature representation and speech enhancement. In the proposed RBM, the acoustic features are progressively extracted by multiple-stacked RBMs during the pre-training phase. The hidden acoustic features from the previous RBM are combined with the raw input data that serve as the new inputs to the present RBM. By adding the raw data to RBMs, the layer-wise features related to the raw data are progressively extracted, that is helpful to mine valuable information in the raw data. The results using the TIMIT database showed that the proposed method successfully attenuated the noise and gained improvements in the speech quality and intelligibility. The STOI, PESQ and SDR are improved by 16.86%, 25.01% and 3.84dB over the unprocessed noisy speech.

Nowadays, deep neural network has become the prime approach for enhancing speech signals as it yields good results compared to the traditional methods. This paper describes the transformation in the enhanced speech signal by applying the deep convolutional neural network (Deep CNN), which can model nonlinear relationships and compare it with the Wiener filtering method, which is the best technique for speech enhancement among the traditional methods. Denoising is performed in the frequency domain and converted back to the time domain to analyze performance metrics such as speech quality and speech intelligibility. The speech quality is analyzed based on the signal to noise ratio (SNR) and perceptual evaluation of speech quality (PESQ). Speech intelligibility is analyzed by short-time objective intelligibility (STOI). Both the methods evaluated the denoised speech, and the analysis made on the results shows that the SNR of the conventional Wiener filtering method is much improved when compared with Deep CNN. However, the PESQ and STOI of Deep CNN-based enhanced speech outperform the Wiener filtering method. The performance metrics indicate that Deep CNN achieves better results than the conventional technique. [ABSTRACT FROM AUTHOR]

This paper proposes a robust signal-to-noise ratio (SNR) estimation technique from continuous speech based on the non-negative frequency-weighted energy operator using the envelope of derivative of the speech signal. The proposed SNR evaluation method is implemented in two phases. The first phase of SNR estimation consists of speech region identification by glottal activity detection (GAD) along with two pre-processing and post-processing methods. Speech enhancement with background subtraction is the pre-processing and obstruent detection with duration modification is the post-processing modules to GAD, which improves the speech and non-speech region detection performance. The next stage comprises estimation of SNR from instantaneous energy contour obtained from the envelope of the derivative of the speech signal, which is based on non-negative frequency-weighted energy estimation. The ratio of instantaneous energy of speech region to non-speech region is considered as the estimated segmental SNR. The final SNR is calculated by averaging segmental SNR and compared with true SNR to get the estimation efficiency of the proposed method. The performance of the proposed method for SNR estimation from speech sample is evaluated using TIMIT, NOIZEUS, and TESDHE acoustic speech corpus. The TIMIT and NOIZEUS databases are used to evaluate the performance for a different type of noise and TESDHE database is used for analysis under a different mode of data. The result of our study shows that the proposed system provides accurate SNR estimation across different types of noise and different modes of signal for various signal levels and outperforms the other methods under observation. [ABSTRACT FROM AUTHOR]

To meet the human perceived quality of experience (QoE) while communicating over various Voice over Internet protocol (VoIP) applications, for example Google Meet, Microsoft Skype, Apple FaceTime, etc. a precise speech quality assessment metric is needed. The metric should be able to detect and segregate different types of noise degradations present in the surroundings before measuring and monitoring the quality of speech in real-time. Our research is motivated by the lack of clear evidence presenting speech quality metric that can firstly distinguish different types of noise degradations before providing speech quality prediction decision. To that end, this paper presents a novel non-intrusive speech quality assessment metric using context-aware neural networks in which the noise class (context) of the degraded or noisy speech signal is first identified using a classifier then deep neutral networks (DNNs) based speech quality metrics (SQMs) are trained and optimized for each noise class to obtain the noise class-specific (context-specific) optimized speech quality predictions (MOS scores). The noisy speech signals, that is, clean speech signals degraded by different types of background noises are taken from the NOIZEUS speech corpus. Results demonstrate that even in the presence of less number of speech samples available from the NOIZEUS speech corpus, the proposed metric outperforms in different contexts compared to the metric where the contexts are not classified before speech quality prediction. [ABSTRACT FROM AUTHOR]

In modern telecommunication systems the presence of noise in background degrades the overall intelligibility and quality of the speech signal. The problem of enhancing speech signals and reducing acoustic noise from the noisy environment using adaptive filtering algorithms with incorporation of blind source separation approach has drawn a particular attention in the recent past. In this paper a dual channel double backward distributive weighted adaptive filtering algorithm is proposed for speech quality enhancement. The proposed method has been evaluated using the objective measures such as Perceptual Evaluation of Speech Quality (PESQ) and Short Time Objective Intelligibility (STOI) in different noise setup and the results achieved indicate that this is a better method for speech quality improvement. [ABSTRACT FROM AUTHOR]

Because of their simple design structure, end-to-end deep learning (E2E-DL) models have gained a lot of attention for speech enhancement. A number of DL models have achieved excellent results in eliminating the background noise and enhancing the quality as well as the intelligibility of noisy speech. Designing resource-efficient and compact models during real-time processing is still a key challenge. In order to enhance the accomplishment of E2E models, the sequential and local characteristics of speech signal should be efficiently taken into consideration while modeling. In this paper, we present resource-efficient and compact neural models for end-to-end noise-robust waveform-based speech enhancement. Combining the Convolutional Encode-Decoder (CED) and Recurrent Neural Networks (RNNs) in the Convolutional Recurrent Network (CRN) framework, we have aimed at different speech enhancement systems. Different noise types and speakers are used to train and test the proposed models. With LibriSpeech and the DEMAND dataset, the experiments show that the proposed models lead to improved quality and intelligibility with fewer trainable parameters, notably reduced model complexity, and inference time than existing recurrent and convolutional models. The quality and intelligibility are improved by 31.61% and 17.18% over the noisy speech. We further performed cross corpus analysis to demonstrate the generalization of the proposed E2E SE models across different speech datasets. [ABSTRACT FROM AUTHOR]

Speech-recognition tests are an important component of audiology. However, the development of such tests can be time consuming. The aim of this study was to investigate whether a Text-To-Speech (TTS) system can reduce the cost of development, and whether comparable results can be achieved in terms of speech recognition and listening effort. For this, the everyday sentences of the German Göttingen sentence test were synthesized for both a female and a male speaker using a TTS system. In a preliminary study, this system was rated as good, but worse than the natural reference. Due to the Covid-19 pandemic, the measurements took place online. Each set of speech material was presented at three fixed signal-to-noise ratios. The participants' responses were recorded and analyzed offline. Compared to the natural speech, the adjusted psychometric functions for the synthetic speech, independent of the speaker, resulted in an improvement of the speech-recognition threshold (SRT) by approximately 1.2 dB. The slopes, which were independent of the speaker, were about 15 percentage points per dB. The time periods between the end of the stimulus presentation and the beginning of the verbal response (verbal response time) were comparable for all speakers, suggesting no difference in listening effort. The SRT values obtained in the online measurement for the natural speech were comparable to published data. In summary, the time and effort for the development of speech-recognition tests may be significantly reduced by using a TTS system. This finding provides the opportunity to develop new speech tests with a large amount of speech material. [ABSTRACT FROM AUTHOR]

In October 2018, local digital radio was launched to cover the agglomeration of Wroclaw. The implementation of this undertaking required many tests, including qualitative ones, that refer to both music and speech. This paper presents the results of subjective tests based on the evaluation of speech quality of signals recorded at various points in Wroclaw. Measurements were carried out in accordance with the recommendations of the International Telecommunication Union as well as in ordinary acoustic conditions in listeners' flats. The rating was made for male and female voices. The most important conclusion is that for speech signal assessment in meaning of the quality the test conditions do not influence the obtained results. The other fact confirmed in the experiment was that the receiving place of DAB+ signal in the Single-Frequency Network also does not affect the perceived voice quality. [ABSTRACT FROM AUTHOR]

Automatic speech emotion recognition (SER) has gained popularity over the last decade and numerous Challenges have emerged. While the latest Challenges have shown that deep neural networks achieve the best results, existing input features are still a bottleneck and cause severe performance degradation in realistic “in-the-wild” scenarios. In this paper, we propose two innovations to tackle this issue. First, we propose to combine the bag-of-audio-words methodology with modulation spectrum features for environmental robustness. Second, we take advantage of the inherent quality-awareness properties of modulation spectrum and propose the use of a quality feature as an additional feature to be used by the speech emotion recognizer. Experiments are conducted with three multi-lingual speech datasets used in recent SER Challenges degraded by different noise sources and levels, and room reverberation. Experimental results show the proposed features i) consistently outperforming benchmark systems, ii) providing complementary information to classical features, hence improving performance with feature fusion, and iii) showing robustness against environment and language mismatch. Moreover, we show that when the proposed system is provided with quality information, further improvements are obtained. Overall, the proposed bag of modulation spectrum features are shown to be a promising candidate for “in-the-wild” SER. [ABSTRACT FROM AUTHOR]