Search

With the wide deployment of digital image capturing equipment, the need of denoising to produce a crystal clear image from noisy capture environment has become indispensable. In this work, a novel type-2 fuzzy-based filter is proposed for denoising images corrupted by impulse noise, especially for the high-density salt-and-pepper noise. It operates two stages, namely, type-2 fuzzy identifier and matrix completion denoiser. In the proposed method, the type-2 fuzzy identifier is first employed to identify and trim the entries contaminated by impulse noise in the data matrix from fuzzy system. Then, the trimmed data matrix is utilized to retrieve the noiseless data matrix with the matrix completion technology. Herein, a novel matrix completion technique is developed without $a priori$ rank information compared with its counterparts. Simulation results are presented which vividly show the denoised images obtained by the proposed method can achieve crystal clear image with strong structural integrity, and are showing good performance in terms of peak signal-to-noise ratio (PSNR).

Previous works in synthetic speech detection have focused on features based on magnitude or phase spectrum. In this study, to extract useful discriminative information for synthetic speech detection, the authors propose a feature based on magnitude-phase spectrum (MPS), combining magnitude- and phase-spectra information. The proposed feature is termed as constant-Q magnitude-phase coefficient (CMPC), which is obtained by combining constant-Q transform (CQT), MPS, uniform resampling, and discrete cosine transform. The CQT used in this study is a long-term window transform, which can provide the basis for CMPC to capture important artefacts of synthetic speech. Such artefacts are obtained using a unit selection algorithm, which have difficulties when based on the short-term window transform. Uniform resampling aims to convert MPS from the octave domain into the linear domain. The discrete cosine transform is used when extracting principal components to remove correlations among the feature dimensions. The experimental results on AVspoof and ASVspoof 2015 corpora show that CMPC performs better than some commonly used features based on magnitude or phase spectrum alone. Their system based on CMPC outperforms many known systems.

In this study, we propose a method for acoustic event diarization based on a feature of deep embedding and a clustering algorithm of integer linear programming. The deep embedding learned by deep auto-encoder network is used to represent the properties of different classes of acoustic events, and then the integer linear programming is adopted for merging audio segments belonging to the same class of acoustic events. Four kinds of TV/movie audios (21.5 h in total) are used as experimental data, including Sport, Situation comedy, Award ceremony, and Action movie. We compare the deep embedding with state-of-the-art features. Further, the clustering algorithm of integer linear programming is compared with other clustering algorithms adopted in previous works. Finally, the proposed method is compared to both supervised and unsupervised methods on four kinds of TV/movie audios. The results show that the proposed method is superior to other unsupervised methods based on agglomerative information bottleneck, Bayesian information criterion and spectral clustering, and is little inferior to the supervised method based on deep neural network in terms of acoustic event error.

The playback speech contains information from the environment, playback and recorder used. This work focuses on proposal of a novel normalization scheme, namely, low frequency frame-wise normalization (LFFN) as one of the modules in feature extraction process that is hypothesized to help in capturing the artifacts from the playback speech. It is based on low frequency bin processing that is performed frame-wise and hence its name. The constant-Q transform (CQT) based features are found to provide the benchmark results for detection of spoofing attacks. In this work, LFFN is combined with CQT to extract two new features from octave and linear power spectra, respectively. The first one is obtained by CQT, LFFN and octave segmentation that is referred to as constant-Q normalization segmentation coefficients (CQNSC). The latter uses conventional constant-Q cepstral coefficient (CQCC) and LFFN to obtain constant-Q normalization cepstral coefficients (CQNCC). The studies are performed on ASVspoof 2017 version 2.0 corpus that is designed for studying playback speech detection. The experimental results show the effectiveness of proposed LFFN with CQT based features. We obtain equal error rate of 10.63% and 10.31% for CQNSC and CQNCC features on the evaluation set of ASVspoof 2017 version 2.0 corpus, respectively.

Automatic speaker verification systems are prone to various spoofing attacks. The convolutional neural networks are found to be effective for detection of spoofing attacks. However, they lack spatial information and relationship of low-level features with the pooling layer. On the other hand, capsule networks use vectors to record spatial information and the probability of presence simultaneously. They are known to be effective for detection of forged images and videos. In this work, we study capsule networks for replay attack detection. We consider different input features to capsule network and study on recent ASVspoof 2019 physical access corpus. The studies suggest the proposed capsule network based system performs effectively and the performance is comparable to state-of-the-art single systems for replay attack detection.

Modern text-to-speech (TTS) and voice conversion (VC) systems produce natural sounding speech that questions the security of automatic speaker verification (ASV). This makes detection of such synthetic speech very important to safeguard ASV systems from unauthorized access. Most of the existing spoofing countermeasures perform well when the nature of the attacks is made known to the system during training. However, their performance degrades in face of unseen nature of attacks. In comparison to the synthetic speech created by a wide range of TTS and VC methods, genuine speech has a more consistent distribution. We believe that the difference between the distribution of synthetic and genuine speech is an important discriminative feature between the two classes. In this regard, we propose a novel method referred to as feature genuinization that learns a transformer with convolutional neural network (CNN) using the characteristics of only genuine speech. We then use this genuinization transformer with a light CNN classifier. The ASVspoof 2019 logical access corpus is used to evaluate the proposed method. The studies show that the proposed feature genuinization based LCNN system outperforms other state-of-the-art spoofing countermeasures, depicting its effectiveness for detection of synthetic speech attacks., Accepted for publication in Interspeech 2020

Most of the research on anti-spoofing countermeasures are specific to a type of spoofing attacks, where models are trained on data of a particular nature, either synthetic or replay. However, one does not have such leverage as there is no prior knowledge about the kind of spoofing attack in practice. Therefore, there is a requirement to assess the scope of generalized countermeasures for anti-spoofing. The ASVspoof 2019challengecoversboth synthetic as well as replay attacks, which makes the database suitable for such study. In this work, we consider widely popular constant-Q cepstral coefficient features along with two other promising front-ends that capture long-term signal characteristics to assess their scope as generalized countermeasures. Additionally, a comprehensive study is made across different editions of ASVspoof corpora to highlight the need of robust generalized countermeasures in unseen conditions.

In order to improve the performance of hand-crafted features to detect playback speech, two discriminative features, constant-Q variance-based octave coefficients and constant-Q mean-based octave coefficients, are proposed for playback speech detection in this work. They rely on our findings that variance-based modified log magnitude spectrum and mean-based modified log magnitude spectrum can enhance the discriminative power between genuine speech and playback speech. Then constant-Q variance-based octave coefficients (constant-Q mean-based octave coefficients) can be obtained by combining variance-based modified log magnitude spectrum (mean-based modified log magnitude spectrum), octave segmentation, and discrete cosine transform. Finally, constant-Q variance-based octave coefficients and constant-Q mean-based octave coefficients are evaluated on ASVspoof 2017 corpus version 2.0 and ASVspoof 2019 physical access, respectively. Experimental results show that variance-based modified log magnitude spectrum and mean-based modified log magnitude spectrum can produce discriminative features toward playback speech. Further results on the two databases show that constant-Q variance-based octave coefficients and constant-Q mean-based octave coefficients can perform better than some common features, such as mel frequency cepstral coefficients and constant-Q cepstral coefficients.

Considerable attention has been paid to acquisition device recognition over the past decade in the forensic community, especially in digital image forensics. In contrast, acquisition device clustering from speech recordings is a new problem that aims to merge the recordings acquired by the same device into a single cluster without having prior information about the recordings and training classifiers in advance. In this paper, we propose a method for mobile phone clustering from speech recordings by using a new feature of deep representation and a spectral clustering algorithm. The new feature is learned by a deep auto-encoder network for representing the intrinsic trace left behind by each phone in the recordings, and spectral clustering is used to merge recordings acquired by the same phone into a single cluster. The impacts of the structures of the deep auto-encoder network on the performance of the new feature are discussed. Different features are compared with one another. The proposed method is compared with others and evaluated under special conditions. The results show that the proposed method is effective under these conditions and the new feature outperforms other features.

This paper proposes an unsupervised method for analyzing speaker roles in multi-participant conversational speech. First, features for characterizing the differences of various roles are extracted from the outputs of speaker diarization. Then, an algorithm of role clustering based on the criterion of maximizing the inter-cluster distance without using any convergence threshold is proposed to obtain the number of roles and to merge the utterances belonging to the same role into one cluster. The contributions of different combinations of individual feature subsets are compared for the proposed method on the outputs from speaker diarization, and the combined feature subsets obtain higher F scores than the individual ones for clustering speaker roles. The impacts of both speaker diarization errors and feature dimensions on the performance of the proposed method are also discussed. Experiments are done on the outputs of both manual annotations and automatic speaker diarization to compare the proposed method with both the state-of-the-art clustering method and the supervised method. Evaluations show that the proposed method is superior to the previous clustering method and close to the conventional supervised method in terms of F scores under two different experimental conditions.

In current playback speech detection (PSD) Letter, commonly used features are often extracted from magnitude spectrum while phase spectrum information is not used. In order to extract more discriminative information for PSD, the idea of magnitude–phase spectrum (MPS) is proposed. Then a new feature based on MPS is proposed, namely constant-Q magnitude–phase octave coefficients (CMPOC). The experimental result on ASVspoof 2017 evaluation set using CMPOC indicates that: (i) the performance of CMPOC is better than features extracted not only from magnitude spectrum but also from MPS. (ii) CMPOC performs better than some commonly used features. (iii) Their system gives better performance than some known systems.

The constant-Q cepstral coefficients (CQCC) feature is one of the most effective feature in the field of spoof speech detection. The extraction of this feature involves constant-Q transform that captures long range information from the signal. It is followed by uniform resampling of the octave power spectrum to have linear power spectrum from which the CQCC features are obtained. However, we hypothesize that the information obtained from octave power spectrum is complementary with that captured by the linear spectrum. In this regard, we propose to combine the coefficients generated using both linear and octave power spectrum. The combined feature is referred to as extended CQCC (eCQCC) which is hypothesized to have better discriminative information for detection of spoof attacks. The studies for spoof detection are conducted on both synthetic voice and replay based spoofing attacks using ASVspoof 2015 and ASVspoof 2017 Version 2.0 database, respectively. The studies confirm that the proposed eCQCC feature consistently outperforms the baseline CQCC feature in all tasks.

Over the recent years, detection of spoofing attack has become pivotal for automatic speaker verification systems. The long-term information has been found to be useful for the detection of synthetic speech attacks. In this work, we focus on investigating high frequency information from the long-term constant-Q transform (CQT) based features. For this purpose, first we derive two novel feature, namely, inverted constant-Q coefficients (ICQC) and inverted constant-Q cepstral coefficients (ICQCC) using discrete cosine transform (DCT) on inverted octave power spectrum and inverted linear power spectrum, respectively. Further, we extend the work by use of overlapped block transform with DCT which can avoid the features extracted from full frequency-band are easily affected by local noise in some particular frequency bands. The two other novel features thus derived from inverted octave blocks and inverted linear blocks are referred to as constant-Q block coefficients (ICBC) and inverted constant-Q linear block coefficients (ICLBC), respectively. We study the proposed features on ASVspoof 2015, noisy ASVspoof 2015 and ASVspoof 2019 logical access corpora. For the proposed four features, our experimental results showed that ICQBC, ICBC, ICQC provide the best performance on ASVspoof 2015, noisy ASVspoof 2015 and ASVspoof 2019 logical access databases, respectively.

Speaker verification (SV) systems using deep neural network embeddings, so-called the x-vector systems, are becoming popular due to its good performance superior to the i-vector systems. The fusion of these systems provides improved performance benefiting both from the discriminatively trained x-vectors and generative i-vectors capturing distinct speaker characteristics. In this paper, we propose a novel method to include the complementary information of i-vector and x-vector, that is called generative x-vector. The generative x-vector utilizes a transformation model learned from the i-vector and x-vector representations of the background data. Canonical correlation analysis is applied to derive this transformation model, which is later used to transform the standard x-vectors of the enrollment and test segments to the corresponding generative x-vectors. The SV experiments performed on the NIST SRE 2010 dataset demonstrate that the system using generative x-vectors provides considerably better performance than the baseline i-vector and x-vector systems. Furthermore, the generative x-vectors outperform the fusion of i-vector and x-vector systems for long-duration utterances, while yielding comparable results for short-duration utterances., Comment: Accepted for publication at SLT 2018

Acquisition device clustering from speech recordings is a new and critical problem in the field of speech forensic, which aims at merging speech recordings acquired by the same device into one cluster without both pre-knowing prior information of the processed data and pre-training classifier. We propose a mobile phone clustering method, in which deep Gaussian supervector learned by deep neural network is used to represent the intrinsic trace left behind by mobile phone in speech recordings, and then spectral clustering technique is adopted to merge speech recordings acquired by the same mobile phone into one cluster. The performance of the proposed method is evaluated on a public corpus of speech recordings acquired by mobile phones. The results show that the proposed method is effective for mobile phone clustering from acquired speech recordings.

Speaker role clustering is to obtain the number of different roles and to merge the utterances of the same role into one cluster in an unsupervised way, which is important for rich transcription of multi-speaker spoken documents. This paper presents an approach to role clustering using turn features and maximum distances of inter-clusters. The turn features of each speaker are extracted from audio outputs of speaker diarization, and used as the initial clusters. During clustering iteration, the cluster-pair (e.g. C A and C B ) with the minimum distance is merged and the cluster number is decreased by one if the distance of the N c - 1 clusters (after merging C A and C B ) is bigger than that of the N c clusters (not merging C A and C B ); otherwise, the clustering iteration is finished. Evaluated on four types of multi-speaker spoken documents, the proposed approach outperforms the previous clustering approach and is close to the supervised approach in terms of K scores.

In order to improve movie audio scene (MAS) recognition accuracy, weighted finite-state transducer (WFST) is proposed to recognize MAS in this paper. WFST is introduced firstly, how to construct WFST is introduced secondly, WFST is used to recognize MAS using FBANK, MFCC and PLPCC, separately. The experimental results on twenty MASs using the three features shows that WFST can recognize MAS well, FBANK feature performs better than MFCC and PLPCC, which can reach 79.9%.

In order to extract effective audio feature using autoencoder, different from traditional bottle-neck autoencoder, bottle-body autoencoder is presented in this paper, which is constructed using restricted Boltzmann machine with the same neurons at every layer. Bottle-body feature, which is obtained by using pseudo-inverse method to initialize weights, is applied to audio signal classification. The proposed approach is evaluated on the BBC Sound Effects Library, and shows a 14.90% and 16.20% improvement on classification accuracy than traditional Mel-frequency cepstral coefficient and bottle-neck feature.

Source recording device matching from two speech recordings is a new and important problem of digital media forensics. It aims to answer the question that whether or not two speech recordings are recorded by the same recording device. In this study we propose a source cell phone matching scheme. The Gaussian supervector (GSV) based on Mel-frequency cepstral coefficients (MFCCs) is extracted from the speech recording and is sparse represented with respect to a dictionary learned by K-SVD algorithm. The reduced-dimensional sparse representation coefficient is utilized to characterize the intrinsic fingerprint of the recording device. Then, KISS metric learning based similarity matching is conducted on a pair of fingerprints extracted from the two speech recordings. Evaluation experiments were conducted on a database of speech recordings recorded by 14 cell phones. The experimental results demonstrated the feasibility of the proposed scheme.