Search

Background Classification of the type of calcaneal fracture on CT images is essential in driving treatment. However, human-based classification can be challenging due to anatomical complexities and CT image constraints. The use of computer-aided classification system in standard practice is additionally hindered by the availability of training images. The aims of this study is to 1) propose a deep learning network combined with data augmentation technique to classify calcaneal fractures on CT images into the Sanders system, and 2) assess the efficiency of such approach with differential training methods. Methods In this study, the Principle component analysis (PCA) network was selected for the deep learning neural network architecture for its superior performance. CT calcaneal images were processed through PCA filters, binary hashing, and a block-wise histogram. The Augmentor pipeline including rotation, distortion, and flips was applied to generate artificial calcaneus fractured images. Two types of training approaches and five data sample sizes were investigated to evaluate the performance of the proposed system with and without data augmentation. Results Compared to the original performance, use of augmented images during training improved network performance accuracy by almost twofold in classifying Sanders fracture types for all dataset sizes. A fivefold increase in the number of augmented training images improved network classification accuracy by 35%. The proposed deep CNN model achieved 72% accuracy in classifying CT calcaneal images into the four Sanders categories when trained with sufficient augmented artificial images. Conclusion The proposed deep-learning algorithm coupled with data augmentation provides a feasible and efficient approach to the use of computer-aided system in assisting physicians in evaluating calcaneal fracture types.

Background and objectives The calcaneus is the most fracture-prone tarsal bone and injuries to the surrounding tissue are some of the most difficult to treat. Currently there is a lack of consensus on treatment or interpretation of computed tomography (CT) images for calcaneus fractures. This study proposes a novel computer-assisted method for automated classification and detection of fracture locations in calcaneus CT images using a deep learning algorithm. Methods Two types of Convolutional Neural Network (CNN) architectures with different network depths, a Residual network (ResNet) and a Visual geometry group (VGG), were evaluated and compared for the classification performance of CT scans into fracture and non-fracture categories based on coronal, sagittal, and transverse views. The bone fracture detection algorithm incorporated fracture area matching using the speeded-up robust features (SURF) method, Canny edge detection, and contour tracing. Results Results showed that ResNet was comparable in accuracy (98%) to the VGG network for bone fracture classification but achieved better performance for involving a deeper neural network architecture. ResNet classification results were used as the input for detecting the location and type of bone fracture using SURF algorithm. Conclusions Results from real patient fracture data sets demonstrate the feasibility using deep CNN and SURF for computer-aided classification and detection of the location of calcaneus fractures in CT images.

This paper investigated a feature learning problem, where localized and part-based representations are embedded in dimension-reduction progress. We propose a subspace learning model based on nonnegative matrix factorization (NMF) framework. The proposed NMF model incorporates wisely the spatial location information constraint to the base matrix, the manifold and max-margin penalty to the projective feature matrix for enhancing the capability of exploiting much information of the original data. Experiments on facial expression recognition scenery reveal powerful data representation ability of the proposed NMF method that enables dictionary learning and feature learning jointly.

This paper develops a deep learning model for the beauty product image retrieval problem. The proposed model has two main components- an encoder and a memory. The encoder extracts and aggregates features from a deep convolutional neural network at multiple scales to get feature embeddings. With the use of an attention mechanism and a data augmentation method, it learns to focus on foreground objects and neglect background on images, so can it extract more relevant features. The memory consists of representative states of all database images as its stacks, and it can be updated during training process. Based on the memory, we introduce a distance loss to regularize embedding vectors from the encoder to be more discriminative. Our model is fully end-to-end, requires no manual feature aggregation and post-processing. Experimental results on the Perfect-500K dataset demonstrate the effectiveness of the proposed model with a significant retrieval accuracy.

Automatic sound event recognition (SER) has recently attracted renewed interest. Although practical SER system has many useful applications in everyday life, SER is challenging owing to the variations among sounds and noises in the real-world environment. This paper presents a novel feature extraction and classification method to solve the problem of SER. An audio–visual descriptor, called the auditory-receptive-field binary pattern, is designed based on the spectrogram image feature, the cepstral features, and the human auditory receptive field model. The extracted features are then fed into a classifier to perform event classification. The proposed classifier, called the hierarchical-diving deep belief network, is a deep neural network system that hierarchically learns the discriminative characteristics from physical feature representation to the abstract concept. The performance of our proposed system was verified using several experiments under various conditions. Using the RWCP dataset, the proposed system achieved a recognition rate of 99.27% for real-world sound data in 105 categories. Under noisy conditions, the developed system is very robust, with which it achieved 95.06% recognition rate with 0 dB signal-to-noise ratio. Using the TUT sound event dataset, the proposed system achieves error rates of 0.81 and 0.73 in sound event detection in home and residential area scenes. The experimental results reveal that the proposed system outperformed the other systems in this field.

In this paper, a dimensionality reduction method applied on facial expression recognition is investigated. An unsupervised learning framework, projective complex matrix factorization (proCMF), is introduced to project high-dimensional input facial images into a lower dimension subspace. The proCMF model is related to both the conventional projective nonnegative matrix factorization (proNMF) and the cosine dissimilarity metric in the simple manner by transforming real data into the complex domain. A projective matrix is then found through solving an unconstraint complex optimization problem. The gradient descent method was utilized to optimize a complex cost function. Extensive experiments carried on the extended Cohn-Kanade and the JAFFE databases show that the proposed proCMF model provides even better performance than state-of-the-art methods for facial expression recognition.

Speaker identification is an important topic with relevance to various disciplines. This paper proposes a novel speaker identification system, which consists of two major components—feature extraction and sparse representation classifier (SRC). Although SRC has been utilized for many classification purposes, few studies have provided insight into the link between the commonly used speaker identification feature, i-vector, and SRC. To combine i-vector and SRC sufficiently, we use probabilistic principal component analysis and Bartlett test to extract high-quality i-vector to construct a discriminative dictionary in SRC, supporting effective speaker identification. Besides improving dictionary from the i-vector aspect, we also utilize dictionary learning to further enhance the content of the dictionary. Two learning methods are proposed—robust principal component analysis dictionary and SVD-dictionary. Furthermore, we propose constructing a noise dictionary and combine it with the original dictionary to absorb and suppress noise when implementing the sparse coding. Various coding methods are utilized and analyzed. A comparison to the methods for speaker identification reveals that the proposed method outperforms the baselines and confirms its feasibility.

Deep learning and computer vision that become popular in recent years are advantage techniques in medical diagnosis. A large database of Optical Coherence Tomography (OCT) images can be used to train a deep learning model which can support and suggest effectively illnesses and status of a patient. Therefore, semantic image segmentation is used to detect and categorize anomaly regions in OCT images. However, numerous existing approaches ignored spatial structure as well as contextual information in a given image. To overcome existing problems, this work proposes a novel method which takes advantage of the deep convolutional neural network, attention block, pyramid pooling module and auxiliary connections between layers. Attention block helps to detect the spatial structure of a given image. Beside, pyramid pooling module has a responsibility to identify the shape and margin of the anomaly region. In additional, auxiliary connections support to enrich useful information pass through one layer as well as reduce overfitting problem. Our work produces higher accuracy than state-of-the-art methods with 78.19% comparing to Deeplab_ v3 76.19% and Bisenet 76.85% in term of dice coefficient. Additionally, a number of parameters in our work is smaller than the previous approaches.

This research provides the development of Non-Negative Matrix Factorization method for classifying gene expression data. Furthermore, we also compare the NMF method with Uni-Orthogonal NMF for classifying the data. We use two schemas of datasets which use complete dataset and incomplete dataset. The incomplete dataset contains missing values which used to prove that the methods can handle the missing values problem on the classification problem. Sparse regularization in dimensionality reduction using ridge term (Lz-Norm) is applied in this study to see the effect of sparse regularization in the incomplete dataset. In this paper, we propose an approach to classify the diseases from gene expression data using the combination of matrix factorization methods and support vector machine (SVM). The experimental results show that adding ridge term in Y-orthogonal NMF make accuracy higher in training data with incomplete data. Y-orthogonal NMF using ridge term is the best method for classifying expressed genes with incomplete data than the other compared methods.

In recent years, numerous studies of the use of a Fully Convolutional Network (FCN) for image semantic segmentation have been published. This work introduces an end-to-end Object Bounding Transformed Network (OBTNet) which combines the advantages of the Object Boundary Guided (OBG) and Doman Transform (DT). OBG is an object boundary based approach that increases the integrity of object shape. Based on OBG, we propose an Object Boundary Network (OBN) as the object region and object boundary generator. In addition, our system achieves object region preserving and object boundary preserving by employing DT. The proposed system uses the pretrained multi-scale ResNet101 as the base network and uses multi-scale atrous convolution to preserve the dimensions of the feature map, increasing the accuracy of semantic segmentation. Experiments show that our system yielded a mean IOU of 77.74% and outperformed the baseline model on the VOC2012 test set.

Gaussian process (GP) is a flexible kernel-based learning method which has found widespread applications in signal processing. In this paper, a supervised approach is proposed to handle single-channel speech separation (SCSS) problem. We focus on modeling a nonlinear mapping between mixed and clean speeches based on GP regression, in which reconstructed audio signal is estimated by the predictive mean of GP model. The nonlinear conjugate gradient method was utilized to perform the hyper-parameter optimization. The experiment on a subset of TIMIT speech dataset is carried out to confirm the validity of the proposed approach.

This study proposes a speech enhancement method based on compressive sensing. The main procedures involved in the proposed method are performed in the frequency domain. First, an overcomplete dictionary is constructed from the trained speech frames. The atoms of this redundant dictionary are spectrum vectors that are trained by the K-SVD algorithm to ensure the sparsity of the dictionary. For a noisy speech spectrum, formant detection and a quasi-SNR criterion are first utilized to determine whether a frequency bin in the spectrogram is reliable, and a corresponding mask is designed. The mask-extracted reliable components in a speech spectrum are regarded as partial observations and a measurement matrix is constructed. The problem can therefore be treated as a compressive sensing problem. The K atoms of a K-sparsity speech spectrum are found using an orthogonal matching pursuit algorithm. Because the K atoms form the speech signal subspace, the removal of the noise projected onto these K atoms is achieved by multiplying the noisy spectrum with the optimized gain that corresponds to each selected atom. The proposed method is experimentally compared with the baseline methods and demonstrates its superiority.

This paper presents Program Guardian, which is a speaker recognition-based screening system for smart TV. The system identifies a specific person from his or her voice such that the smart TV can provide suitable programs for that person. This system is based on a robust speaker recognition system that uses robust principal component analysis (RPCA) and a sparse representation classifier (SRC). First, i-vectors that are generated from supervectors of Gaussian mixture models (GMMs) are used to generate the basic atoms of an over-complete dictionary. The i-vectors are then transformed using RPCA. The SRC is produced from transformed i-vector-based RPCA vectors. Finally, the sparse representation classifier corresponding to the target speaker with the least reconstruction error is constructed. NIST speaker recognition evaluation data base is used in our experiment. The results show that the proposed speaker recognition system is feasible and offers advantages over accuracy.

This paper proposes a speaker recognition system using acoustic features that are based on spectral-temporal receptive fields (STRFs). The STRF is derived from physiological models of the mammalian auditory system in the spectral-temporal domain. With the STRF, a signal is expressed by rate (in Hz) and scale (in cycles/octaves). The rate and scale are used to specify the temporal response and spectral response, respectively. This paper uses the proposed STRF based feature to perform speaker recognition. First, the energy of each scale is calculated using the STRF representation. A logarithmic operation is then applied to the scale energies. Finally, a discrete cosine transform is utilized to the generation of the proposed STRF feature. This paper also presents a feature set that combines the proposed STRF feature with conventional Mel frequency cepstral coefficients (MFCCs). The support vector machines (SVMs) are adopted to be the speaker classifiers. To evaluate the performance of the proposed speaker recognition system, experiments on 36-speaker recognition were conducted. Comparing with the MFCC baseline, the proposed feature set increases the speaker recognition rates by 3.85 % and 18.49 % on clean and noisy speeches, respectively. The experiments results demonstrate the effectiveness of adopting STRF based feature in speaker recognition.

In this work, we address human action recognition problem under viewpoint variation. The proposed model is formulated by wisely combining convolution neural network (CNN) model with principle component analysis (PCA). In this context, we pass real depth videos through a CNN model in a frame-wise manner. The view invariant features are extracted by employing convolution layers as mid-outputs and considered as 3D nonnegative tensors. The PCA algorithm is separately imposed on view-invariant high-level space of image and video groups to seek both local and holistic hidden dynamics information. To deal with noisy data and temporal misalignment, we utilize the Fourier temporal pyramid to encode temporal and obtain the final descriptors. Our proposed framework supplies a robust discriminative representation with low dimension and computational requirements. We evaluate the proposed method on two standard multiview depth video datasets. The experimental results show that our method has superior performance compared to other approaches.

Techniques for analyzing complex-valued data are required in numerous fields, such as signal processing. This work develops a novel complex-valued latent variable model, named locality-preserving discriminative complex-valued Gaussian process latent variable model (LPD-CGPLVM), for discovering a compressed complex-valued representation of data. The developed LPD-CGPLVM operates on the complex-valued domain. Additionally, we attempt to preserve both global and local data structures while promoting discrimination. A new objective function that imposes a locality-preserving and a discriminative term for complex-valued data is presented. Complex-valued gradient descent is then utilized to obtain a complex-valued representation of high-dimensional data and the hyperparameters in the LPD-CGPLVM. The proposed method was evaluated using two pattern recognition applications - face recognition with occlusion and music emotion recognition. The experimental results thus obtained demonstrated the superior accuracy of the proposed method, especially for situations with only a small number of training data.

Modeling musical timbre is critical for various music information retrieval (MIR) tasks. This work addresses the task of classifying playing techniques, which involves extremely subtle variations of timbre among different categories. A deep collaborative learning framework is proposed to represent a music with greater discriminative power than previously achieved. Firstly, a novel variational autoencoder (VAE) is developed to eliminate the variation of acoustic features within a class. Secondly, a Gaussian process classifier is jointly learned to distinguish the variations of timbres between classes, which increases the discriminative power of the learned representations. We derive a new lower bound that guides a VAE-based representation. Experiments were conducted on a database of seven classes of guitar playing techniques. The experimental results demonstrated that the proposed method outperforms baselines in terms of the Fl-score and accuracy.

Learning a low-dimensional image representation yields effective and efficient face recognition. The use of such a representation helps to weaken the curse of dimensionality. However, the traditional facial representation method is not robust against partial occlusions or variations of expression. To solve this problem, this paper proposes a more reliable, complex-valued representation of facial image. The robust representation is based on the proposed locality-preserving complex-valued Gaussian process latent variable model (LP-CGPLVM). In the LP-CGPLVM, the Euler formula is utilized to transform original facial images into the complex domain. A proper complex GP is employed to model the mapping between the complex-valued high-dimensional data and the corresponding low-dimensional representation. Moreover, the locality-preserving constraint is taken into consideration to preserve the neighborhood data structure. The experimental results indicate that our proposed method is robust against partial occlusions and various facial expressions.

Matrix factorization (MF) and its extensions have been intensively studied in computer vision and machine learning. In this paper, unsupervised and supervised learning methods based on MF technique on complex domain are introduced. Projective complex matrix factorization (PCMF) and discriminant projective complex matrix factorization (DPCMF) present two frameworks of projecting complex data to a lower dimension space. The optimization problems are formulated as the minimization of the real-valued functions of complex variables. Motivated by independence among extracted features, Fisher linear discriminant is used as hard constraint on supervised model. Experimental results on facial expression recognition (FER) show improved classification performance in comparison to real-valued features of both unsupervised and supervised NMFs.

Audio scenes are often composed of a variety of sound events from different sources. Their content exhibits wide variations in both frequency and time domain. Convolutional neural networks (CNNs) provide an effective way to extract spatial information of multidimensional data such as image, audio, and video; they have the ability to learn hierarchical representation from time-frequency features of audio signals. In this paper, we develop a convolutional neural network and employ a multi-scale multi-feature extraction methods for acoustic scene classification. We conduct experiments on the TUT Acoustic Scenes 2016 dataset. Experimental results show that the use of multi-scale multi-feature extraction methods improves significantly the performance of the system. Our proposed approach obtains a high accuracy of 85.9% that outperforms the baseline approach by a large margin of 8.7%.

A new NMF model, spatial constrained graph sparse nonnegative matrix factorization (SGSNMF) is adopted for facial expression recognition. In this model, the extracted features preserve the topological structure of the original images and achieve sparseness from L 2 constraint on coefficient matrix, meanwhile the base satisfy pixel dispersion penalty. The proposed method takes advantage of the project gradient decent and is based on the alternating nonnegative least square framework. Experiments on two facial expression recognition scenarios that involve a whole face and an occluded face reveal that the proposed algorithm outperforms the prevalent NMF methods.

Vehicle classification is one of the major part for the smart road management system and traffic management system. The use of appropriate algorithms has a significant impact in the process of classification. In this paper, we propose a deep neural network, named center strengthened convolutional neural network (CS- CNN), for handling central part image feature enhancement with non-fixed size input. The main hallmark of this proposed architecture is center enhancement that extract additional feature from central of image by ROI pooling. Another, our CS-CNN, based on VGG network architecture, joint with ROI pooling layer to get elaborate feature maps. Our proposed method will be compared with other typical deep learning architecture like VGG-s and VGG-Verydeep-16. In the experiments, we show the outstanding performance which getting more than 97% accuracy on vehicle classification with only few training data from Caltech256 datasets.

This paper develops an approach to speech-based emotion verification based on emotion variance modeling and discriminant scale-frequency maps. The proposed system consists of two parts---feature extraction and emotion verification. In the first part, for each sound frame, important atoms from the Gabor dictionary are selected by using the matching pursuit algorithm. The scale, frequency, and magnitude of the atoms are extracted to construct a nonuniform scale-frequency map, which supports auditory discriminability by the analysis of critical bands. Next, sparse representation is used to transform scale-frequency maps into sparse coefficients to enhance the robustness against emotion variance and achieve error-tolerance improvement. In the second part, emotion verification, two scores are calculated. A novel sparse representation verification approach based on Gaussian-modeled residual errors is proposed to generate the first score from the sparse coefficients. Such a classifier can minimize emotion variance and improve recognition accuracy. The second score is calculated by using the emotional agreement index (EAI) from the same coefficients. These two scores are combined to obtain the final detection result. Experiments on an emotional database of spoken speech were conducted and indicate that the proposed approach can achieve an average equal error rate (EER) of as low as 6.61%. A comparison among different approaches reveals that the proposed method is superior to the others and confirms its feasibility.

This paper proposes a robust environmental sound recognition system using a fast noise suppression approach for home automation applications. The system comprises a fast subspace-based noise suppression module and a sound classification module. For the noise suppression module, we propose a noise suppression method that applies fast subspace approximations in the wavelet domain. We show that this method offers a lower computational cost than conventional methods. In the sound classification module, we use a feature extraction method that is also based on the wavelet subspace, derived from seventeen critical bands in a signal's wavelet packet transform. Furthermore, we create a multiclass support vector machine by employing probability product kernels. The experimental results for ten classes of various environmental sounds show that the proposed system offers robust performance in environmental sound recognition tasks.

This work presents an access control system, which is a speaker identification system based on whispered speech. Speaker identification is a main function of an access control system. Hence, a novel speaker identification system using instantaneous frequencies is proposed. The input speech signals pass through both signal independent and signal dependent filters firstly. Then, we derive the signal’s instantaneous frequencies by applying the Hilbert transform. The analyzed instantaneous frequencies are proceeded to be modeled as probability density models. We use these probability density models as the feature in the proposed speaker identification system. In this work, we compare the use of parametric and nonparametric probability density estimation for instantaneous frequency modeling. Furthermore, we propose an approximated probability product kernel support vector machine (APPKSVM). In the APPKSVM, Riemann sum is applied in approximating the probability product kernel. The whisper sounds from the CHAIN speech corpus were used in the experiments. Results of the experiments show the superiority of the proposed speaker identification system.

This brief presents the chip implementation of a support vector machine (SVM)-based speaker verification system. The proposed chip comprises a speaker feature extraction (SFE) module, an SVM module, and a decision module. The SFE module performs autocorrelation analysis, linear predictive coefficient (LPC) extraction, and LPC-to-cepstrum conversion. The SVM module includes a Gaussian kernel unit and a scaling unit. The purpose of the Gaussian kernel unit is first to evaluate the kernel value of a test vector and a support vector. Four Gaussian kernel processing elements (GK-PEs) are designed to process four support vectors simultaneously. Each GK-PE is designed in the pipeline fashion and is capable of performing 2-norm and exponential operations. An enhanced CORDIC architecture is proposed to calculate the exponential value. As well as the Gaussian kernel unit, a scaling unit is also developed for use in the SVM module. The scaling unit is used to perform scaling multiplications and the remaining operations of SVM decision value evaluation. Finally, the decision module accumulates the frame scores that are generated by all of the test frames, and then compare it with a threshold to see if the test utterance is spoken by the claimed speaker. This designed chip is characterized by its high speed and its ability to handle a large number of support vectors in the SVM. The prototype chip is a semicustom chip that is fabricated using Taiwan Semiconductor Manufacturing Company 0.90-nm CMOS technology on a die with a size of $\sim 7.9\times 7.9$ mm $^{2}$ .

This study proposes a novel multi-label music emotion recognition (MER) system. An emotion cannot be defined clearly in the real world because the classes of emotions are usually considered overlapping. Accordingly, this study proposes an MER system that is based on hierarchical Dirichlet process mixture model (HPDMM), whose components can be shared between models of each emotion. Moreover, the HDPMM is improved by adding a discriminant factor to the proposed system based on the concept of linear discriminant analysis. The proposed system represents an emotion using weighting coefficients that are related to a global set of components. Moreover, three methods are proposed to compute the weighting coefficients of testing data, and the weighting coefficients are used to determine whether or not the testing data contain certain emotional content. In the tasks of music emotion annotation and retrieval, experimental results show that the proposed MER system outperforms state-of-the-art systems in terms of F- score and mean average precision.

In this work, we propose an Asymmetric Kernel Convolutional Neural Network (AKCNN) for Acoustic Scenes Classification (ASC). Its kernel shape is not the traditional square but asymmetric in width and height. It also uses Weight Normalization (WN) to accelerate the training process because it can early converge the training loss and accuracy. The best of all, WN can help increase the accuracy of ASC. TUT Acoustic Scenes 2016 Dataset [1] is used for evaluation. The result shows that AKCNN achieves accuracy 86.7%. If we rank the score in DCASE2016 ASC Challenge, it shows that the system would have a higher score than the 5th place.

This work develops a topic model-based hierarchical representation for identifying the latent characteristics behind the frame-level musical features. Frame-level features and music clips are regarded as acoustic words and acoustic documents, respectively. A Gaussian hierarchical latent Dirichlet allocation (G-hLDA) is proposed to find the latent topics behind the acoustic document. The G-hLDA directly handles the continuous features instead of transforming them into discrete words, reducing information loss from discretization-based vector quantization. Specially, each latent topic that is identified by G-hLDA is represented as a node in the infinitely deep, infinitely branching tree. For a music clip, the number of its acoustic words at each node is computed to form the hierarchical representation. The proposed representation hierarchically captures not only the shared components but also the unique components among music clips, resulting in improved performance. The experimental results on the guitar playing technique database demonstrate that the proposed method outperforms baselines.

This work develops a system for recognizing common hand gestures. The main idea that underlies the developed system is the incorporation of Bhattacharyya divergence into Bayesian sensing hidden Markov models (BS-HMM). The system consists of two stages. First, a sequence of depth images is captured by Microsoft Kinect. The hand region is identified from the depth images by tracking the position of the hand using information about the skeleton, yielding the segmented depth images. A histogram of the oriented normal 4D (HON4D) and a histogram of oriented gradient (HOG) are then extracted from the segmented depth images to represent the motion patterns. Second, all training feature vectors are transformed by combining every k consecutive feature vectors into a sequence of distributions. The proposed Bhattacharyya divergence based BS-HMM (BDBS-HMM) is trained using the sequence of distributions. The proposed system is compared to the standard HMM and the BS-HMM using MSRGesture3D database and our database. Experimental results indicated that the proposed method outperforms the baseline methods.

This paper proposes a novel deep convolutional neural network (CNN), called sparse coding convolutional neural network (SC-CNN), to address the problem of sound event recognition and retrieval task. Unlike the general framework of a CNN, in which feature learning process is performed hierarchically, the proposed framework models the whole memorizing procedures in the human brain, including encoding, storage, and recollection. Sound data from the RWCP sound scene dataset with added noise from NOISEX-92 noise dataset are used to compare the performance of the proposed system with the state-of-the-art baselines. The experimental results indicated that the proposed SC-CNN outperformed the state-of-the-art systems in sound event recognition and retrieval. In the sound event recognition task, the proposed system achieved an accuracy of 94.6%, 100% and 100% under 0db, 10db and clean noise conditions, respectively. In the retrieval task, the proposed system improves the mAP rate of the general CNN by approximately 6%.

Convolutional neural network (CNN) is more and more important in pattern recognition. In this work, we adopt label relations and long short-term memory (LSTM) to develop an accurate CNN-based scene classification algorithm. Traditional scene classification algorithms assume that labels are mutually exclusive. However, this is not reasonable when an image has a variety of objects and hence has multiple labels. In this work, we apply two label relations, which are exclusive and hierarchy relations, to improve the accuracy of multiple-label scene classification. For example, it is impossible that an image has both the labels of “factory” and “garden”. If the label “factory” is assigned to an image, the probability that it has the label of “garden” should be lowered. We also use image captioning to construct a scene classification model and propose an LSTM based method to further explore label relations and obtain more accurate results for scenic image labeling.

Deep learning is a popular method for monaural source separation, and especially for extracting a singing voice from a single-channel song. However, deep learning-based source separation ignores the geometrical structure of the input data. This work develops a novel approach to source separation that is based on non-negative matrix factorization (NMF) and deep recurrent neural networks (DRNN) with a locality-preserving constraint. First, NMF was used to learn patterns from training data. The learned patterns are linearly combined with the output of DRNN. Second, a locality-preserving constraint is developed to exploit the inner-structure of the input data in the DRNN learning process. Experimental results obtained using the MIR-1K dataset reveal that the proposed algorithm outperforms the baselines.

This work presents a novel approach for audio event recognition. The approach develops a weighted kernel fisher sparse analysis method based on multiple maps. The proposed method consists of maps extraction and kernel weighted Fisher sparse analysis. Two maps are firstly extracted from each audio file, i.e. scale-frequency map and damping-frequency map. The scale and frequency of the Gabor atoms are extracted to construct a scale-frequency map. On the other hand, the damping-frequency map is generated according to the frequency and damping factor of damped atoms. Gabor atoms can be utilized to model human auditory perception, and the damped atoms can be used to model commonly observed damped oscillations in natural signals. This work fuses the advantages of these two dictionaries to improve the performance of the system. During the recognition stage, this work constructs a kernel sparse representation-based classifier via the proposed kernel weighted Fisher sparse analysis to enhance separability. The proposed kernel weighted Fisher sparse analysis combines sparse representation with heteroscedastic kernel weighted discriminant analysis (HKWDA), which is useful for providing a discriminative recognition of audio events because a weighted pairwise Chernoff criterion is utilized in the kernel space. Experiments on a 20-class audio event database indicate that the proposed approach can achieve an accuracy rate of 82.70%. Also, integrating the scale-frequency map with MFCCs increases the accuracy rate to 87.70%.

Deep neural network (DNN) have become a popular means of separating a target source from a mixed signal. Most of DNN-based methods modify only the magnitude spectrum of the mixture. The phase spectrum is left unchanged, which is inherent in the short-time Fourier transform (STFT) coefficients of the input signal. However, recent studies have revealed that incorporating phase information can improve the quality of separated sources. To estimate simultaneously the magnitude and the phase of STFT coefficients, this work paper developed a fully complex-valued deep neural network (FCDNN) that learns the nonlinear mapping from complex-valued STFT coefficients of a mixture to sources. In addition, to reinforce the sparsity of the estimated spectra, a sparse penalty term is incorporated into the objective function of the FCDNN. Finally, the proposed method is applied to singing source separation. Experimental results indicate that the proposed method outperforms the state-of-the-art DNN-based methods.

In this paper, we discuss the multipitch streaming (MPS) problem for a multi-source audio signal having interweaving pitch contours. We propose two approaches to tackle this challenge, one relates to a feature extracted from the energy levels distributed in multi-channel recordings for better characterization of the source, and the other uses particle swarm optimization (PSO) to enlarge the search space and alleviate the initialization problem in constrained clustering of the features representing different sources. Experiments on music and speech samples having highly interweaving pitch contours are presented to assess its effectiveness.

This work presents a novel feature extraction approach called nonuniform scale-frequency map for environmental sound classification in home automation. For each audio frame, important atoms from the Gabor dictionary are selected by using the Matching Pursuit algorithm. After the system disregards phase and position information, the scale and frequency of the atoms are extracted to construct a scale-frequency map. Principal Component Analysis (PCA) and Linear Discriminate Analysis (LDA) are then applied to the scale-frequency map, subsequently generating the proposed feature. During the classification phase, a segment-level multiclass Support Vector Machine (SVM) is operated. Experiments on a 17-class sound database indicate that the proposed approach can achieve an accuracy rate of 86.21%. Furthermore, a comparison reveals that the proposed approach is superior to the other time-frequency methods.

This research proposes a novel Bayesian sparse representation (BSR) method along with extracting facial parameters of SIFT to create sparse dictionaries, which are invariant to rotation, scale, and shift. By using K-means and information theory, a new dictionary called extended dictionary is developed. Compared with conventional orthogonal matching pursuit (OMP) algorithm, the proposed system that utilized Bayesian method to model the optimization problem of sparse representation can reduce the uncertainty of observed signals and expand the modeling ability of dictionaries by using variance. The experimental results show that the proposed extended dictionary can enhance the sparsity. Furthermore, it can improve accuracy rates of face identification and residues of reconstruction.

Given the increasing attention paid to speech emotion classification in recent years, this work presents a novel speech emotion classification approach based on the multiple kernel Gaussian process. Two major aspects of a classification problem that play an important role in classification accuracy are addressed, i.e. feature extraction and classification. Prosodic features and other features widely used in sound effect classification are selected. A semi-nonnegative matrix factorization algorithm is then applied to the proposed features in order to obtain more information about the features. Following feature extraction, a multiple kernel Gaussian process (GP) is used for classification, in which two similarity notions from our data in the learning algorithm are presented by combining the linear kernel and radial basis function (RBF) kernel. According to our results, the proposed speech emotion classification apporach achieve an accuracy of 77.74%. Moreover, comparing different apporaches reveals that the proposed system performs best than other apporaches.

Nowadays, sound event detection (SED) is a popular study in machine listening area. Detecting overlapping sound events, in which many sound events occur simultaneously, is challenging and interesting topic. Besides, non-negative matrix factorization (NMF) and its derived methods are suitably used to perform SED. This paper presents a survey of recent approaches in SED based on NMF methods that provides the understanding and strategies for detecting polyphonic sound events.

Single-channel source separation is an approach to decomposing a single-channel recording into its sources without understanding how the sources are mixed. This work develops a sparse regularized nonnegative matrix factorization scheme with spatial dispersion penalty (SpaSNMF). To preserve spatial locality structured information on the basis for sound source separation, intra-sample structure constraints that are learnt from the input data are utilized. Based on the hypothesis that adjacent spectrogram points should not be dispersed in basis spectra, this framework is provided for supervised source separation. To improve the separation performance, group sparse penalties are simultaneously constructed. A multiple-update-rule optimization scheme was used to solve the objective function of the proposed SpaSNMF. Experiments on single-channel source separation reveal that the proposed method provides more robust basis factors and achieves better results than standard NMF and its extensions.

This paper concerns the development of locality-preserving methods for object recognition. The major purpose is consideration of both descriptor-level locality and image-level locality throughout the recognition process. Two dual-layer locality-preserving methods are developed, in which locality-constrained linear coding (LLC) is used to represent an image. In the learning phase, the discriminative locality-preserving K-SVD (DLP-KSVD) in which the label information is incorporated into the locality-preserving term is proposed. In addition to using class labels to learn a linear classifier, the label-consistent LP-KSVD (LCLP-KSVD) is proposed to enhance the discriminability of the learned dictionary. In LCLP-KSVD, the objective function includes a label-consistent term that penalizes sparse codes from different classes. For testing, additional information about the locality of query samples is obtained by treating the locality-preserving matrix as a feature. The recognition results that were obtained in experiments with the Caltech101 database indicate that the proposed method outperforms existing sparse coding based approaches.

In this paper, we introduce a new color image segmentation by using superpixels as feature representation and Manhattan Nonnegative Matrix Factorization (MahNMF) for accurate segmentation. Firstly, the image pixels are grouped into superpixels and considered as the coarse features. The next step is then conducted by factorizing the matrix feature into two nonnegative matrices, which respectively imply representative features and their combination coefficients per superpixel. Exploiting superpixels as features can avoid using too much global information to obtain an advance in time complexity, and using MahNMF can analyze these features for getting segmented image. The experiments show the promise of this new approach.

Nowadays the segmentation task becomes an important pre-processing stage for the iris classification system. The earlier works in the iris classification field demonstrate a promising result when the classification is performed under an ideal environment. However, the reduction of accuracy is observed when the iris images are captured in non-ideal circumstances. This work is based on the previous work that propose iris segmentation system with ί-Means clustering algorithm. In this work, we evaluate the performance of NMF-based clustering approach to replace the ί-Means algorithm. The iris images from UBIRIS dataset are used to verify the reliability of our work to perform iris region extraction in the unconstrained environments.

The purpose of single source separation is to recover a particular signal from a mixed signal. This work develops a novel source separation method for use with an automatic speech recognition (ASR) system. The proposed method is based on non-negative matrix factorization (NMF), which is extensively used in single channel source separation. In the cost function, a flexible distance, αβ-divergence, is used. Additionally, a mixture signal in high-dimensional space contains a low-dimensional manifold. To preserve this embedded structure, a graph regularization constraint is added to the objective function for optimization. The experimental results thus obtained reveal that the proposed method outperforms baseline methods.

This paper presents an ensemble-based speaker recognition using unsupervised data selection. Ensemble learning is a type of machine learning that applies a combination of several weak learners to achieve an improved performance than a single learner. A speech utterance is divided into several subsets based on its acoustic characteristics using unsupervised data selection methods. The ensemble classifiers are then trained with these non-overlapping subsets of speech data to improve the recognition accuracy. This new approach has two advantages. First, without any auxiliary information, we use ensemble classifiers based on unsupervised data selection to make use of different acoustic characteristics of speech data. Second, in ensemble classifiers, we apply the divide-and-conquer strategy to avoid a local optimization in the training of a single classifier. Our experiments on the 2010 and 2008 NIST Speaker Recognition Evaluation datasets show that using ensemble classifiers yields a significant performance gain.

Music is a powerful force that evokes human emotions. Several investigations of music emotion recognition (MER) have been conducted in recent years. This paper proposes a system for detecting emotion in music that is based on a deep Gaussian process (GP). The system consists of two parts-feature extraction and classification. In the feature extraction part, five types of features that are associated with emotions are selected for representing the music signal; these are rhythm, dynamics, timbre, pitch and tonality. A music clip is decomposed into frames and these features are extracted from each frame. Next, statistical values, such as mean and standard deviation, of frame-based features are calculated to generate a 38-dimensional feature vector. In the classification part, a deep GP is utilized for emotion recognition. We treat classification problem from the perspective of regression. Finally, 9 classes of emotion are categorized by 9 one-versus-all classifiers. The experimental results demonstrate that the proposed system performs well in emotion recognition.

Speech separation aims to estimate the target signals produced by individual speech sources from a mixture signal. In this paper, we especially review on data-driven separation methods, where algorithms will be enhanced to produce better dictionary learning which considers the geometric of input data and efficiently performs separation mixture. We review the existing algorithms using non-negative matrix factorization, sparse coding, mixture local dictionary, group lasso, and graph regularization to produce knowledge bases. We also review the extension of NMF by incorporating two state-of-art techniques i.e. bilevel optimization and deep neural network.