Search

Since the proposal of a fast learning algorithm for deep belief networks in 2006, the deep learning techniques have drawn ever-increasing research interests because of their inherent capability of overcoming the drawback of traditional algorithms dependent on hand-designed features. Deep learning approaches have also been found to be suitable for big data analysis with successful applications to computer vision, pattern recognition, speech recognition, natural language processing, and recommendation systems. In this paper, we discuss some widely-used deep learning architectures and their practical applications. An up-to-date overview is provided on four deep learning architectures, namely, autoencoder, convolutional neural network, deep belief network, and restricted Boltzmann machine. Different types of deep neural networks are surveyed and recent progresses are summarized. Applications of deep learning techniques on some selected areas (speech recognition, pattern recognition and computer vision) are highlighted. A list of future research topics are finally given with clear justifications. [ABSTRACT FROM AUTHOR]

Over the last few years, the deep neural network is utilized to solve the collaborative filtering problem, a method of which has achieved immense success on computer vision, speech recognition as well as natural language processing. On one hand, the deep neural network can be used to capture the side information of users and items. On the other hand, it is also capable of modeling interactions between users and items. Most of existing approaches exploit the neural network with solo structure to model user–item interactions such that the learning representation may be insufficient over the extremely sparse rating data. Recently, a large number of neural networks with mixed structures are devised for learning better representations. A carefully designed hybrid network is able to achieve considerable accuracy but only requires a small amount of extra computation. In order to model user–item interactions, we elaborate a hybrid neural network consisting of the global neural network and several local neural blocks. The multi-layer perceptron is adopted to build the global neural network and the residual network is used to form the local neural block which is inserted into two adjacent global layers. The hybrid network is further combined with the generalized matrix factorization to capture both the linear and nonlinear relationships between users and items. It is verified by experimental results on benchmark datasets that our method is superior to certain state-of-the-art approaches in terms of top-n item recommendation. • Devise a hybrid neural network consisting of a global network and local blocks. • Apply the hybrid neural network to model user-item interactions. • Verify the hybrid network superiority in solving CF by comparing with rivals. [ABSTRACT FROM AUTHOR]

Recently, deep learning techniques have been widely applied to fault diagnosis due to their outstanding feature extraction abilities. The success of deep-learning-based fault diagnosis methods is highly dependent on the quantity and quality of the training data. In practical scenarios, it is challenging to obtain sufficient high-quality training data for fault diagnosis tasks due to complex environments, which would affect the effectiveness of the deep learning methods. In this paper, a new fault diagnosis method is proposed for motor bearing fault diagnosis under small samples. The Siamese neural networks (SNNs) are employed to extract the fault features. A multi-stage training strategy is proposed to train the SNNs with the aim to prevent the training stagnation problem and handle the small sample problem. A multi-source feature fusion network is developed to make full use of the multi-source sensor data by fusing the extracted fault features for further fault diagnosis. The proposed method is applied to motor bearing fault diagnosis on two real-world datasets. Experimental results demonstrate the effectiveness and feasibility of the introduced method for motor bearing fault diagnosis under small samples. [ABSTRACT FROM AUTHOR]

In this paper, a hybrid learning approach, which combines the extreme learning machine (ELM) with a new switching delayed PSO (SDPSO) algorithm, is proposed for the problem of the short-term load forecasting (STLF). In particular, the input weights and biases of ELM are optimized by a new developed SDPSO algorithm, where the delayed information of locally best particle and globally best particle are exploited to update the velocity of particle. By testing the proposed SDPSO-ELM in a comprehensive manner on a tanh function, this approach obtain better generalization performance and can also avoid adding unnecessary hidden nodes and overtraining problems. Moreover, it has shown outstanding performance than other state-of-the-art ELMs. Finally, the proposed SDPSO-ELM algorithm is successfully applied to the STLF of power system. Experiment results demonstrate that the proposed learning algorithm can get better forecasting results in comparison with the radial basis function neural network (RBFNN) algorithm. [ABSTRACT FROM AUTHOR]

Facial expression recognition is an important research issue in the pattern recognition field. In this paper, we intend to present a novel framework for facial expression recognition to automatically distinguish the expressions with high accuracy. Especially, a high-dimensional feature composed by the combination of the facial geometric and appearance features is introduced to the facial expression recognition due to its containing the accurate and comprehensive information of emotions. Furthermore, the deep sparse autoencoders (DSAE) are established to recognize the facial expressions with high accuracy by learning robust and discriminative features from the data. The experiment results indicate that the presented framework can achieve a high recognition accuracy of 95.79% on the extended Cohn–Kanade (CK+) database for seven facial expressions, which outperforms the other three state-of-the-art methods by as much as 3.17%, 4.09% and 7.41%, respectively. In particular, the presented approach is also applied to recognize eight facial expressions (including the neutral) and it provides a satisfactory recognition accuracy, which successfully demonstrates the feasibility and effectiveness of the approach in this paper. [ABSTRACT FROM AUTHOR]