Search

Deep learning is a popular topic among machine learning researchers nowadays, with great strides being made in recent years to develop robust artificial neural networks for faster convergence to a reasonable accuracy. Network architecture and hyperparameters of the model are fundamental aspects of model convergence. One such important parameter is the initial values of weights, also known as weight initialization. In this paper, we perform two research tasks concerned with the weights of neural networks. First, we develop three novel weight initialization algorithms inspired by the neuroscientific construction of the mammalian brains and then test them on benchmark datasets against other algorithms to compare and assess their performance. We call these algorithms the lognormal weight initialization, modified lognormal weight initialization, and skewed weight initialization. We observe from our results that these initialization algorithms provide state-of-the-art results on all of the benchmark datasets. Second, we analyze the influence of training an artificial neural network on its weight distribution by measuring the correlation between the quantitative metrics of skewness and kurtosis against the model accuracy using linear regression for different weight initializations. Results indicate a positive correlation between network accuracy and skewness of the weight distribution but no affirmative relation between accuracy and kurtosis. This analysis provides further insight into understanding the inner mechanism of neural network training using the shape of weight distribution. Overall, the works in this paper are the first of their kind in incorporating neuroscientific knowledge into the domain of artificial neural network weights. [ABSTRACT FROM AUTHOR]

EEG has been proved to be an effective tool for researchers' cognition and mental workload by detecting the changes of brain activity potential. The mental workload of pilots in aviation flight is closely related to the characteristics of flight tasks. The previous methods have problems such as lack of objectivity, low EEG analysis ability and lack of real-time analysis ability. In order to solve these problems, this paper proposes a multi-dimensional data fusion brain workload calculation method based on flight effect evaluation, which integrates vision, operational behavior and visual gaze, and classifies and analyzes them in combination with EEG data. This method evaluates the mental workload of pilots from three aspects: visual gaze behavior, control behavior and flight effect in the simulated flight experimental environment, and realizes a more objective mental workload analysis. Then, the synchronously collected EEG data are segmented and sampled to form a dataset, and an LSTM neural network model integrating attention mechanism is established, in which the attention mechanism is used to improve the feature processing ability of the network model for the classification of complex EEG data. After machine learning training, the final model can achieve 94% detection accuracy for 2-s EEG data, and has the ability of real-time analysis in the application environment. Compared with the previous similar LSTM model, the accuracy is improved by 6%, which also shows the effectiveness of the model. [ABSTRACT FROM AUTHOR]

With the limitation of air traffic and the rapid increase in the number of flights, flight delay is becoming more frequent. Flight delay leads to financial and time losses for passengers and increases operating costs for airlines. Therefore, the establishment of an accurate prediction model for flight delay becomes vital to build an efficient airline transportation system. The air transportation system has a huge amount of data and complex operation modes, which is suitable for analysis by using machine learning methods. This paper discusses the factors that may affect the flight delay, and presents a new flight delay prediction model. The five warning levels are defined based on flight delay database by using K-means clustering algorithm. After extracting the key factors related to flight operation by the grey relational analysis (GRA) algorithm, an improved machine learning algorithm called GRA — Genetic algorithm (GA) — back propagation neural network, GRA-GA-BP, is introduced, which is optimized by GA. The calculation results show that, compared with models before optimization and other two algorithms in previous papers, the proposed prediction model based on GRA-GA-BP algorithm shows a higher prediction accuracy and more stability. In terms of operation efficiency and memory consumption, it also has good performance. The analysis presented in this paper indicates that this model can provide effective early warnings for flight delay, and can help airlines to intervene in flights with abnormal trend in advance. [ABSTRACT FROM AUTHOR]

Multi-Instance Multi-Label learning (MIML) is a popular framework for supervised classification where an example is described by multiple instances and associated with multiple labels. Previous MIML approaches have focused on predicting labels for instances. The idea of tackling the problem is to identify its equivalence in the traditional supervised learning framework. Motivated by the recent advancement in deep learning, in this paper, we still consider the problem of predicting labels and attempt to model deep learning in MIML learning framework. The proposed approach enables us to train deep convolutional neural network with images from social networks where images are well labeled, even labeled with several labels or uncorrelated labels. Experiments on real-world datasets demonstrate the effectiveness of our proposed approach. [ABSTRACT FROM AUTHOR]

Aiming to solve the problem of tracking drift during movement, which was caused by the lack of discriminability of the feature information and the failure of a fixed template to adapt to the change of object appearance, the paper proposes an object tracking algorithm combining attention mechanism and correlation filter theory based on the framework of full convolutional Siamese neural networks. Firstly, the apparent information is processed by using the attention mechanism thought, where the object and search area features are optimized according to the spatial attention and channel attention module. At the same time, the cross-attention module is introduced to process the template branch and search area branch, respectively, which makes full use of the diversified context information of the search area. Then, the background perception correlation filter model with scale adaptation and learning rate adjustment is adopted into the model construction, using as a layer in the network model to realize the object template update. Finally, the optimal object location is determined according to the confidence map with similarity calculation. Experimental results show that the designed method in the paper can promote the object tracking performance under various challenging environments effectively; the success rate increases by 16.2%, and the accuracy rate increases by 16%. [ABSTRACT FROM AUTHOR]

The lack of research on the metering characteristics of electricity power meters under complex conditions is a major obstacle to the on-site verification of electrical energy metering equipment. Establishing a predictive model for electricity power meter errors offers an effective way of dealing with this issue. Deep learning has been proven to have the capacity to reduce end-to-end dimensionality and improve recognition. Through the analysis of the back propagation process in residual networks, an improved residual network is set out in this paper. While preserving the advantages of residual network gradient propagation, it adds an adjustable shortcut and designs a convex k -parameter strategy that can be improved according to different processing objects. Experimental results show that the predicted errors produced by the proposed technique are significantly lower than in a comparable model. At the same time, the improved residual network does not increase the network's complexity. [ABSTRACT FROM AUTHOR]

Deep learning algorithms have shown superior performance than traditional algorithms when dealing with computationally intensive tasks in many fields. The algorithm model based on deep learning has good performance and can improve the recognition accuracy in relevant applications in the field of computer vision. TensorFlow is a flexible opensource machine learning platform proposed by Google, which can run on a variety of platforms, such as CPU, GPU, and mobile devices. TensorFlow platform can also support current popular deep learning models. In this paper, an image recognition toolkit based on TensorFlow is designed and developed to simplify the development process of more and more image recognition applications. The toolkit uses convolutional neural networks to build a training model, which consists of two convolutional layers: one batch normalization layer before each convolutional layer, and the other pooling layer after each convolutional layer. The last two layers of the model use the full connection layer to output recognition results. Batch gradient descent algorithm is adopted in the optimization algorithm, and it integrates the advantages of both the gradient descent algorithm and the stochastic gradient descent algorithm, which greatly reduces the number of convergence iterations and has little influence on the convergence effect. The total training parameters of the toolkit model reach 1.7 million. In order to prevent overfitting problems, the dropout layer before each full connection layer is added and the threshold of 0.5 is set in the design. The convolution neural network model is trained and tested by the MNIST set on TensorFlow. The experimental result shows that the toolkit achieves the recognition accuracy of 99% on the MNIST test set. The development of the toolkit provides powerful technical support for the development of various image recognition applications, reduces its difficulty, and improves the efficiency of resource utilization. [ABSTRACT FROM AUTHOR]

With the outstanding performance in 2014 at the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14), an effective convolutional neural network (CNN) model named GoogLeNet has drawn the attention of the mainstream machine learning field. In this paper we plan to take an insight into the application of the GoogLeNet in the Handwritten Chinese Character Recognition (HCCR) on the database HCL2000 and CASIA-HWDB with several necessary adjustments and also state-of-the-art improvement methods for this end-to-end approach. Through the experiments we have found that the application of the GoogLeNet for the Handwritten Chinese Character Recognition (HCCR) results into significant high accuracy, to be specific more than 99% for the final version, which is encouraging for us to further research. [ABSTRACT FROM AUTHOR]

At present, the application of deep learning algorithms in two-dimensional color image detection is being continuously innovated and broken. With the popularity of depth cameras, color image detection methods with depth information need to be upgraded. To solve this problem, a multi-target detection algorithm based on 3D DSF R-CNN (Double Stream Faster R-CNN, Convolution Neural Network based on Candidate Region) is proposed in this paper. The RGB information and the depth information of the image are given to two input elements of the convolution network with the same structure and weight sharing, and an optimal fusion weight algorithm is used to determine the weight of the fusion target in accordance with the recognition accuracy of the recognition targets under the single modal information, so as to ensure the most efficient fusion result. After several convolution operations, the independent features are extracted and the two networks are fused according to the optimal weights in the convolution layer. With the conducting of convolution and extract the fused features, and finally get the output through the full link layer. Compared with the previous two-dimensional convolution network algorithm, this algorithm improves the detection rate and success rate while ensuring the detection time. The experimental result shows that this method has strong robustness for complex illumination and partial occlusion, and has excellent detection results under non-restrictive conditions. [ABSTRACT FROM AUTHOR]

The advancement of healthcare technology is impossible without machine learning (ML). There have been numerous advances in ML to analyze, predict, and diagnose medical data. Integrating a centralized scheme and therapy for classifying and diagnosing illnesses and disorders is a major obstacle in modern healthcare. To standardize all medical data into a single repository, researchers have proposed using ML using the centralized artificial neural network model (ML-CANNM). Random tree, support vector machine, and gradient booster are just a few proposed ML classifiers. Artificial neural networks (ANNs) have been trained using a variety of medical datasets to predict and analyze outcomes. ML-CANNM collects patient data from various studies and uses ML and ANNs to determine the results. Three layers make up an ANN. ML is used to classify the given patients' data in the input layer. In the hidden layer, classification data are compared to a training dataset. The output layer's job is to identify, classify, and diagnose diseases. As a result, disease diagnosis and detection are integrated into a single healthcare database. The proposed framework has proven that ML-CANNM works with more accuracy and lesser execution time. Thus, the numerical outcome suggested ML-CANNM increased accuracy ratio of 99.2% and a prediction ratio of 97.5%. The findings further show that the execution time is enhanced by less than 2 h, decision table using ML and results in an efficiency ratio of 97.5%. [ABSTRACT FROM AUTHOR]

This paper presents a methodology for integrating connectionist and symbolic approaches to 2D image recognition. The proposed integration paradigm exploits the synergy of the two approaches for both the training and the recognition phases of an image recognition system. In the training phase, a symbolic module provides an approximate solution to a given image-recognition problem in terms of symbolic models. Such models are hierarchically organized into different abstraction levels, and include contextual descriptions. After mapping such models into a complex neural architecture, a neural training process is carried out to optimize the solution of the recognition problem. The so-obtained neural networks are used during the recognition phase for pattern classification. In this phase, the role of symbolic modules consists of managing complex aspects of information processing: abstraction levels, contextual information, and global recognition hypotheses. A hybrid system implementing the proposed integration paradigm is presented, and its advantages over single approaches are assessed. Results on Magnetic Resonance image recognition are reported, and comparisons with some well-known classifiers are made. [ABSTRACT FROM AUTHOR]

Support vector machines (SVMs) are a relatively recent machine learning technique. One of the SVM problems is that SVM is considerably slower in test phase caused by the large number of support vectors, which limits its practical use. To address this problem, we propose an artificial bee colony (ABC) algorithm to search for an optimal subset of the set of support vectors obtained through the training of the SVM, such that the original discriminant function is best approximated. Experimental results show that the proposed ABC algorithm outperforms some other compared methods in terms of the classification accuracy when the solution is reduced to the same size. [ABSTRACT FROM AUTHOR]

In the field of handwriting recognition, classifier combination received much more interest than the study of powerful individual classifiers. This is mainly due to the enormous variability among the patterns to be classified, that typically requires the definition of complex high dimensional feature spaces: as the overall complexity increases, the risk of inconsistency in the decision of the classifier increases as well. In this framework, we propose a new combining method based on the use of a Bayesian Network. In particular, we suggest to reformulate the classifier combination problem as a pattern recognition one in which each input pattern is associated to a feature vector composed by the output of the classifiers to be combined. A Bayesian Network is then used to automatically infer the probability distribution for each class and eventually to perform the final classification. Experiments have been performed by using two different pools of classifiers, namely an ensemble of Learning Vector Quantization neural networks and an ensemble of Back Propagation neural networks, and handwritten specimen from the UCI Machine Learning Repository. The obtained performance has been compared with those exhibited by multi-classifier systems adopting the classifiers, but three of the most effective and widely used combining rules: the Majority Vote, the Weighted Majority Vote and the Borda Count. [ABSTRACT FROM AUTHOR]

This study borrowed a technique from molecular sequence analysis and applied it to genre identification, which is the process of determining the type or family of a given document. For example, is the document a letter, a news story, a horoscope, a joke, an advertisement, a pornographic story, etc. Genre identification allows a computer user to further filter email and web sites in a way that is totally different from topic-based methods. This study presents original research in an application of machine learning to the genre identification problem. The specific method selected for the application was neural modeling. The data for the study came from a database constructed by the author and his colleagues. The data consisted of descriptive features and the genre classification, as judged by a human, from over 5,000 different documents. Ten different genres were represented. The descriptive features consisted of 89 different measurements of each document such as average word length, the number of numeric terms, the proportion of present tense verbs, etc. The data was divided into two sets, with 75% set aside for training and 25% reserved separately for testing. The first neural network applied was a very basic single layer network that achieved 79% correct classifications on the testing data. This performance was equivalent to the previous best method on the problem, decision trees. When more complex neural networks were applied to the problem, performance increased significantly. The best performance of 86% correct classifications was achieved by a network with a single hidden layer of 300 units. Increasing the number of hidden layers, or changing the number of hidden units did not improve performance. The best score is also a significant improvement over scores obtained from topic-based filters. The neural networks were further used to determine which input features were most influential in the classifications by the networks. The average magnitude of the weights coming from each feature was computed after training. The analysis indicated that 10% of the features were not of any use to the networks. An additional 10% were very influential and were responsible for most of the performance of the networks. The remaining 80% varied between marginally useful to useful. The analysis of the features indicated that second-order information was being exploited by the networks for better performance. This means that on this problem, neural networks will outperform statistical models or other methods that only utilize first-order information. [ABSTRACT FROM AUTHOR]