Search

Analyzing and predicting the trend of price fluctuation has been receiving more and more attention, as price risk has become the focus of risk control research in heating oil futures market. A novel time series prediction model combined with the complex network method is put forward in the paper. First of all, this paper counts the cumulative time interval of different nodes in the network, and fits its growth trend with the Fourier model. Then a novel price fluctuation prediction model is established based on the effective information such as some topology properties extracted from the network. The results show that the Fourier model can predict the emergence time of new nodes in the next stage, and the established price fluctuation prediction model can infer the names of nodes in the prediction interval, so as to determine the forward-looking behavior of price evolution. Besides, liken to the NAR neural network, the prediction results obtained by the proposed method also show superiority, which has important theoretical value and academic significance for early warning and prediction of price behavior in the heating oil futures market. [ABSTRACT FROM AUTHOR]

In this paper, we propose a hybrid method to forecast the stock prices called High-order-fuzzy-fluctuation-Trends-based Back Propagation(HTBP)Neural Network model. First, we compare each value of the historical training data with the previous day's value to obtain a fluctuation trend time series (FTTS). On this basis, the FTTS blur into fuzzy time series (FFTS) based on the fluctuation of the increasing, equality, decreasing amplitude and direction. Since the relationship between FFTS and future wave trends is nonlinear, the HTBP neural network algorithm is used to find the mapping rules in the form of self-learning. Finally, the results of the algorithm output are used to predict future fluctuations. The proposed model provides some innovative features:(1)It combines fuzzy set theory and neural network algorithm to avoid overfitting problems existed in traditional models. (2)BP neural network algorithm can intelligently explore the internal rules of the actual existence of sequential data, without the need to analyze the influence factors of specific rules and the path of action. (3)The hybrid modal can reasonably remove noises from the internal rules by proper fuzzy treatment. This paper takes the TAIEX data set of Taiwan stock exchange as an example, and compares and analyzes the prediction performance of the model. The experimental results show that this method can predict the stock market in a very simple way. At the same time, we use this method to predict the Shanghai stock exchange composite index, and further verify the effectiveness and universality of the method. [ABSTRACT FROM AUTHOR]

Leveraging smart metering solutions to support energy efficiency on the individual household level poses novel research challenges in monitoring usage and providing accurate load forecasting. Forecasting electricity usage is an especially important component that can provide intelligence to smart meters. In this paper, we propose an enhanced approach for load forecasting at the household level. The impacts of residents’ daily activities and appliance usages on the power consumption of the entire household are incorporated to improve the accuracy of the forecasting model. The contributions of this paper are threefold: (1) we addressed short-term electricity load forecasting for 24 hours ahead, not on the aggregate but on the individual household level, which fits into the Residential Power Load Forecasting (RPLF) methods; (2) for the forecasting, we utilized a household specific dataset of behaviors that influence power consumption, which was derived using segmentation and sequence mining algorithms; and (3) an extensive load forecasting study using different forecasting algorithms enhanced by the household activity patterns was undertaken. [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG) is a common base signal used to monitor brain activities and diagnose sleep disorders. Manual sleep stage scoring is a time-consuming task for sleep experts and is limited by inter-rater reliability. In this paper, we propose an automatic sleep stage annotation method called SleepEEGNet using a single-channel EEG signal. The SleepEEGNet is composed of deep convolutional neural networks (CNNs) to extract time-invariant features, frequency information, and a sequence to sequence model to capture the complex and long short-term context dependencies between sleep epochs and scores. In addition, to reduce the effect of the class imbalance problem presented in the available sleep datasets, we applied novel loss functions to have an equal misclassified error for each sleep stage while training the network. We evaluated the performance of the proposed method on different single-EEG channels (i.e., Fpz-Cz and Pz-Oz EEG channels) from the Physionet Sleep-EDF datasets published in 2013 and 2018. The evaluation results demonstrate that the proposed method achieved the best annotation performance compared to current literature, with an overall accuracy of 84.26%, a macro F1-score of 79.66% and κ = 0.79. Our developed model can be applied to other sleep EEG signals and aid the sleep specialists to arrive at an accurate diagnosis. The source code is available at . [ABSTRACT FROM AUTHOR]

Antenna selection in Multiple-Input Multiple-Output (MIMO) systems has attracted increasing attention due to the challenge of keeping a balance between communication performance and computational complexity. Recently, deep learning based methods have achieved promising performance in many application fields. This paper proposed a deep learning (DL) based antenna selection technique. First, we generated the label of training antenna systems by maximizing the channel capacity. Then, we adopted the deep convolutional neural network (CNN) on the channel matrices to explicitly exploit the massive latent cues of attenuation coefficients. Finally, we used the adopted CNN to assign the class label and then select the optimal antenna subset. Experimental results demonstrate that our method can achieve better performance than the state-of-the-art baselines for data-driven based antenna selection. [ABSTRACT FROM AUTHOR]

In general, the stock prices of the same industry have a similar trend, but those of different industries do not. When investing in stocks of different industries, one should select the optimal model from lots of trading models for each industry because any model may not be suitable for capturing the stock trends of all industries. However, the study has not been carried out at present. In this paper, firstly we select 424 S&P 500 index component stocks (SPICS) and 185 CSI 300 index component stocks (CSICS) as the research objects from 2010 to 2017, divide them into 9 industries such as finance and energy respectively. Secondly, we apply 12 widely used machine learning algorithms to generate stock trading signals in different industries and execute the back-testing based on the trading signals. Thirdly, we use a non-parametric statistical test to evaluate whether there are significant differences among the trading performance evaluation indicators (PEI) of different models in the same industry. Finally, we propose a series of rules to select the optimal models for stock investment of every industry. The analytical results on SPICS and CSICS show that we can find the optimal trading models for each industry based on the statistical tests and the rules. Most importantly, the PEI of the best algorithms can be significantly better than that of the benchmark index and “Buy and Hold” strategy. Therefore, the algorithms can be used for making profits from industry stock trading. [ABSTRACT FROM AUTHOR]

In palynology, the visual classification of pollen grains from different species is a hard task which is usually tackled by human operators using microscopes. Its complete automatization would save a high quantity of resources and provide valuable improvements especially for allergy-related information systems, but also for other application fields as paleoclimate reconstruction, quality control of honey based products, collection of evidences in criminal investigations or fabric dating and tracking. This paper presents three state-of-the-art deep learning classification methods applied to the recently published POLEN23E image dataset. The three methods make use of convolutional neural networks: the first one is strictly based on the idea of transfer learning, the second one is based on feature extraction and the third one represents a hybrid approach, combining transfer learning and feature extraction. The results from the three methods are indeed very good, reaching over 97% correct classification rates in images not previously seen by the models, where other authors reported around 70. [ABSTRACT FROM AUTHOR]

Deep learning techniques, e.g., Convolutional Neural Networks (CNNs), have been explosively applied to the research in the fields of information retrieval and natural language processing. However, few research efforts have addressed semantic indexing with deep learning. The use of semantic indexing in the biomedical literature has been limited for several reasons. For instance, MEDLINE citations contain a large number of semantic labels from automatically annotated MeSH terms, and for a great deal of the literature, only the information of the title and the abstract is readily available. In this paper, we propose a Boltzmann Convolutional neural network framework (B-CNN) for biomedicine semantic indexing. In our hybrid learning framework, the CNN can adaptively deal with features of documents that have sequence relationships, and can capture context information accordingly; the Deep Boltzmann Machine (DBM) merges global (the entity in each document) and local information through its training with undirected connections. Additionally, we have designed a hierarchical coarse to fine style indexing structure for learning and classifying documents, and a novel feature extension approach with word sequence embedding and Wikipedia categorization. Comparative experiments were conducted for semantic indexing of biomedical abstract documents; these experiments verified the encouraged performance of our B-CNN model. [ABSTRACT FROM AUTHOR]

This paper presents a new method to reduce the computational cost when using Neural Networks as Language Models, during recognition, in some particular scenarios. It is based on a Neural Network that considers input contexts of different length in order to ease the use of a fallback mechanism together with the precomputation of softmax normalization constants for these inputs. The proposed approach is empirically validated, showing their capability to emulate lower order N-grams with a single Neural Network. A machine translation task shows that the proposed model constitutes a good solution to the normalization cost of the output softmax layer of Neural Networks, for some practical cases, without a significant impact in performance while improving the system speed. [ABSTRACT FROM AUTHOR]

A residual networks family with hundreds or even thousands of layers dominates major image recognition tasks, but building a network by simply stacking residual blocks inevitably limits its optimization ability. This paper proposes a novel residual network architecture, residual networks of residual networks (RoR), to dig the optimization ability of residual networks. RoR substitutes optimizing residual mapping of residual mapping for optimizing original residual mapping. In particular, RoR adds levelwise shortcut connections upon original residual networks to promote the learning capability of residual networks. More importantly, RoR can be applied to various kinds of residual networks (ResNets, Pre-ResNets, and WRN) and significantly boost their performance. Our experiments demonstrate the effectiveness and versatility of RoR, where it achieves the best performance in all residual-network-like structures. Our RoR-3-WRN58-4 + SD models achieve new state-of-the-art results on CIFAR-10, CIFAR-100, and SVHN, with the test errors of 3.77%, 19.73%, and 1.59%, respectively. RoR-3 models also achieve state-of-the-art results compared with ResNets on the ImageNet data set. [ABSTRACT FROM AUTHOR]

Combining a deep neural network with fuzzy theory, this paper proposes an advertising click-through rate (CTR) prediction approach based on a fuzzy deep neural network (FDNN). In this approach, fuzzy Gaussian-Bernoulli restricted Boltzmann machine (FGBRBM) is first applied to input raw data from advertising datasets. Next, fuzzy restricted Boltzmann machine (FRBM) is used to construct the fuzzy deep belief network (FDBN) with the unsupervised method layer by layer. Finally, fuzzy logistic regression (FLR) is utilized for modeling the CTR. The experimental results show that the proposed FDNN model outperforms several baseline models in terms of both data representation capability and robustness in advertising click log datasets with noise. [ABSTRACT FROM AUTHOR]

In this paper, we seek to improve deep neural networks by generalizing the pooling operations that play a central role in the current architectures. We pursue a careful exploration of approaches to allow pooling to learn and to adapt to complex and variable patterns. The two primary directions lie in: (1) learning a pooling function via (two strategies of) combining of max and average pooling, and (2) learning a pooling function in the form of a tree-structured fusion of pooling filters that are themselves learned. In our experiments every generalized pooling operation we explore improves performance when used in place of average or max pooling. We experimentally demonstrate that the proposed pooling operations provide a boost in invariance properties relative to conventional pooling and set the state of the art on several widely adopted benchmark datasets. These benefits come with only a light increase in computational overhead during training (ranging from additional 5 to 15 percent in time complexity) and a very modest increase in the number of model parameters (e.g., additional 1, 9, and 27 parameters for mixed, gated, and 2-level tree pooling operators, respectively). To gain more insights about our proposed pooling methods, we also visualize the learned pooling masks and the embeddings of the internal feature responses for different pooling operations. Our proposed pooling operations are easy to implement and can be applied within various deep neural network architectures. [ABSTRACT FROM PUBLISHER]

Advanced Metering Infrastructure (AMI) realizes a two-way communication of electricity data through by interconnecting with a computer network as the core component of the smart grid. Meanwhile, it brings many new security threats and the traditional intrusion detection method can’t satisfy the security requirements of AMI. In this paper, an intrusion detection system based on Online Sequence Extreme Learning Machine (OS-ELM) is established, which is used to detecting the attack in AMI and carrying out the comparative analysis with other algorithms. Simulation results show that, compared with other intrusion detection methods, intrusion detection method based on OS-ELM is more superior in detection speed and accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new automatic hyperparameter selection approach for determining the optimal network configuration (network structure and hyperparameters) for deep neural networks using particle swarm optimization (PSO) in combination with a steepest gradient descent algorithm. In the proposed approach, network configurations were coded as a set of real-number m-dimensional vectors as the individuals of the PSO algorithm in the search procedure. During the search procedure, the PSO algorithm is employed to search for optimal network configurations via the particles moving in a finite search space, and the steepest gradient descent algorithm is used to train the DNN classifier with a few training epochs (to find a local optimal solution) during the population evaluation of PSO. After the optimization scheme, the steepest gradient descent algorithm is performed with more epochs and the final solutions (pbest and gbest) of the PSO algorithm to train a final ensemble model and individual DNN classifiers, respectively. The local search ability of the steepest gradient descent algorithm and the global search capabilities of the PSO algorithm are exploited to determine an optimal solution that is close to the global optimum. We constructed several experiments on hand-written characters and biological activity prediction datasets to show that the DNN classifiers trained by the network configurations expressed by the final solutions of the PSO algorithm, employed to construct an ensemble model and individual classifier, outperform the random approach in terms of the generalization performance. Therefore, the proposed approach can be regarded an alternative tool for automatic network structure and parameter selection for deep neural networks. [ABSTRACT FROM AUTHOR]

In this article, we propose a new data mining algorithm, by which one can both capture the non-linearity in data and also find the best subset model. To produce an enhanced subset of the original variables, a preferred selection method should have the potential of adding a supplementary level of regression analysis that would capture complex relationships in the data via mathematical transformation of the predictors and exploration of synergistic effects of combined variables. The method that we present here has the potential to produce an optimal subset of variables, rendering the overall process of model selection more efficient. This algorithm introduces interpretable parameters by transforming the original inputs and also a faithful fit to the data. The core objective of this paper is to introduce a new estimation technique for the classical least square regression framework. This new automatic variable transformation and model selection method could offer an optimal and stable model that minimizes the mean square error and variability, while combining all possible subset selection methodology with the inclusion variable transformations and interactions. Moreover, this method controls multicollinearity, leading to an optimal set of explanatory variables. [ABSTRACT FROM AUTHOR]

Music emotion recognition (MER) field rapidly expanded in the last decade. Many new methods and new audio features are developed to improve the performance of MER algorithms. However, it is very difficult to compare the performance of the new methods because of the data representation diversity and scarcity of publicly available data. In this paper, we address these problems by creating a data set and a benchmark for MER. The data set that we release, a MediaEval Database for Emotional Analysis in Music (DEAM), is the largest available data set of dynamic annotations (valence and arousal annotations for 1,802 songs and song excerpts licensed under Creative Commons with 2Hz time resolution). Using DEAM, we organized the ‘Emotion in Music’ task at MediaEval Multimedia Evaluation Campaign from 2013 to 2015. The benchmark attracted, in total, 21 active teams to participate in the challenge. We analyze the results of the benchmark: the winning algorithms and feature-sets. We also describe the design of the benchmark, the evaluation procedures and the data cleaning and transformations that we suggest. The results from the benchmark suggest that the recurrent neural network based approaches combined with large feature-sets work best for dynamic MER. [ABSTRACT FROM AUTHOR]

Abstract: Selection of the optimal neural architecture to solve a pattern classification problem entails to choose the relevant input units, the number of hidden neurons and its corresponding interconnection weights. This problem has been widely studied in many research works but their solutions usually involve excessive computational cost in most of the problems and they do not provide a unique solution. This paper proposes a new technique to efficiently design the MultiLayer Perceptron (MLP) architecture for classification using the Extreme Learning Machine (ELM) algorithm. The proposed method provides a high generalization capability and a unique solution for the architecture design. Moreover, the selected final network only retains those input connections that are relevant for the classification task. Experimental results show these advantages. [Copyright &y& Elsevier]

The Levenberg–Marquardt (LM) algorithm is a nonlinear parameter learning algorithm that converges accurately and quickly. This paper demonstrates for the first time to our knowledge, a real-time implementation of the LM algorithm on field programmable gate arrays (FPGAs). It was used to train neural networks to solve the eXclusive Or function (XOR), and for 3D-to-2D camera calibration parameter estimation. A Xilinx Virtex-5 ML506 was used to implement the LMA as a hardware-in-the-loop system. The XOR function was approximated in only 13 iterations from zero initial conditions, usually the same function is approximated in thousands of iterations using the error backpropagation algorithm. Also, this type of training not only reduced the number of iterations but also achieved a speed up in excess of 3 \times 10^6 when compared to the software implementation. A real-time camera calibration and parameter estimation was performed successfully on FPGAs. Compared to the software implementation the FPGA implementation led to an increase in the mean squared error and standard deviation by only 17.94% and 8.04% respectively. The FPGA increased the calibration speed by a factor of 1.41 \times 10^6. There are a wide range of systems problems solved via nonlinear parameter optimization, this study demonstrated that a hardware solution for systems such as automated optical inspection systems or systems dealing with projective geometry estimation and motion compensation systems in robotic vision systems is possible in real time. [ABSTRACT FROM PUBLISHER]

Abstract: This paper presents an overview of a data analysis method based on self-organizing maps (SOM), a well-known unsupervised neural network learning algorithm, which was applied to a lead-free wave soldering process. The aim of the study was to determine whether the neural network modeling method could be a useful and time-saving way to analyze data from a discrete manufacturing process, such as wave soldering, which is a widely used technique in the electronics industry to solder components on printed circuit boards. The data variables were mostly various process parameters, but also some solder defect numbers were present in the data as a measure of the product quality. The data analysis procedure went as follows. At first, the process data were modeled using the SOM-algorithm. Next, the neuron reference vectors of the formed map were clustered to reveal the desired dominating elements of each territory of the map. At the final stage, the clusters were utilized as sub-models to indicate variable dependencies in these sub-models. The results show that the method presented here can be a good way to analyze this type of process data because interesting interactions between certain process parameters and solder defects were found by means of this data-driven modeling method. [Copyright &y& Elsevier]

The paper addresses the problem of predicting hourly load demand using adaptive artificial neural networks (ANNs). A particle swarm optimization (PSO) algorithm is employed to adjust the network's weights in the training phase of the ANNs. The advantage of using a PSO algorithm over other conventional training algorithms such as the back-propagation (BP) is that potential solutions will be flown through the problem hyperspace with accelerated movement towards the best solution. Thus the training phase should result in obtaining the weights configuration associated with the minimum output error. Data are wavelet transformed during the preprocessing stage and then inserted into the neural network to extract redundant information from the load curve. This results in better load characterization which creates a more reliable forecasting model. The transformed data of historical load and weather information were trained and tested over various periods of time. The generalized error estimation is done by using the reverse part of the data as a "test" set. The results were compared with traditional BP algorithm and offered a high forecasting precision. [ABSTRACT FROM AUTHOR]

In multilabel learning, each instance in the training set is associated with a set of labels and the task is to output a label set whose size is unknown a priori for each unseen instance. In this paper, this problem is addressed in the way that a neural network algorithm named BP-MLL, i.e., Backpropagation for Multilabel Learning, is proposed. It is derived from the popular Backpropagation algorithm through employing a novel error function capturing the characteristics of multilabel learning, i.e., the labels belonging to an instance should be ranked higher than those not belonging to that instance. Applications to two real-world multilabel learning problems, i.e., functional genomics and text categorization, show that the performance of BP-MLL is superior to that of some well-established multilabel learning algorithms. [ABSTRACT FROM AUTHOR]