Search

Machine learning (ML), a form of artificial intelligence (AI), is being increasingly employed in neurology. Reported performance metrics often match or exceed the efficiency of average clinicians. The neurologist is easily baffled by the underlying concepts and terminologies associated with ML studies. The superlative performance metrics of ML algorithms often hide the opaque nature of its inner workings. Questions regarding ML model's interpretability and reproducibility of its results in real-world scenarios, need emphasis. Given an abundance of time and information, the expert clinician should be able to deliver comparable predictions to ML models, a useful benchmark while evaluating its performance. Predictive performance metrics of ML models should not be confused with causal inference between its input and output. ML and clinical gestalt should compete in a randomized controlled trial before they can complement each other for screening, triaging, providing second opinions and modifying treatment. [ABSTRACT FROM AUTHOR]

The 3D measurement system based on line-structured light uses a camera to capture laser stripes due to changing in the shape of an object, and uses the acquired pixel coordinates for 3D reconstruction. System calibration is an important step in 3D measurement. The current camera calibration algorithm research mainly focuses on improving the algorithm itself, and there is less research on the influence of external factors. This paper proposes a coplanar hybrid calibration algorithm based on the error screening model by combining the error screening model, mathematical model and neural network model. It is mainly divided into two steps. The first step is to use the radial array constraint calibration algorithm based on the error screening model to solve the camera's internal and external parameters. The second step uses the camera internal and external parameters obtained in the first step to convert the pixel coordinates into real three-dimensional coordinates, and compares the calculated three-dimensional coordinates with the actual coordinates. Using machine learning to establish a compensation network, get a compensation function, and use the resulting 3D world coordinates to perform point cloud stitching. Experiments show that compared with the traditional calibration algorithm, the calibration algorithm has a small error and reduces the calibration error by about 6.5%. [ABSTRACT FROM AUTHOR]

Analyzing and predicting the trend of price fluctuation has been receiving more and more attention, as price risk has become the focus of risk control research in heating oil futures market. A novel time series prediction model combined with the complex network method is put forward in the paper. First of all, this paper counts the cumulative time interval of different nodes in the network, and fits its growth trend with the Fourier model. Then a novel price fluctuation prediction model is established based on the effective information such as some topology properties extracted from the network. The results show that the Fourier model can predict the emergence time of new nodes in the next stage, and the established price fluctuation prediction model can infer the names of nodes in the prediction interval, so as to determine the forward-looking behavior of price evolution. Besides, liken to the NAR neural network, the prediction results obtained by the proposed method also show superiority, which has important theoretical value and academic significance for early warning and prediction of price behavior in the heating oil futures market. [ABSTRACT FROM AUTHOR]

A residual networks family with hundreds or even thousands of layers dominates major image recognition tasks, but building a network by simply stacking residual blocks inevitably limits its optimization ability. This paper proposes a novel residual network architecture, residual networks of residual networks (RoR), to dig the optimization ability of residual networks. RoR substitutes optimizing residual mapping of residual mapping for optimizing original residual mapping. In particular, RoR adds levelwise shortcut connections upon original residual networks to promote the learning capability of residual networks. More importantly, RoR can be applied to various kinds of residual networks (ResNets, Pre-ResNets, and WRN) and significantly boost their performance. Our experiments demonstrate the effectiveness and versatility of RoR, where it achieves the best performance in all residual-network-like structures. Our RoR-3-WRN58-4 + SD models achieve new state-of-the-art results on CIFAR-10, CIFAR-100, and SVHN, with the test errors of 3.77%, 19.73%, and 1.59%, respectively. RoR-3 models also achieve state-of-the-art results compared with ResNets on the ImageNet data set. [ABSTRACT FROM AUTHOR]

Combining a deep neural network with fuzzy theory, this paper proposes an advertising click-through rate (CTR) prediction approach based on a fuzzy deep neural network (FDNN). In this approach, fuzzy Gaussian-Bernoulli restricted Boltzmann machine (FGBRBM) is first applied to input raw data from advertising datasets. Next, fuzzy restricted Boltzmann machine (FRBM) is used to construct the fuzzy deep belief network (FDBN) with the unsupervised method layer by layer. Finally, fuzzy logistic regression (FLR) is utilized for modeling the CTR. The experimental results show that the proposed FDNN model outperforms several baseline models in terms of both data representation capability and robustness in advertising click log datasets with noise. [ABSTRACT FROM AUTHOR]

In this paper, we propose a hybrid method to forecast the stock prices called High-order-fuzzy-fluctuation-Trends-based Back Propagation(HTBP)Neural Network model. First, we compare each value of the historical training data with the previous day's value to obtain a fluctuation trend time series (FTTS). On this basis, the FTTS blur into fuzzy time series (FFTS) based on the fluctuation of the increasing, equality, decreasing amplitude and direction. Since the relationship between FFTS and future wave trends is nonlinear, the HTBP neural network algorithm is used to find the mapping rules in the form of self-learning. Finally, the results of the algorithm output are used to predict future fluctuations. The proposed model provides some innovative features:(1)It combines fuzzy set theory and neural network algorithm to avoid overfitting problems existed in traditional models. (2)BP neural network algorithm can intelligently explore the internal rules of the actual existence of sequential data, without the need to analyze the influence factors of specific rules and the path of action. (3)The hybrid modal can reasonably remove noises from the internal rules by proper fuzzy treatment. This paper takes the TAIEX data set of Taiwan stock exchange as an example, and compares and analyzes the prediction performance of the model. The experimental results show that this method can predict the stock market in a very simple way. At the same time, we use this method to predict the Shanghai stock exchange composite index, and further verify the effectiveness and universality of the method. [ABSTRACT FROM AUTHOR]

Brain tumors are major causes of death in today's world and methods of detecting them prematurely require vast improvement. The objective of this project is to detect the tumors early from MR image scans by utilizing deep convolutional networks to locate the tumor. The tumor is divided at first in the main stage and the generated bounding box is utilized for the center of the tumor in second step. This is trailed by division based on the bounding box of the tumor center division result. Examinations are performed with the BraTS 2017 validation set. This is a numerous division issue. [ABSTRACT FROM AUTHOR]

In palynology, the visual classification of pollen grains from different species is a hard task which is usually tackled by human operators using microscopes. Its complete automatization would save a high quantity of resources and provide valuable improvements especially for allergy-related information systems, but also for other application fields as paleoclimate reconstruction, quality control of honey based products, collection of evidences in criminal investigations or fabric dating and tracking. This paper presents three state-of-the-art deep learning classification methods applied to the recently published POLEN23E image dataset. The three methods make use of convolutional neural networks: the first one is strictly based on the idea of transfer learning, the second one is based on feature extraction and the third one represents a hybrid approach, combining transfer learning and feature extraction. The results from the three methods are indeed very good, reaching over 97% correct classification rates in images not previously seen by the models, where other authors reported around 70. [ABSTRACT FROM AUTHOR]

Abstract: The hearing sense is one of the five senses essential for human life. The hearing sense is required to acquire language, learn how to speak, and gain general knowledge. Most importantly, hearing enables individuals to communicate and socialize with the world around them. Deaf people suffer from hearing loss which affects many aspects of their lives; they cannot communicate or socialize effectively with normal people around them. In addition, the safety of deaf people is compromised in many scenarios because they cannot take the right decision emanating from the process of analyzing sounds in the brain, especially alarming sounds. The treatment of hearing loss is dependent on its level. Medicine fails to treat many cases of severe or profound deafness; therefore, people lose hope to live normally. In this paper, we propose a Deaf Assistance Digital System (DADS) that aims to convert major alarm sounds and words which are not distinguishable by deaf people into recognizable alerts for deaf people. The proposed system strives to help deaf people live normally by alerting them about sounds like baby crying or door bell. Moreover, DADS enhances the safety of deaf people by alerting them about alarming sounds and cautionary words. DADS consists of a speech recognition engine and a sound recognition engine. The speech recognition engine is based on the Soundx algorithm while the sound recognition system is implemented using neural networks. The implemented sound recognition and voice recognition engines have more than 90% accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we seek to improve deep neural networks by generalizing the pooling operations that play a central role in the current architectures. We pursue a careful exploration of approaches to allow pooling to learn and to adapt to complex and variable patterns. The two primary directions lie in: (1) learning a pooling function via (two strategies of) combining of max and average pooling, and (2) learning a pooling function in the form of a tree-structured fusion of pooling filters that are themselves learned. In our experiments every generalized pooling operation we explore improves performance when used in place of average or max pooling. We experimentally demonstrate that the proposed pooling operations provide a boost in invariance properties relative to conventional pooling and set the state of the art on several widely adopted benchmark datasets. These benefits come with only a light increase in computational overhead during training (ranging from additional 5 to 15 percent in time complexity) and a very modest increase in the number of model parameters (e.g., additional 1, 9, and 27 parameters for mixed, gated, and 2-level tree pooling operators, respectively). To gain more insights about our proposed pooling methods, we also visualize the learned pooling masks and the embeddings of the internal feature responses for different pooling operations. Our proposed pooling operations are easy to implement and can be applied within various deep neural network architectures. [ABSTRACT FROM PUBLISHER]

In this paper, we suggest basing the development of classification methods on traditional techniques but approximating them in whole or in part with artificial neural networks (ANNs). Compared to a direct ANN approach, the underlying traditional method is often easier to analyse. Classification failures can be better understood and corrected for, while at the same time faster execution can be obtained due to the parallel ANN structure. Furthermore, such a two‐step design philosophy partly eliminates the 'guesswork' associated with the design of ANNs. The expected gain is thus that the benefits from the traditional field, and the ANN field can be obtained by combining the best features from both fields. We illustrate our approach by working through an explicit example, namely, a Nearest Neighbour classifier applied to a subset of the MNIST database of handwritten digits. Two different approaches are discussed for how to translate the traditional method into an ANN. The first approach is based on a constructive implementation which directly reflects the original algorithm. The second approach uses ANNs to approximate the whole, or part of, the original method. An important part of the approach is to show how improvements can be introduced. In line with the presented philosophy, this is done by extending the traditional method in several ways followed by ANN approximation of the modified algorithms. The extensions are based on a windowed version of the nearest neighbour algorithm. We show that the improvements carry over to the ANN implementations. We further investigate the stability of the solutions by modifying the training set. It is shown that the errors do not change significantly. This also holds true for the ANN approximations providing confidence that the two‐step strategy is robust. [ABSTRACT FROM AUTHOR]

Modern computing architecture based on the separation of memory and processing leads to a well known problem called the von Neumann bottleneck, a restrictive limit on the data bandwidth between CPU and RAM. This paper introduces a new approach to computing we call AHaH computing where memory and processing are combined. The idea is based on the attractor dynamics of volatile dissipative electronics inspired by biological systems, presenting an attractive alternative architecture that is able to adapt, self-repair, and learn from interactions with the environment. We envision that both von Neumann and AHaH computing architectures will operate together on the same machine, but that the AHaH computing processor may reduce the power consumption and processing time for certain adaptive learning tasks by orders of magnitude. The paper begins by drawing a connection between the properties of volatility, thermodynamics, and Anti-Hebbian and Hebbian (AHaH) plasticity. We show how AHaH synaptic plasticity leads to attractor states that extract the independent components of applied data streams and how they form a computationally complete set of logic functions. After introducing a general memristive device model based on collections of metastable switches, we show how adaptive synaptic weights can be formed from differential pairs of incremental memristors. We also disclose how arrays of synaptic weights can be used to build a neural node circuit operating AHaH plasticity. By configuring the attractor states of the AHaH node in different ways, high level machine learning functions are demonstrated. This includes unsupervised clustering, supervised and unsupervised classification, complex signal prediction, unsupervised robotic actuation and combinatorial optimization of procedures–all key capabilities of biological nervous systems and modern machine learning algorithms with real world application. [ABSTRACT FROM AUTHOR]

In this paper, a learning algorithm considering derivative information is proposed for neural networks. Based on backpropagation (BP) algorithm, this algorithm takes the derivative information of the samples into account. Because the partial derivative of the output variable of neural networks to the input variable can be obtained from the weights of the networks, the error between the calculated value and the theoretical value of the partial derivative can be reduced by adjusting the weights of the networks. This algorithm can effectively avoid the over-fitting and improve the generalization capability for the networks. The neural networks considering derivative information has been shown by experimental results to have better generalization performance than BP networks. [ABSTRACT FROM AUTHOR]

Virtual network embedding arranges virtual network services onto substrate network components. The performance of embedding algorithms determines the effectiveness and efficiency of a virtualized network, making it a critical part of the network virtualization technology. To achieve better performance, the algorithm needs to automatically detect the network status which is complicated and changes in a time-varying manner, and to dynamically provide solutions that can best fit the current network status. However, most existing algorithms fail to provide automatic embedding solutions in an acceptable running time. In this paper, we combine deep reinforcement learning with a novel neural network structure based on graph convolutional networks, and propose a new and efficient algorithm for automatic virtual network embedding. In addition, a parallel reinforcement learning framework is used in training along with a newly-designed multi-objective reward function, which has proven beneficial to the proposed algorithm for automatic embedding of virtual networks. Extensive simulation results under different scenarios show that our algorithm achieves best performance on most metrics compared with the existing state-of-the-art solutions, with upto 39.6% and 70.6% improvement on acceptance ratio and average revenue, respectively. Moreover, the results also demonstrate that the proposed solution possesses good robustness. [ABSTRACT FROM AUTHOR]

In multilabel learning, each instance in the training set is associated with a set of labels and the task is to output a label set whose size is unknown a priori for each unseen instance. In this paper, this problem is addressed in the way that a neural network algorithm named BP-MLL, i.e., Backpropagation for Multilabel Learning, is proposed. It is derived from the popular Backpropagation algorithm through employing a novel error function capturing the characteristics of multilabel learning, i.e., the labels belonging to an instance should be ranked higher than those not belonging to that instance. Applications to two real-world multilabel learning problems, i.e., functional genomics and text categorization, show that the performance of BP-MLL is superior to that of some well-established multilabel learning algorithms. [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG) is a common base signal used to monitor brain activities and diagnose sleep disorders. Manual sleep stage scoring is a time-consuming task for sleep experts and is limited by inter-rater reliability. In this paper, we propose an automatic sleep stage annotation method called SleepEEGNet using a single-channel EEG signal. The SleepEEGNet is composed of deep convolutional neural networks (CNNs) to extract time-invariant features, frequency information, and a sequence to sequence model to capture the complex and long short-term context dependencies between sleep epochs and scores. In addition, to reduce the effect of the class imbalance problem presented in the available sleep datasets, we applied novel loss functions to have an equal misclassified error for each sleep stage while training the network. We evaluated the performance of the proposed method on different single-EEG channels (i.e., Fpz-Cz and Pz-Oz EEG channels) from the Physionet Sleep-EDF datasets published in 2013 and 2018. The evaluation results demonstrate that the proposed method achieved the best annotation performance compared to current literature, with an overall accuracy of 84.26%, a macro F1-score of 79.66% and κ = 0.79. Our developed model can be applied to other sleep EEG signals and aid the sleep specialists to arrive at an accurate diagnosis. The source code is available at . [ABSTRACT FROM AUTHOR]

Antenna selection in Multiple-Input Multiple-Output (MIMO) systems has attracted increasing attention due to the challenge of keeping a balance between communication performance and computational complexity. Recently, deep learning based methods have achieved promising performance in many application fields. This paper proposed a deep learning (DL) based antenna selection technique. First, we generated the label of training antenna systems by maximizing the channel capacity. Then, we adopted the deep convolutional neural network (CNN) on the channel matrices to explicitly exploit the massive latent cues of attenuation coefficients. Finally, we used the adopted CNN to assign the class label and then select the optimal antenna subset. Experimental results demonstrate that our method can achieve better performance than the state-of-the-art baselines for data-driven based antenna selection. [ABSTRACT FROM AUTHOR]

In general, the stock prices of the same industry have a similar trend, but those of different industries do not. When investing in stocks of different industries, one should select the optimal model from lots of trading models for each industry because any model may not be suitable for capturing the stock trends of all industries. However, the study has not been carried out at present. In this paper, firstly we select 424 S&P 500 index component stocks (SPICS) and 185 CSI 300 index component stocks (CSICS) as the research objects from 2010 to 2017, divide them into 9 industries such as finance and energy respectively. Secondly, we apply 12 widely used machine learning algorithms to generate stock trading signals in different industries and execute the back-testing based on the trading signals. Thirdly, we use a non-parametric statistical test to evaluate whether there are significant differences among the trading performance evaluation indicators (PEI) of different models in the same industry. Finally, we propose a series of rules to select the optimal models for stock investment of every industry. The analytical results on SPICS and CSICS show that we can find the optimal trading models for each industry based on the statistical tests and the rules. Most importantly, the PEI of the best algorithms can be significantly better than that of the benchmark index and “Buy and Hold” strategy. Therefore, the algorithms can be used for making profits from industry stock trading. [ABSTRACT FROM AUTHOR]

This paper presents a causal discovery model for time series analysis, which can find several causal lags of one variable over another, presenting the results by means of a temporal causal graph. The proposed model is based on multi-head attention neural networks and includes convolutional networks, allowing high prediction performance and the ability to estimate confounders. The developed framework presents an innovative methodology for validating potential causes: through a second neural network model, a direct and inverse model is trained with the potential causal lag between the causal variable under study and the target variable. In addition, another potential causal discovery method based on a non-linear correlation test is implemented. The main results show that the proposed model outperforms other causal algorithms concerning the F1 Score and achieves a considerable performance in finding temporal lags. In addition, the network fit (evaluated through the mean squared error) is significantly improved when only the causal variables predicted by the proposed algorithm (with their corresponding causal lags) are used for model training, relative to the incorporation of the entire time series dataset for the prediction of the target variable. • Causal discovery model able to find several causal lags of one variable over another. • Proposed model is based on multi-head attention and convolutional networks. • Proposed model allows high prediction performance and finding confounders. • The framework presents methodology for validating potential causes. [ABSTRACT FROM AUTHOR]

Deep learning techniques, e.g., Convolutional Neural Networks (CNNs), have been explosively applied to the research in the fields of information retrieval and natural language processing. However, few research efforts have addressed semantic indexing with deep learning. The use of semantic indexing in the biomedical literature has been limited for several reasons. For instance, MEDLINE citations contain a large number of semantic labels from automatically annotated MeSH terms, and for a great deal of the literature, only the information of the title and the abstract is readily available. In this paper, we propose a Boltzmann Convolutional neural network framework (B-CNN) for biomedicine semantic indexing. In our hybrid learning framework, the CNN can adaptively deal with features of documents that have sequence relationships, and can capture context information accordingly; the Deep Boltzmann Machine (DBM) merges global (the entity in each document) and local information through its training with undirected connections. Additionally, we have designed a hierarchical coarse to fine style indexing structure for learning and classifying documents, and a novel feature extension approach with word sequence embedding and Wikipedia categorization. Comparative experiments were conducted for semantic indexing of biomedical abstract documents; these experiments verified the encouraged performance of our B-CNN model. [ABSTRACT FROM AUTHOR]

This paper presents a new method to reduce the computational cost when using Neural Networks as Language Models, during recognition, in some particular scenarios. It is based on a Neural Network that considers input contexts of different length in order to ease the use of a fallback mechanism together with the precomputation of softmax normalization constants for these inputs. The proposed approach is empirically validated, showing their capability to emulate lower order N-grams with a single Neural Network. A machine translation task shows that the proposed model constitutes a good solution to the normalization cost of the output softmax layer of Neural Networks, for some practical cases, without a significant impact in performance while improving the system speed. [ABSTRACT FROM AUTHOR]

Model-based iterative reconstruction algorithms for low-dose X-ray computed tomography (CT) are computationally expensive. To address this problem, we recently proposed a deep convolutional neural network (CNN) for low-dose X-ray CT and won the second place in 2016 AAPM Low-Dose CT Grand Challenge. However, some of the textures were not fully recovered. To address this problem, here we propose a novel framelet-based denoising algorithm using wavelet residual network which synergistically combines the expressive power of deep learning and the performance guarantee from the framelet-based denoising algorithms. The new algorithms were inspired by the recent interpretation of the deep CNN as a cascaded convolution framelet signal representation. Extensive experimental results confirm that the proposed networks have significantly improved performance and preserve the detail texture of the original images. [ABSTRACT FROM AUTHOR]

Advanced Metering Infrastructure (AMI) realizes a two-way communication of electricity data through by interconnecting with a computer network as the core component of the smart grid. Meanwhile, it brings many new security threats and the traditional intrusion detection method can’t satisfy the security requirements of AMI. In this paper, an intrusion detection system based on Online Sequence Extreme Learning Machine (OS-ELM) is established, which is used to detecting the attack in AMI and carrying out the comparative analysis with other algorithms. Simulation results show that, compared with other intrusion detection methods, intrusion detection method based on OS-ELM is more superior in detection speed and accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new automatic hyperparameter selection approach for determining the optimal network configuration (network structure and hyperparameters) for deep neural networks using particle swarm optimization (PSO) in combination with a steepest gradient descent algorithm. In the proposed approach, network configurations were coded as a set of real-number m-dimensional vectors as the individuals of the PSO algorithm in the search procedure. During the search procedure, the PSO algorithm is employed to search for optimal network configurations via the particles moving in a finite search space, and the steepest gradient descent algorithm is used to train the DNN classifier with a few training epochs (to find a local optimal solution) during the population evaluation of PSO. After the optimization scheme, the steepest gradient descent algorithm is performed with more epochs and the final solutions (pbest and gbest) of the PSO algorithm to train a final ensemble model and individual DNN classifiers, respectively. The local search ability of the steepest gradient descent algorithm and the global search capabilities of the PSO algorithm are exploited to determine an optimal solution that is close to the global optimum. We constructed several experiments on hand-written characters and biological activity prediction datasets to show that the DNN classifiers trained by the network configurations expressed by the final solutions of the PSO algorithm, employed to construct an ensemble model and individual classifier, outperform the random approach in terms of the generalization performance. Therefore, the proposed approach can be regarded an alternative tool for automatic network structure and parameter selection for deep neural networks. [ABSTRACT FROM AUTHOR]

Following the great success of deep convolutional neural networks (CNNs) in computer vision, this paper proposes a complex-valued CNN (CV-CNN) specifically for synthetic aperture radar (SAR) image interpretation. It utilizes both amplitude and phase information of complex SAR imagery. All elements of CNN including input-output layer, convolution layer, activation function, and pooling layer are extended to the complex domain. Moreover, a complex backpropagation algorithm based on stochastic gradient descent is derived for CV-CNN training. The proposed CV-CNN is then tested on the typical polarimetric SAR image classification task which classifies each pixel into known terrain types via supervised training. Experiments with the benchmark data sets of Flevoland and Oberpfaffenhofen show that the classification error can be further reduced if employing CV-CNN instead of conventional real-valued CNN with the same degrees of freedom. The performance of CV-CNN is comparable to that of existing state-of-the-art methods in terms of overall classification accuracy. [ABSTRACT FROM PUBLISHER]

In this article, we propose a new data mining algorithm, by which one can both capture the non-linearity in data and also find the best subset model. To produce an enhanced subset of the original variables, a preferred selection method should have the potential of adding a supplementary level of regression analysis that would capture complex relationships in the data via mathematical transformation of the predictors and exploration of synergistic effects of combined variables. The method that we present here has the potential to produce an optimal subset of variables, rendering the overall process of model selection more efficient. This algorithm introduces interpretable parameters by transforming the original inputs and also a faithful fit to the data. The core objective of this paper is to introduce a new estimation technique for the classical least square regression framework. This new automatic variable transformation and model selection method could offer an optimal and stable model that minimizes the mean square error and variability, while combining all possible subset selection methodology with the inclusion variable transformations and interactions. Moreover, this method controls multicollinearity, leading to an optimal set of explanatory variables. [ABSTRACT FROM AUTHOR]

This paper proposes a novel steam valving controller using radial basis function (RBF) networks-based approximate model method. Approximate model method is a kind of direct linearization approach that is derived based on the approximation of the plant's input-output model via Taylor expansion. RBF networks are used to identify the plant to implement the approximate model control law. In order to improve the performance of the approximate model controller, RBF networks weights are adjusted online using BP algorithms with an adaptive learning rate. Several simulations results demonstrate the effectiveness of the proposed controller for team valving control. [ABSTRACT FROM AUTHOR]

The paper addresses the problem of predicting hourly load demand using adaptive artificial neural networks (ANNs). A particle swarm optimization (PSO) algorithm is employed to adjust the network's weights in the training phase of the ANNs. The advantage of using a PSO algorithm over other conventional training algorithms such as the back-propagation (BP) is that potential solutions will be flown through the problem hyperspace with accelerated movement towards the best solution. Thus the training phase should result in obtaining the weights configuration associated with the minimum output error. Data are wavelet transformed during the preprocessing stage and then inserted into the neural network to extract redundant information from the load curve. This results in better load characterization which creates a more reliable forecasting model. The transformed data of historical load and weather information were trained and tested over various periods of time. The generalized error estimation is done by using the reverse part of the data as a "test" set. The results were compared with traditional BP algorithm and offered a high forecasting precision. [ABSTRACT FROM AUTHOR]

The process of learning the behavior of a given pro- gram by using machine-learning techniques (based on system-call audit data) is effective to detect intrusions. Rule learning, neural networks, statistics, and hidden Markov models (HMMs) are some of the kinds of representative methods for intrusion detection. Among them, neural networks are known for good performance in learning system-call sequences. In order to apply this knowledge to real-world problems successfully, it is important to determine the structures and weights of these call sequences. However, finding the appropriate structures requires very long time periods be- cause there are no suitable analytical solutions. In this paper, a novel intrusion-detection technique based on evolutionary neural networks (ENNs) is proposed. One advantage of using ENNs is that it takes less time to obtain superior neural networks than when using conventional approaches. This is because they discover the structures and weights of the neural networks simultaneously. Experimental results with the 1999 Defense Advanced Research Projects Agency (DARPA) Intrusion Detection Evaluation (IDEVAL) data confirm that ENNs are promising tools for intrusion detection. [ABSTRACT FROM AUTHOR]

Leveraging smart metering solutions to support energy efficiency on the individual household level poses novel research challenges in monitoring usage and providing accurate load forecasting. Forecasting electricity usage is an especially important component that can provide intelligence to smart meters. In this paper, we propose an enhanced approach for load forecasting at the household level. The impacts of residents’ daily activities and appliance usages on the power consumption of the entire household are incorporated to improve the accuracy of the forecasting model. The contributions of this paper are threefold: (1) we addressed short-term electricity load forecasting for 24 hours ahead, not on the aggregate but on the individual household level, which fits into the Residential Power Load Forecasting (RPLF) methods; (2) for the forecasting, we utilized a household specific dataset of behaviors that influence power consumption, which was derived using segmentation and sequence mining algorithms; and (3) an extensive load forecasting study using different forecasting algorithms enhanced by the household activity patterns was undertaken. [ABSTRACT FROM AUTHOR]

Music emotion recognition (MER) field rapidly expanded in the last decade. Many new methods and new audio features are developed to improve the performance of MER algorithms. However, it is very difficult to compare the performance of the new methods because of the data representation diversity and scarcity of publicly available data. In this paper, we address these problems by creating a data set and a benchmark for MER. The data set that we release, a MediaEval Database for Emotional Analysis in Music (DEAM), is the largest available data set of dynamic annotations (valence and arousal annotations for 1,802 songs and song excerpts licensed under Creative Commons with 2Hz time resolution). Using DEAM, we organized the ‘Emotion in Music’ task at MediaEval Multimedia Evaluation Campaign from 2013 to 2015. The benchmark attracted, in total, 21 active teams to participate in the challenge. We analyze the results of the benchmark: the winning algorithms and feature-sets. We also describe the design of the benchmark, the evaluation procedures and the data cleaning and transformations that we suggest. The results from the benchmark suggest that the recurrent neural network based approaches combined with large feature-sets work best for dynamic MER. [ABSTRACT FROM AUTHOR]

The Levenberg–Marquardt (LM) algorithm is a nonlinear parameter learning algorithm that converges accurately and quickly. This paper demonstrates for the first time to our knowledge, a real-time implementation of the LM algorithm on field programmable gate arrays (FPGAs). It was used to train neural networks to solve the eXclusive Or function (XOR), and for 3D-to-2D camera calibration parameter estimation. A Xilinx Virtex-5 ML506 was used to implement the LMA as a hardware-in-the-loop system. The XOR function was approximated in only 13 iterations from zero initial conditions, usually the same function is approximated in thousands of iterations using the error backpropagation algorithm. Also, this type of training not only reduced the number of iterations but also achieved a speed up in excess of 3 \times 10^6 when compared to the software implementation. A real-time camera calibration and parameter estimation was performed successfully on FPGAs. Compared to the software implementation the FPGA implementation led to an increase in the mean squared error and standard deviation by only 17.94% and 8.04% respectively. The FPGA increased the calibration speed by a factor of 1.41 \times 10^6. There are a wide range of systems problems solved via nonlinear parameter optimization, this study demonstrated that a hardware solution for systems such as automated optical inspection systems or systems dealing with projective geometry estimation and motion compensation systems in robotic vision systems is possible in real time. [ABSTRACT FROM PUBLISHER]

Developing a model based digital human meridian system is one of the interesting ways of understanding and improving acupuncture treatment, safety analysis for acupuncture operation, doctor training, or treatment scheme evaluation. In accomplishing this task, how to construct a proper model to describe the behavior of human meridian systems is one of the very important issues. From experiments, it has been found that the hysteresis phenomenon occurs in the relations between stimulation input and the corresponding response of meridian systems. Therefore, the modeling of hysteresis in a human meridian system is an unavoidable task for the construction of model based digital human meridian systems. As hysteresis is a nonsmooth, nonlinear and dynamic system with a multi-valued mapping, the conventional identification method is difficult to be employed to model its behavior directly. In this paper, a neural network based identification method of hysteresis occurring in human meridian systems is presented. In this modeling scheme, an expanded input space is constructed to transform the multi-valued mapping of hysteresis into a one-to-one mapping. For this purpose, a modified hysteretic operator is proposed to handle the extremum-missing problem. Then, based on the constructed expanded input space with the modified hysteretic operator, the so-called Extreme Learning Machine (ELM) neural network is utilized to model hysteresis inherent in human meridian systems. As hysteresis in meridian system is a dynamic system, a dynamic ELM neural network is developed. In the proposed dynamic ELM neural network, the output state of each hidden neuron is fed back to its own input to describe the dynamic behavior of hysteresis. The training of the recurrent ELM neural network is based on the least-squares algorithm with QR decomposition. [ABSTRACT FROM AUTHOR]

Radial basis function (RBF) networks have advantages of easy design, good generalization, strong tolerance to input noise, and online learning ability. The properties of RBF networks make it very suitable to design flexible control systems. This paper presents a review on different approaches of designing and training RBF networks. The recently developed algorithm is introduced for designing compact RBF networks and performing efficient training process. At last, several problems are applied to test the main properties of RBF networks, including their generalization ability, tolerance to input noise, and online learning ability. RBF networks are also compared with traditional neural networks and fuzzy inference systems. [ABSTRACT FROM AUTHOR]

Classification is one of the data mining problems receiving enormous attention in the database community. Although artificial neural networks (ANNs) have been successfully applied in a wide range of machine learning applications, they are however often regarded as black boxes, i.e., their predictions cannot be explained. To enhance the explanation of ANNs, a novel algorithm to extract symbolic rules from ANNs has been proposed in this paper. ANN methods have not been effectively utilized for data mining tasks because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by human experts. With the proposed approach, concise symbolic rules with high accuracy, that are easily explainable, can be extracted from the trained ANNs. Extracted rules are comparable with other methods in terms of number of rules, average number of conditions for a rule, and the accuracy. The effectiveness of the proposed approach is clearly demonstrated by the experimental results on a set of benchmark data mining classification problems. [ABSTRACT FROM AUTHOR]

This paper deals with the parallel layer perceptron (PLP) complexity control, bias and variance dilemma, using a multiobjective (MOBJ) training algorithm. To control the bias and variance the training process is rewritten as a bi-objective problem, considering the minimization of both training error and norm of the weight vector, which is a measure of the network complexity. This method is applied to regression and classification problems and compared with several other training procedures and topologies. The results show that the PLP MOBJ training algorithm presents good generalization results, outperforming traditional methods in the tested examples. [ABSTRACT FROM AUTHOR]

An NNC-Tree is a decision tree (DT) with each nonterminal node containing a nearest neighbor classifier (NNC). Compared with the conventional axis-parallel DTs (APDTs), the NNC-Trees can be more efficient, because the decision boundary made by an NNC is more complex than an axis- parallel hyperplane. Compared with single-layer NNCs, the NNC-Trees can classify given data in a hierarchical structure that is often useful for many applications. This paper proposes an algorithm for inducing NNC-Trees based on the R4-rule, which was proposed by the author for finding the smallest nearest neighbor based multilayer perceptrons (NN-MLPs). There are mainly two contributions here. 1) A heuristic but effective method is given to define the teacher signals (group labels) for the data assigned to each nonterminal node. 2) The R4-rule is modified so that an NNC with proper size can be designed automatically in each nonterminal node. Experiments with several public databases show that the proposed algorithm can produce NNC-Trees effectively and efficiently. [ABSTRACT FROM AUTHOR]

Portal hypertension is a complex medical condition characterized by elevated blood pressure in the portal venous system. The conventional diagnosis of such disease often involves invasive procedures such as liver biopsy, endoscopy, or imaging techniques with contrast agents, which can be uncomfortable for patients and carry inherent risks. This study presents a deep neural network method in support of the non-invasive diagnosis of portal hypertension in patients with chronic liver diseases. The proposed method utilizes readily available clinical data, thus eliminating the need for invasive procedures. A dataset composed of standard laboratory parameters is used to train and validate the deep neural network regressor. The experimental results exhibit reasonable performance in distinguishing patients with portal hypertension from healthy individuals. Such performances may be improved by using larger datasets of high quality. These findings suggest that deep neural networks can serve as useful auxiliary diagnostic tools, aiding healthcare professionals in making timely and accurate decisions for patients suspected of having portal hypertension. [ABSTRACT FROM AUTHOR]

In this paper, a proposed algorithm that dynamically changes the neural network structure is presented. The structure is changed based on some features in the cascade correlation algorithm. Cascade correlation is an important algorithm that is used to solve the actual problem by artificial neural networks as a new architecture and supervised learning algorithm. This process optimizes the architectures of the network which intends to accelerate the learning process and produce better performance in generalization. Many researchers have to date proposed several growing algorithms to optimize the feedforward neural network architectures. The proposed algorithm has been tested on various medical data sets. The results prove that the proposed algorithm is a better method to evaluate the accuracy and flexibility resulting from it. [ABSTRACT FROM AUTHOR]

Manually curating biomedical knowledge from publications is necessary to build a knowledge based service that provides highly precise and organized information to users. The process of retrieving relevant publications for curation, which is also known as document triage, is usually carried out by querying and reading articles in PubMed. However, this query-based method often obtains unsatisfactory precision and recall on the retrieved results, and it is difficult to manually generate optimal queries. To address this, we propose a machine-learning assisted triage method. We collect previously curated publications from two databases UniProtKB/Swiss-Prot and the NHGRI-EBI GWAS Catalog, and used them as a gold-standard dataset for training deep learning models based on convolutional neural networks. We then use the trained models to classify and rank new publications for curation. For evaluation, we apply our method to the real-world manual curation process of UniProtKB/Swiss-Prot and the GWAS Catalog. We demonstrate that our machine-assisted triage method outperforms the current query-based triage methods, improves efficiency, and enriches curated content. Our method achieves a precision 1.81 and 2.99 times higher than that obtained by the current query-based triage methods of UniProtKB/Swiss-Prot and the GWAS Catalog, respectively, without compromising recall. In fact, our method retrieves many additional relevant publications that the query-based method of UniProtKB/Swiss-Prot could not find. As these results show, our machine learning-based method can make the triage process more efficient and is being implemented in production so that human curators can focus on more challenging tasks to improve the quality of knowledge bases. [ABSTRACT FROM AUTHOR]

An ever-increasing number of computer vision and image/video processing challenges are being approached using deep convolutional neural networks, obtaining state-of-the-art results in object recognition and detection, semantic segmentation, action recognition, optical flow, and super resolution. Hardware acceleration of these algorithms is essential to adopt these improvements in embedded and mobile computer vision systems. We present a new architecture, design, and implementation, as well as the first reported silicon measurements of such an accelerator, outperforming previous work in terms of power, area, and I/O efficiency. The manufactured device provides up to 196 GOp/s on 3.09 \text mm^2 of silicon in UMC 65-nm technology and can achieve a power efficiency of 803 GOp/s/W. The massively reduced bandwidth requirements make it the first architecture scalable to TOp/s performance. [ABSTRACT FROM AUTHOR]

The paper presents the results of research on the hybrid industrial tomograph electrical impedance tomography (EIT) and ultrasonic tomography (UST) (EIT-UST), operating on the basis of electrical and ultrasonic data. The emphasis of the research was placed on the algorithmic domain. However, it should be emphasized that all hardware components of the hybrid tomograph, including electronics, sensors and transducers, have been designed and mostly made in the Netrix S.A. laboratory. The test object was a tank filled with water with several dozen percent concentration. As part of the study, the original multiple neural networks system was trained, the characteristic feature of which is the generation of each of the individual pixels of the tomographic image, using an independent artificial neural network (ANN), with the input vector for all ANNs being the same. Despite the same measurement vector, each of the ANNs generates its own independent output value for a given tomogram pixel, because, during training, the networks get their respective weights and biases. During the tests, the results of three tomographic methods were compared: EIT, UST and EIT-UST hybrid. The results confirm that the use of heterogeneous tomographic systems (hybrids) increases the reliability of reconstruction in various measuring cases, which is used to solve quality problems in managing production processes. [ABSTRACT FROM AUTHOR]

The estimation of nested functions (i.e., functions of functions) is one of the central reasons for the success and popularity of machine learning. Today, artificial neural networks are the predominant class of algorithms in this area, known as representational learning. Here, we introduce Representational Gradient Boosting (RGB), a nonparametric algorithm that estimates functions with multi-layer architectures obtained using backpropagation in the space of functions. RGB does not need to assume a functional form in the nodes or output (e.g., linear models or rectified linear units), but rather estimates these transformations. RGB can be seen as an optimized stacking procedure where a meta algorithm learns how to combine different classes of functions (e.g., Neural Networks (NN) and Gradient Boosting (GB)), while building and optimizing them jointly in an attempt to compensate each other’s weaknesses. This highlights a stark difference with current approaches to meta-learning that combine models only after they have been built independently. We showed that providing optimized stacking is one of the main advantages of RGB over current approaches. Additionally, due to the nested nature of RGB we also showed how it improves over GB in problems that have several high-order interactions. Finally, we investigate both theoretically and in practice the problem of recovering nested functions and the value of prior knowledge. [ABSTRACT FROM AUTHOR]

Imbalanced problem occurs when the size of one class, i.e. the minority class, is much lower than that of the other classes, i.e. the majority classes. Conventional data level methods are employed as the preprocessing approaches to balance the datasets before the classifier learning. Since the balanced data remains unchanged during the learning process, one pre-deleted sample would never be used to train the classifier, which may result in information loss. To solve this problem, this work presents an One-sided Dynamic Undersampling (ODU) technique which adopts all samples in the training process, and dynamically determines whether a majority sample should be used for the classifier learning. Thus, ODU can dynamically undersample the majority class to balance the dataset. To validate the effectiveness of ODU, we integrate it into No-Propagation neural networks to propose an ODU No-Propagation Neural Networks (ODUNPNN). ODUNPNN takes all training samples into consideration, and dynamically undersamples majority class after each iteration, i.e. ODUNPNN integrates undersampling approach into the classifier learning process. Experimental results on both synthetic and real-world imbalance datasets demonstrate that ODUNPNN outperforms the NPNN-based algorithms, and results in comparative performance compared with LASVM-AL, EasyEnsemble, and DyS on real-world imbalance datasets. The contributions of this paper are: (1) ODUNPNN integrates undersampling approach into the classifier learning process. (2) ODUNPNN dynamically balances training data in each iteration. (3) ODU technique can be integrated into other classification learning machines. [ABSTRACT FROM AUTHOR]

The research on service supply chain has attracted more and more focus from both academia and industrial community. In a service supply chain, the selection of supplier portfolio is an important and difficult problem due to the fact that a supplier portfolio may include multiple suppliers from a variety of fields. To address this problem, we propose a novel supplier portfolio selection method based on a well known machine learning approach, i.e., Ranking Neural Network (RankNet). In the proposed method, we regard the problem of supplier portfolio selection as a ranking problem, which integrates a large scale of decision making features into a ranking neural network. Extensive simulation experiments are conducted, which demonstrate the feasibility and effectiveness of the proposed method. The proposed supplier portfolio selection model can be applied in a real corporation easily in the future. [ABSTRACT FROM AUTHOR]

The incidence of colon, rectal, and colorectal cancer is very high, and diagnosis is often made in the advanced stages of the disease. In cases where peritoneal carcinomatosis is limited, patients can benefit from newer treatment options if the disease is promptly identified, and they are referred to specialized centers. Therefore, an essential diagnostic benefit would be identifying those factors that could lead to early diagnosis. A retrospective study was performed using patient data gathered from 2010 to 2020. The collected data were represented by routine blood tests subjected to stringent inclusion and exclusion criteria. In order to determine the presence or absence of peritoneal carcinomatosis in colorectal cancer patients, three types of machine learning approaches were applied: a neuro-evolutive methodology based on artificial neural network (ANN), support vector machines (SVM), and random forests (RF), all combined with differential evolution (DE). The optimizer (DE in our case) determined the internal and structural parameters that defined the ANN, SVM, and RF in their optimal form. The RF strategy obtained the best accuracy in the testing phase (0.75). Using this RF model, a sensitivity analysis was applied to determine the influence of each parameter on the presence or absence of peritoneal carcinomatosis. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM), a simple single-hidden-layer feed-forward neural network with fast implementation, has been successfully applied in many fields. This paper proposes an ELM with a constructional structure (CS-ELM) for improving the performance of ELM in dealing with regression problems. In the CS-ELM, there are some partial input subnets (PISs). The first step in designing the PISs is to divide the data-attribute-space into several sub-spaces through using an improved extension clustering algorithm (IECA). The input data attributes in the same sub-space can build a PIS and the similar information of the data attributes is stored in the corresponding PIS. Additionally, a double parallel structure is applied in the CS-ELM, in which there is a special channel that directly connects the input layer neurons to the output layer neurons. In this regard, the proposed procedure can be called ELM with a data-attribute-space-oriented double parallel (DASODP) structure (DASODP–ELM). To test the validity of the proposed method, it is applied to 4 regression applications. The experimental results indicate that, compared with ELM, DASODP–ELM with less number of parameters can achieve higher regression precision in the generalization phase. [ABSTRACT FROM AUTHOR]