Search

Machine learning (ML), a form of artificial intelligence (AI), is being increasingly employed in neurology. Reported performance metrics often match or exceed the efficiency of average clinicians. The neurologist is easily baffled by the underlying concepts and terminologies associated with ML studies. The superlative performance metrics of ML algorithms often hide the opaque nature of its inner workings. Questions regarding ML model's interpretability and reproducibility of its results in real-world scenarios, need emphasis. Given an abundance of time and information, the expert clinician should be able to deliver comparable predictions to ML models, a useful benchmark while evaluating its performance. Predictive performance metrics of ML models should not be confused with causal inference between its input and output. ML and clinical gestalt should compete in a randomized controlled trial before they can complement each other for screening, triaging, providing second opinions and modifying treatment. [ABSTRACT FROM AUTHOR]

Applying parallel-in-time algorithms to multiscale Hamiltonian systems to obtain stable long-time simulations is very challenging. In this paper, we present novel data-driven methods aimed at improving the standard parareal algorithm developed by Lions et al. in 2001, for multiscale Hamiltonian systems. The first method involves constructing a correction operator to improve a given inaccurate coarse solver through solving a Procrustes problem using data collected online along parareal trajectories. The second method involves constructing an efficient, high-fidelity solver by a neural network trained with offline generated data. For the second method, we address the issues of effective data generation and proper loss function design based on the Hamiltonian function. We show proof-of-concept by applying the proposed methods to a Fermi-Pasta-Ulam (FPU) problem. The numerical results demonstrate that the Procrustes parareal method is able to produce solutions that are more stable in energy compared to the standard parareal. The neural network solver can achieve comparable or better runtime performance compared to numerical solvers of similar accuracy. When combined with the standard parareal algorithm, the improved neural network solutions are slightly more stable in energy than the improved numerical coarse solutions. [ABSTRACT FROM AUTHOR]

In this paper, we study numerical solution of ruin probability of continuous time model. We develop an effective optimal adaptive particle swarm optimization-triangular neural network (PSO-TNN), which consists of three parts: particle swarm optimization algorithm (PSO) improved trigonometric function, extreme learning machine algorithm with initial conditions (IELM) and improved reduction algorithm. The results obtained that PSO-TNN is superior to triangular neural network (TNN) and physics-informed neural networks (PINNs), and PSO is superior to Aquila Optimizer (AO), Smell Agent Optimization (SAO), African vultures optimization algorithm (AVOA), Arithmetic optimization algorithm (AOA) in the optimization of neural network. Because the relationship between the number of neural networks and the mean square error is uncertain, we propose the adaptive reduction algorithm (AR). Through the comparison of numerical solutions with the analytical solutions and traditional numerical solutions, the PSO-TNN algorithm clearly reduced the mean square error and relative error. The PSO-TNN algorithm shows a clear improvement in terms of accuracy and overall efficiency. [ABSTRACT FROM AUTHOR]

In this paper, we discuss applications of neural networks to recognizing knots and, in particular, to the unknotting problem. One of the motivations for this study is to understand how neural networks work on the example of a problem for which rigorous mathematical algorithms for its solution are known. We represent knots by rectangular Dynnikov diagrams and apply neural networks to distinguish a given diagram's class from a finite family of topological types. The data presented to the program is generated by applying Dynnikov moves to initial samples. The significance of using these diagrams and moves is that in this context the problem of determining whether a diagram is unknotted is a finite search of a bounded combinatorial space. In this way, this paper provides a foundation for further work where the neural network itself will learn to use the Dynnikov moves for knot recognition. Source code of the programs is available at https://github.com/nerusskikh/deepknots. [ABSTRACT FROM AUTHOR]

Background: Machine Learning (ML) is becoming increasingly popular in healthcare, particularly for improving the timing and accuracy of diagnosis. ML can provide disease prediction by analyzing vast amounts of healthcare data, thereby, empowering patients and healthcare providers with information to make informed decisions about disease prevention. Due to the rising cost of treatment, one of the most important topics in clinical data analysis is the prediction and prevention of cardiovascular disease. It is difficult to manually calculate the chances of developing heart disease due to a myriad of contributing factors. Objective: The aim of this paper is to develop and compare various intelligent systems built with ML algorithms for predicting whether a person is likely to develop heart disease using the publicly available Cleveland Heart Disease dataset. This paper describes an alternative multilayer perceptron (MLP) training technique that utilizes a particle swarm optimization (PSO) algorithm for heart disease detection. Methods: The proposed MLP-PSO hybrid algorithm and ten different ML algorithms are used in this study to predict heart disease. Various classification metrics are used to evaluate the performance of the algorithms. Results: The proposed MLP-PSO outperforms all other algorithms, obtaining an accuracy of 84.61%. Conclusions: According to our findings, the current MLP-PSO classifier enables practitioners to diagnose heart disease earlier, more accurately, and more effectively. [ABSTRACT FROM AUTHOR]

Modern smart grids are built based on top of advanced computing and networking technologies, where condition monitoring relies on secure cyberphysical connectivity. Over the network infrastructure, transported data containing confidential information, must be protected as smart grids are vulnerable and subject to various cyberattacks. Various machine learning based classifiers were proposed for intrusion detection in smart grids. However, each of them has respective advantage and disadvantages. Aiming to improve the performance of existing machine learning based classifiers, this paper proposes an adaptive deep learning algorithm with a data pre-processing module, a neural network pre-training module and a classifier module, which work together classify intrusion data types using their high-dimensional data features. The proposed Adaptive Deep Learning (ADL) algorithm obtains the number of layers and the number of neurons per layer by determining the characteristic dimension of the network traffic. With transfer learning, the proposed ADL algorithm can extract the original data dimensions and obtain new abstract features. By combining deep learning models with traditional machine learning-based classification models, the performance of classification of network traffic data is significantly improved. By using the Network Security Laboratory-Knowledge Discovery in Databases (NSL-KDD) dataset, experimental results show that the proposed ADL algorithm improves the effectiveness of existing intrusion detection methods and reduces the training time, indicating a promising candidate to enhance network security in smart grids. [ABSTRACT FROM AUTHOR]

Path planning is an essential component of mobile robotics. Classical path planning algorithms, such as wavefront and rapidly exploring random tree, are used heavily in autonomous robots. With the recent advances in machine learning, development of learning-based path planning algorithms has been experiencing a rapid growth. A unified path planning interface that facilitates the development and benchmarking of existing and new algorithms is needed. This paper presents PathBench, a platform for developing, visualizing, training, testing, and benchmarking of existing and future, classical and learning-based path planning algorithms in 2D and 3D grid world environments. Many existing path planning algorithms are supported, e.g. A*, Dijkstra, waypoint planning networks, value iteration networks, and gated path planning networks; integrating new algorithms is easy and clearly specified. The benchmarking ability of PathBench is explored in this paper by comparing algorithms across five different hardware systems and three different map types, including built-in PathBench maps, video game maps, and maps from real world databases. Metrics, such as path length, success rate, and computational time, were used to evaluate algorithms. Algorithmic analysis was also performed on a real-world robot to demonstrate PathBench's support for Robot Operating System. PathBench is open source1. [ABSTRACT FROM AUTHOR]

Monitoring a patient's vital signs is considered one of the most challenging problems in telehealth systems, especially when patients reside in remote locations. Companies now use IoT devices such as wearable devices to participate in telehealth systems. However, the steady adoption of wearables can result in a significant increase in the volume of data being collected and transmitted. As these devices run on limited battery power, they can run out of power quickly due to the high processing requirements of the device for data collection and transmission. Given the importance of medical data, it is imperative that all transmitted data adhere to strict integrity and availability requirements. Reducing the volume of healthcare data and the frequency of transmission can improve a device's battery life via an inference algorithm. Furthermore, this approach creates issues for improving transmission metrics related to accuracy and efficiency, which are traded-off against each other, with increasing accuracy reducing efficiency. This paper demonstrates that machine learning (ML) can be used to overcome the trade-off problem. The damped least-squares algorithm (DLSA) is used to enhance both metrics by taking fewer samples for transmission whilst maintaining accuracy. The algorithm is tested with a standard heart rate dataset to compare the metrics. The results showed that the DLSA provides the best performance, with an efficiency of 3.33 times for reduced sample data size and an accuracy of 95.6%, with similar accuracies observed in seven different sampling cases adopted for testing that demonstrate improved efficiency. This proposed method significantly improve both metrics using ML without sacrificing one metric over the other compared to existing methods with high efficiency. [ABSTRACT FROM AUTHOR]

In this paper, we suggest basing the development of classification methods on traditional techniques but approximating them in whole or in part with artificial neural networks (ANNs). Compared to a direct ANN approach, the underlying traditional method is often easier to analyse. Classification failures can be better understood and corrected for, while at the same time faster execution can be obtained due to the parallel ANN structure. Furthermore, such a two‐step design philosophy partly eliminates the 'guesswork' associated with the design of ANNs. The expected gain is thus that the benefits from the traditional field, and the ANN field can be obtained by combining the best features from both fields. We illustrate our approach by working through an explicit example, namely, a Nearest Neighbour classifier applied to a subset of the MNIST database of handwritten digits. Two different approaches are discussed for how to translate the traditional method into an ANN. The first approach is based on a constructive implementation which directly reflects the original algorithm. The second approach uses ANNs to approximate the whole, or part of, the original method. An important part of the approach is to show how improvements can be introduced. In line with the presented philosophy, this is done by extending the traditional method in several ways followed by ANN approximation of the modified algorithms. The extensions are based on a windowed version of the nearest neighbour algorithm. We show that the improvements carry over to the ANN implementations. We further investigate the stability of the solutions by modifying the training set. It is shown that the errors do not change significantly. This also holds true for the ANN approximations providing confidence that the two‐step strategy is robust. [ABSTRACT FROM AUTHOR]

Classification is one of the foremost machine learning tasks in this modern era. Neural Network (NN) is one of the powerful classification techniques. NN can achieve high classification accuracy on highly imbalanced and complex datasets, but lacks in explanation of its reasoning process which limits its applicability in various domains which require transparent decision along with good accuracy. There are some techniques which extract rules from NN and make it transparent; however, attribute pruning, rule pruning and class overlap algorithms are not sufficiently effective. Therefore, this paper proposes a rule extraction algorithm, called Transparent Rule Extraction using Neural Network (TRENN) to convert NN into white box with greater emphasis on attribute pruning and rule pruning. The proposed TRENN is a pedagogical approach and an extension of one of the existing algorithms named Rule Extraction from Neural Network using Classified and Misclassified data (RxNCM). The proposed TRENN extends the RxNCM with sequential floating backward search for feature and rule selection to improve the comprehensibility of the generated rules. Besides, the proposed TRENN uses probabilistic approach for the treatment of class overlapping problem in the rule updating phase instead of reclassification used in RxNCM where the overlap may persist. Experiments are conducted with eight real datasets collected from the UCI repository. Performance of the TRENN is measured with Precision, Recall, FP-Rate, F-measure, and local and global comprehensibility. It is observed from the experimental results that TRENN performs better than Re-RX, RxNCM and RxREN. [ABSTRACT FROM AUTHOR]

Many real‐world data mining applications involve obtaining predictive models using imbalanced datasets. Frequently, the least common target variables present within datasets are associated with events that are highly relevant for end users. When these variables are nominal, we have a class‐imbalance problem which has been thoroughly studied within machine learning. As for regression tasks where target variables are continuous, few predictive models and evaluation techniques exist. This paper proposes a solution to these challenges. First, we introduce a cost‐sensitive learning algorithm based on a neural network trained on the minimization of a biased loss function. Results show a higher or comparable performance and convergence speed to existent techniques. Second, we develop new approaches for performance assessment of regression tasks within imbalanced domains by proposing new scalar measures, namely Geometric Mean Error (GME) and Class‐Weighted Error (CWE), as well as new graphical‐based measures, namely RECTPR, RECTNR, RECG − Mean and RECCWA curves. Unlike standard measures, our evaluation strategies are shown to be more robust to data imbalance as they reflect the performance of both rare and frequent events. [ABSTRACT FROM AUTHOR]

An extreme learning machine (ELM) is a novel training method for single-hidden layer feedforward neural networks (SLFNs) in which the hidden nodes are randomly assigned and fixed without iterative tuning. ELMs have earned widespread global interest due to their fast learning speed, satisfactory generalization ability and ease of implementation. In this paper, we extend this theory to hypercomplex space and attempt to simultaneously consider multisource information using a hypercomplex representation. To illustrate the performance of the proposed hypercomplex extreme learning machine (HELM), we have applied this scheme to the task of multispectral palmprint recognition. Images from different spectral bands are utilized to construct the hypercomplex space. Extensive experiments conducted on the PolyU and CASIA multispectral databases demonstrate that the HELM scheme can achieve competitive results. The source code together with datasets involved in this paper can be available for free download at . [ABSTRACT FROM AUTHOR]

In a recidivism prediction context, there is no consensus on which modeling strategy should be followed for obtaining an optimal prediction model. In previous papers, a range of statistical and machine learning techniques were benchmarked on recidivism data with a binary outcome. However, two important tree ensemble methods, namely gradient boosting and random forests were not extensively evaluated. In this paper, we further explore the modeling potential of these techniques in the binary outcome criminal prediction context. Additionally, we explore the predictive potential of classical statistical and machine learning methods for censored time-to-event data. A range of statistical manually specified statistical and (semi-)automatic machine learning models is fitted on Dutch recidivism data, both for the binary outcome case and censored outcome case. To enhance generalizability of results, the same models are applied to two historical American data sets, the North Carolina prison data. For all datasets, (semi-) automatic modeling in the binary case seems to provide no improvement over an appropriately manually specified traditional statistical model. There is however evidence of slightly improved performance of gradient boosting in survival data. Results on the reconviction data from two sources suggest that both statistical and machine learning should be tried out for obtaining an optimal model. Even if a flexible black-box model does not improve upon the predictions of a manually specified model, it can serve as a test whether important interactions are missing or other misspecification of the model are present and can thus provide more security in the modeling process. [ABSTRACT FROM AUTHOR]

Analyzing and predicting the trend of price fluctuation has been receiving more and more attention, as price risk has become the focus of risk control research in heating oil futures market. A novel time series prediction model combined with the complex network method is put forward in the paper. First of all, this paper counts the cumulative time interval of different nodes in the network, and fits its growth trend with the Fourier model. Then a novel price fluctuation prediction model is established based on the effective information such as some topology properties extracted from the network. The results show that the Fourier model can predict the emergence time of new nodes in the next stage, and the established price fluctuation prediction model can infer the names of nodes in the prediction interval, so as to determine the forward-looking behavior of price evolution. Besides, liken to the NAR neural network, the prediction results obtained by the proposed method also show superiority, which has important theoretical value and academic significance for early warning and prediction of price behavior in the heating oil futures market. [ABSTRACT FROM AUTHOR]

The 3D measurement system based on line-structured light uses a camera to capture laser stripes due to changing in the shape of an object, and uses the acquired pixel coordinates for 3D reconstruction. System calibration is an important step in 3D measurement. The current camera calibration algorithm research mainly focuses on improving the algorithm itself, and there is less research on the influence of external factors. This paper proposes a coplanar hybrid calibration algorithm based on the error screening model by combining the error screening model, mathematical model and neural network model. It is mainly divided into two steps. The first step is to use the radial array constraint calibration algorithm based on the error screening model to solve the camera's internal and external parameters. The second step uses the camera internal and external parameters obtained in the first step to convert the pixel coordinates into real three-dimensional coordinates, and compares the calculated three-dimensional coordinates with the actual coordinates. Using machine learning to establish a compensation network, get a compensation function, and use the resulting 3D world coordinates to perform point cloud stitching. Experiments show that compared with the traditional calibration algorithm, the calibration algorithm has a small error and reduces the calibration error by about 6.5%. [ABSTRACT FROM AUTHOR]

Trailer mass is one of the important trailer parameters that affects the stability of the tractor-trailer systems. In this paper, two different approaches are proposed to estimate trailer mass for arbitrary tractor-trailer configurations; dynamic system model-based and Machine Learning (ML) approaches. The stability of the dynamic system model-based estimation algorithm is analyzed, establishing the convergence of the estimation error to zero. In the proposed ML-based approach, a Deep Neural Network (DNN) is designed to estimate trailer mass. The inputs of the ML-based method have been selected based on the tractor-trailer dynamic model, and are considered to be normalized by the tractor mass, tire sizes, and geometry so that re-training of the network is not needed for different towing vehicles. The simulation and experimental results justify the accuracy of the trailer mass estimation in various cases and demonstrate that the trailer mass can be estimated with less than 10% error. [ABSTRACT FROM AUTHOR]

This paper presents an integrated hybrid optimization algorithm for training the radial basis function neural network (RBF NN). Training of neural networks is still a challenging exercise in machine learning domain. Traditional training algorithms in general suffer and trap in local optima and lead to premature convergence, which makes them ineffective when applied for datasets with diverse features. Training algorithms based on evolutionary computations are becoming popular due to their robust nature in overcoming the drawbacks of the traditional algorithms. Accordingly, this paper proposes a hybrid training procedure with differential search (DS) algorithm functionally integrated with the particle swarm optimization (PSO). To surmount the local trapping of the search procedure, a new population initialization scheme is proposed using Logistic chaotic sequence, which enhances the population diversity and aid the search capability. To demonstrate the effectiveness of the proposed RBF hybrid training algorithm, experimental analysis on publicly available 7 benchmark datasets are performed. Subsequently, experiments were conducted on a practical application case for wind speed prediction to expound the superiority of the proposed RBF training algorithm in terms of prediction accuracy. [ABSTRACT FROM AUTHOR]

Machine Learning (ML) methods have been proposed in the academic literature as alternatives to statistical ones for time series forecasting. Yet, scant evidence is available about their relative performance in terms of accuracy and computational requirements. The purpose of this paper is to evaluate such performance across multiple forecasting horizons using a large subset of 1045 monthly time series used in the M3 Competition. After comparing the post-sample accuracy of popular ML methods with that of eight traditional statistical ones, we found that the former are dominated across both accuracy measures used and for all forecasting horizons examined. Moreover, we observed that their computational requirements are considerably greater than those of statistical methods. The paper discusses the results, explains why the accuracy of ML models is below that of statistical ones and proposes some possible ways forward. The empirical results found in our research stress the need for objective and unbiased ways to test the performance of forecasting methods that can be achieved through sizable and open competitions allowing meaningful comparisons and definite conclusions. [ABSTRACT FROM AUTHOR]

In this paper, we propose a hybrid method to forecast the stock prices called High-order-fuzzy-fluctuation-Trends-based Back Propagation(HTBP)Neural Network model. First, we compare each value of the historical training data with the previous day's value to obtain a fluctuation trend time series (FTTS). On this basis, the FTTS blur into fuzzy time series (FFTS) based on the fluctuation of the increasing, equality, decreasing amplitude and direction. Since the relationship between FFTS and future wave trends is nonlinear, the HTBP neural network algorithm is used to find the mapping rules in the form of self-learning. Finally, the results of the algorithm output are used to predict future fluctuations. The proposed model provides some innovative features:(1)It combines fuzzy set theory and neural network algorithm to avoid overfitting problems existed in traditional models. (2)BP neural network algorithm can intelligently explore the internal rules of the actual existence of sequential data, without the need to analyze the influence factors of specific rules and the path of action. (3)The hybrid modal can reasonably remove noises from the internal rules by proper fuzzy treatment. This paper takes the TAIEX data set of Taiwan stock exchange as an example, and compares and analyzes the prediction performance of the model. The experimental results show that this method can predict the stock market in a very simple way. At the same time, we use this method to predict the Shanghai stock exchange composite index, and further verify the effectiveness and universality of the method. [ABSTRACT FROM AUTHOR]

In this paper, an optimal weight semi-supervised learning machine for a single-hidden layer feedforward network (SLFN) with time delay is developed. Both input weights and output weights of the SLFN are globally optimized with manifold regularization. By feature mapping, input vectors can be placed at the prescribed positions in the feature space in the sense that the separability of all nonlinearly separable patterns can be maximized, unlabeled data can be leveraged to improve the classification accuracy when labeled data are scarce, and a high degree of recognition accuracy can be achieved with a small number of hidden nodes in the SLFN. Some simulation examples are presented to show the excellent performance of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In this paper, a deep reinforcement learning-based robust control strategy for quadrotor helicopters is proposed. The quadrotor is controlled by a learned neural network which directly maps the system states to control commands in an end-to-end style. The learning algorithm is developed based on the deterministic policy gradient algorithm. By introducing an integral compensator to the actor-critic structure, the tracking accuracy and robustness have been greatly enhanced. Moreover, a two-phase learning protocol which includes both offline and online learning phase is proposed for practical implementation. An offline policy is first learned based on a simplified quadrotor model. Then, the policy is online optimized in actual flight. The proposed approach is evaluated in the flight simulator. The results demonstrate that the offline learned policy is highly robust to model errors and external disturbances. It also shows that the online learning could significantly improve the control performance. [ABSTRACT FROM AUTHOR]

The growing volume and complexity of data has led to the development of the so-called linear algorithms for neural networks like ELM, which maintain the precision of classic algorithms but with higher training speed. This speed increase is due to a simpler architecture, the random fixing of the input weights without being trained and the analytical calculation of the output weights instead of the slowly classical iterative gradient methods as Backpropagation. However, the random fixing of the input weights increases the sensibility to input perturbations like noise. Recently, No-Propagation (No-Prop) algorithm has been introduced as another linear algorithm, which shares with ELM the architecture and the random input weights (hidden weights) initialization. In this paper, an exhaustive comparison of both algorithms and its regularized versions are presented. The simulations results suggest that No-Prop is a competitive alternative to the ELM algorithm. [ABSTRACT FROM AUTHOR]

In the upcoming Internet-of-Things (IoT) era, the communication is often featured by massive connection, sporadic transmission, and small-sized data packets, which poses new requirements on the delay expectation and resource allocation efficiency of the Random Access (RA) mechanisms of the IoT communication stack. A grant-free non-orthogonal random access (NORA) system is considered in this paper, which could simultaneously reduce the access delay and support more Machine Type Communication (MTC) devices with limited resources. In order to address the joint user activity detection (UAD) and channel estimation (CE) problem in the grant-free NORA system, we propose a deep neural network-aided message passing-based block sparse Bayesian learning (DNN-MP-BSBL) algorithm. In the DNN-MP-BSBL algorithm, the iterative message passing process is transferred from a factor graph to a deep neural network (DNN). Weights are imposed on the messages in the DNN and trained to minimize the estimation error. It is shown that the trained weights could alleviate the convergence problem of the MP-BSBL algorithm, especially on crowded RA scenarios. Simulation results show that the proposed DNN-MP-BSBL algorithm could improve the UAD and CE accuracy with a smaller number of iterations, indicating its advantages for low-latency grant-free NORA systems. [ABSTRACT FROM AUTHOR]

In recent years, with the development of computer science, deep learning is held as competent enough to solve the problem of inference and learning in high dimensional space. Therefore, it has received unprecedented attention from both the academia and the business community. Compared with CPU/GPU, FPGA has attracted much attention for its high-energy efficiency, short development cycle and reconfigurability in the aspect of deep learning algorithm. However, because of the limited research on OpenCL optimization on FPGA of deep learning algorithms, OpenCL tools and models applied to CPU/GPU cannot be directly used on FPGA. This makes it difficult for software programmers to use FPGA when implementing deep learning algorithms for a rewarding performance. To solve this problem, this paper proposed an OpenCL computational model based on FPGA template architecture to optimize the time-consuming convolution layer in deep learning. The comparison between the program applying the computational model and the corresponding optimization program provided by Xilinx indicates that the former is 8-40 times higher than the latter in terms of performance. [ABSTRACT FROM AUTHOR]

Due to the fast speed of data generation and collection from advanced equipment, the amount of data obviously overflows the limit of available memory space and causes difficulties achieving high learning accuracy. Several methods based on discard-after-learn concept have been proposed. Some methods were designed to cope with a single incoming datum but some were designed for a chunk of incoming data. Although the results of these approaches are rather impressive, most of them are based on temporally adding more neurons to learn new incoming data without any neuron merging process which can obviously increase the computational time and space complexities. Only online versatile elliptic basis function (VEBF) introduced neuron merging to reduce the space-time complexity of learning only a single incoming datum. This paper proposed a method for further enhancing the capability of discard-after-learn concept for streaming data-chunk environment in terms of low computational time and neural space complexities. A set of recursive functions for computing the relevant parameters of a new neuron, based on statistical confidence interval, was introduced. The newly proposed method, named streaming chunk incremental learning (SCIL), increases the plasticity and the adaptabilty of the network structure according to the distribution of incoming data and their classes. When being compared to the others in incremental-like manner, based on 11 benchmarked data sets of 150 to 581,012 samples with attributes ranging from 4 to 1,558 formed as streaming data, the proposed SCIL gave better accuracy and time in most data sets. [ABSTRACT FROM AUTHOR]

Big web data from sources including online news and Twitter are good resources for investigating deep learning. However, collected news articles and tweets almost certainly contain data unnecessary for learning, and this disturbs accurate learning. This paper explores the performance of word2vec Convolutional Neural Networks (CNNs) to classify news articles and tweets into related and unrelated ones. Using two word embedding algorithms of word2vec, Continuous Bag-of-Word (CBOW) and Skip-gram, we constructed CNN with the CBOW model and CNN with the Skip-gram model. We measured the classification accuracy of CNN with CBOW, CNN with Skip-gram, and CNN without word2vec models for real news articles and tweets. The experimental results indicated that word2vec significantly improved the accuracy of the classification model. The accuracy of the CBOW model was higher and more stable when compared to that of the Skip-gram model. The CBOW model exhibited better performance on news articles, and the Skip-gram model exhibited better performance on tweets. Specifically, CNN with word2vec models was more effective on news articles when compared to that on tweets because news articles are typically more uniform when compared to tweets. [ABSTRACT FROM AUTHOR]

While there have been many studies using machine learning (ML) algorithms to predict process outcomes and device performance in semiconductor manufacturing, the extensively developed technology computer-aided design (TCAD) physical models should play a more significant role in conjunction with ML. While TCAD models have been effective in predicting the trends of experiments, a machine learning statistical model is more capable of predicting the anomalous effects that can be dependent on the chambers, machines, fabrication environment, and specific layouts. In this paper, we use an analytics-statistics mixed training (ASMT) approach using TCAD. Under this method, the TCAD models are incorporated into the machine learning training procedure. The mixed dataset with the experimental and TCAD results improved the prediction in terms of accuracy. With the application of ASMT to the BOSCH process, we show that the mean square error (MSE) can be effectively decreased when the analytics-statistics mixed training (ASMT) scheme is used instead of the classic neural network (NN) used in the baseline study. In this method, statistical induction and analytical deduction can be combined to increase the prediction accuracy of future intelligent semiconductor manufacturing. [ABSTRACT FROM AUTHOR]

Leveraging smart metering solutions to support energy efficiency on the individual household level poses novel research challenges in monitoring usage and providing accurate load forecasting. Forecasting electricity usage is an especially important component that can provide intelligence to smart meters. In this paper, we propose an enhanced approach for load forecasting at the household level. The impacts of residents’ daily activities and appliance usages on the power consumption of the entire household are incorporated to improve the accuracy of the forecasting model. The contributions of this paper are threefold: (1) we addressed short-term electricity load forecasting for 24 hours ahead, not on the aggregate but on the individual household level, which fits into the Residential Power Load Forecasting (RPLF) methods; (2) for the forecasting, we utilized a household specific dataset of behaviors that influence power consumption, which was derived using segmentation and sequence mining algorithms; and (3) an extensive load forecasting study using different forecasting algorithms enhanced by the household activity patterns was undertaken. [ABSTRACT FROM AUTHOR]

For ensemble learning, how to select and combine the candidate classifiers are two key issues which influence the performance of the ensemble system dramatically. Random vector functional link networks (RVFL) without direct input-to-output links is one of suitable base-classifiers for ensemble systems because of its fast learning speed, simple structure and good generalization performance. In this paper, to obtain a more compact ensemble system with improved convergence performance, an improved ensemble of RVFL based on attractive and repulsive particle swarm optimization (ARPSO) with double optimization strategy is proposed. In the proposed method, ARPSO is applied to select and combine the candidate RVFL. As for using ARPSO to select the optimal base RVFL, ARPSO considers both the convergence accuracy on the validation data and the diversity of the candidate ensemble system to build the RVFL ensembles. In the process of combining RVFL, the ensemble weights corresponding to the base RVFL are initialized by the minimum norm least-square method and then further optimized by ARPSO. Finally, a few redundant RVFL is pruned, and thus the more compact ensemble of RVFL is obtained. Moreover, in this paper, theoretical analysis and justification on how to prune the base classifiers on classification problem is presented, and a simple and practically feasible strategy for pruning redundant base classifiers on both classification and regression problems is proposed. Since the double optimization is performed on the basis of the single optimization, the ensemble of RVFL built by the proposed method outperforms that built by some single optimization methods. Experiment results on function approximation and classification problems verify that the proposed method could improve its convergence accuracy as well as reduce the complexity of the ensemble system. [ABSTRACT FROM AUTHOR]

DIBR-3D technology has evolved over the past few years with the demands of consumers increasing in recent times for future free-view 3D videos on their home televisions. The main issue in 3D technology is the lack of 3D content available to watch using the traditional TV systems. Although, some sophisticated devices like stereoscopic cameras have been used to fill the gap between the 3D content demand and 3D content supply. But the content generated through these sophisticated devices can not be displayed on the traditional TV systems, so there needs to be some mechanism which is inline with the traditional TV. Furthermore, the huge collection of existing 2D content should be converted to 3D using depth image-based rendering techniques. This conversion technique can highly contribute in overcoming the shortage problem of the 3D content. This paper presents a novel approach for converting 2D degraded image for DIBR 3D-TV view. This degraded or noisy/blur image is enhanced through image dehazing and Directional Filter Bank (DFB). This enhancement is necessary because of the occlusion effect or hole filling problem that occurs due to imperfect depth map. The enhanced image is then segmented into the foreground image and the background image. After the segmentation, the depth map is generated using image profiles. Moreover, Stereoscopic images are finally produced using the DIBR procedure which is based on the 2D input image and the corresponding depth map. We have verified the results of the proposed approach by comparing the results with the existing state-of-the-art techniques. [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG) is a common base signal used to monitor brain activities and diagnose sleep disorders. Manual sleep stage scoring is a time-consuming task for sleep experts and is limited by inter-rater reliability. In this paper, we propose an automatic sleep stage annotation method called SleepEEGNet using a single-channel EEG signal. The SleepEEGNet is composed of deep convolutional neural networks (CNNs) to extract time-invariant features, frequency information, and a sequence to sequence model to capture the complex and long short-term context dependencies between sleep epochs and scores. In addition, to reduce the effect of the class imbalance problem presented in the available sleep datasets, we applied novel loss functions to have an equal misclassified error for each sleep stage while training the network. We evaluated the performance of the proposed method on different single-EEG channels (i.e., Fpz-Cz and Pz-Oz EEG channels) from the Physionet Sleep-EDF datasets published in 2013 and 2018. The evaluation results demonstrate that the proposed method achieved the best annotation performance compared to current literature, with an overall accuracy of 84.26%, a macro F1-score of 79.66% and κ = 0.79. Our developed model can be applied to other sleep EEG signals and aid the sleep specialists to arrive at an accurate diagnosis. The source code is available at . [ABSTRACT FROM AUTHOR]

Antenna selection in Multiple-Input Multiple-Output (MIMO) systems has attracted increasing attention due to the challenge of keeping a balance between communication performance and computational complexity. Recently, deep learning based methods have achieved promising performance in many application fields. This paper proposed a deep learning (DL) based antenna selection technique. First, we generated the label of training antenna systems by maximizing the channel capacity. Then, we adopted the deep convolutional neural network (CNN) on the channel matrices to explicitly exploit the massive latent cues of attenuation coefficients. Finally, we used the adopted CNN to assign the class label and then select the optimal antenna subset. Experimental results demonstrate that our method can achieve better performance than the state-of-the-art baselines for data-driven based antenna selection. [ABSTRACT FROM AUTHOR]

In general, the stock prices of the same industry have a similar trend, but those of different industries do not. When investing in stocks of different industries, one should select the optimal model from lots of trading models for each industry because any model may not be suitable for capturing the stock trends of all industries. However, the study has not been carried out at present. In this paper, firstly we select 424 S&P 500 index component stocks (SPICS) and 185 CSI 300 index component stocks (CSICS) as the research objects from 2010 to 2017, divide them into 9 industries such as finance and energy respectively. Secondly, we apply 12 widely used machine learning algorithms to generate stock trading signals in different industries and execute the back-testing based on the trading signals. Thirdly, we use a non-parametric statistical test to evaluate whether there are significant differences among the trading performance evaluation indicators (PEI) of different models in the same industry. Finally, we propose a series of rules to select the optimal models for stock investment of every industry. The analytical results on SPICS and CSICS show that we can find the optimal trading models for each industry based on the statistical tests and the rules. Most importantly, the PEI of the best algorithms can be significantly better than that of the benchmark index and “Buy and Hold” strategy. Therefore, the algorithms can be used for making profits from industry stock trading. [ABSTRACT FROM AUTHOR]

Annotation of foot-contact and foot-off events is the initial step in post-processing for most quantitative gait analysis workflows. If clean force plate strikes are present, the events can be automatically detected. Otherwise, annotation of gait events is performed manually, since reliable automatic tools are not available. Automatic annotation methods have been proposed for normal gait, but are usually based on heuristics of the coordinates and velocities of motion capture markers placed on the feet. These heuristics do not generalize to pathological gait due to greater variability in kinematics and anatomy of patients, as well as the presence of assistive devices. In this paper, we use a data-driven approach to predict foot-contact and foot-off events from kinematic and marker time series in children with normal and pathological gait. Through analysis of 9092 gait cycle measurements we build a predictive model using Long Short-Term Memory (LSTM) artificial neural networks. The best-performing model identifies foot-contact and foot-off events with an average error of 10 and 13 milliseconds respectively, outperforming popular heuristic-based approaches. We conclude that the accuracy of our approach is sufficient for most clinical and research applications in the pediatric population. Moreover, the LSTM architecture enables real-time predictions, enabling applications for real-time control of active assistive devices, orthoses, or prostheses. We provide the model, usage examples, and the training code in an open-source package. [ABSTRACT FROM AUTHOR]

Big data classification is very slow when using traditional machine learning classifiers, particularly when using a lazy and slow-by-nature classifier such as the k-nearest neighbors algorithm (KNN). This paper proposes a new approach which is based on sorting the feature vectors of training data in a binary search tree to accelerate big data classification using the KNN approach. This is done using two methods, both of which utilize two local points to sort the examples based on their similarity to these local points. The first method chooses the local points based on their similarity to the global extreme points, while the second method chooses the local points randomly. The results of various experiments conducted on different big datasets show reasonable accuracy rates compared to state-of-the-art methods and the KNN classifier itself. More importantly, they show the high classification speed of both methods. This strong trait can be used to further improve the accuracy of the proposed methods. [ABSTRACT FROM AUTHOR]

In this paper, a new method is proposed for prediction of ship roll motion based on extreme learning machine (ELM). To improve the prediction accuracy and avoid over or under fitting, two techniques are adopted to select the appropriate structure of ELM. First, the inputs of the ELM are selected from the roll motion time series using Lipschitz quotient method. Second, the number of hidden layer nodes is determined via ℓ1 regularized technique. Finally, the ℓ1 regularized ELM is solved by least angle regression (LAR) algorithm. The effectiveness of the proposed method is demonstrated by ship roll motion prediction experiments based on the real measured ship roll motion time series. [ABSTRACT FROM AUTHOR]

In palynology, the visual classification of pollen grains from different species is a hard task which is usually tackled by human operators using microscopes. Its complete automatization would save a high quantity of resources and provide valuable improvements especially for allergy-related information systems, but also for other application fields as paleoclimate reconstruction, quality control of honey based products, collection of evidences in criminal investigations or fabric dating and tracking. This paper presents three state-of-the-art deep learning classification methods applied to the recently published POLEN23E image dataset. The three methods make use of convolutional neural networks: the first one is strictly based on the idea of transfer learning, the second one is based on feature extraction and the third one represents a hybrid approach, combining transfer learning and feature extraction. The results from the three methods are indeed very good, reaching over 97% correct classification rates in images not previously seen by the models, where other authors reported around 70. [ABSTRACT FROM AUTHOR]

Deep learning techniques, e.g., Convolutional Neural Networks (CNNs), have been explosively applied to the research in the fields of information retrieval and natural language processing. However, few research efforts have addressed semantic indexing with deep learning. The use of semantic indexing in the biomedical literature has been limited for several reasons. For instance, MEDLINE citations contain a large number of semantic labels from automatically annotated MeSH terms, and for a great deal of the literature, only the information of the title and the abstract is readily available. In this paper, we propose a Boltzmann Convolutional neural network framework (B-CNN) for biomedicine semantic indexing. In our hybrid learning framework, the CNN can adaptively deal with features of documents that have sequence relationships, and can capture context information accordingly; the Deep Boltzmann Machine (DBM) merges global (the entity in each document) and local information through its training with undirected connections. Additionally, we have designed a hierarchical coarse to fine style indexing structure for learning and classifying documents, and a novel feature extension approach with word sequence embedding and Wikipedia categorization. Comparative experiments were conducted for semantic indexing of biomedical abstract documents; these experiments verified the encouraged performance of our B-CNN model. [ABSTRACT FROM AUTHOR]

This paper presents a new method to reduce the computational cost when using Neural Networks as Language Models, during recognition, in some particular scenarios. It is based on a Neural Network that considers input contexts of different length in order to ease the use of a fallback mechanism together with the precomputation of softmax normalization constants for these inputs. The proposed approach is empirically validated, showing their capability to emulate lower order N-grams with a single Neural Network. A machine translation task shows that the proposed model constitutes a good solution to the normalization cost of the output softmax layer of Neural Networks, for some practical cases, without a significant impact in performance while improving the system speed. [ABSTRACT FROM AUTHOR]

A residual networks family with hundreds or even thousands of layers dominates major image recognition tasks, but building a network by simply stacking residual blocks inevitably limits its optimization ability. This paper proposes a novel residual network architecture, residual networks of residual networks (RoR), to dig the optimization ability of residual networks. RoR substitutes optimizing residual mapping of residual mapping for optimizing original residual mapping. In particular, RoR adds levelwise shortcut connections upon original residual networks to promote the learning capability of residual networks. More importantly, RoR can be applied to various kinds of residual networks (ResNets, Pre-ResNets, and WRN) and significantly boost their performance. Our experiments demonstrate the effectiveness and versatility of RoR, where it achieves the best performance in all residual-network-like structures. Our RoR-3-WRN58-4 + SD models achieve new state-of-the-art results on CIFAR-10, CIFAR-100, and SVHN, with the test errors of 3.77%, 19.73%, and 1.59%, respectively. RoR-3 models also achieve state-of-the-art results compared with ResNets on the ImageNet data set. [ABSTRACT FROM AUTHOR]

Combining a deep neural network with fuzzy theory, this paper proposes an advertising click-through rate (CTR) prediction approach based on a fuzzy deep neural network (FDNN). In this approach, fuzzy Gaussian-Bernoulli restricted Boltzmann machine (FGBRBM) is first applied to input raw data from advertising datasets. Next, fuzzy restricted Boltzmann machine (FRBM) is used to construct the fuzzy deep belief network (FDBN) with the unsupervised method layer by layer. Finally, fuzzy logistic regression (FLR) is utilized for modeling the CTR. The experimental results show that the proposed FDNN model outperforms several baseline models in terms of both data representation capability and robustness in advertising click log datasets with noise. [ABSTRACT FROM AUTHOR]

In this paper, we seek to improve deep neural networks by generalizing the pooling operations that play a central role in the current architectures. We pursue a careful exploration of approaches to allow pooling to learn and to adapt to complex and variable patterns. The two primary directions lie in: (1) learning a pooling function via (two strategies of) combining of max and average pooling, and (2) learning a pooling function in the form of a tree-structured fusion of pooling filters that are themselves learned. In our experiments every generalized pooling operation we explore improves performance when used in place of average or max pooling. We experimentally demonstrate that the proposed pooling operations provide a boost in invariance properties relative to conventional pooling and set the state of the art on several widely adopted benchmark datasets. These benefits come with only a light increase in computational overhead during training (ranging from additional 5 to 15 percent in time complexity) and a very modest increase in the number of model parameters (e.g., additional 1, 9, and 27 parameters for mixed, gated, and 2-level tree pooling operators, respectively). To gain more insights about our proposed pooling methods, we also visualize the learned pooling masks and the embeddings of the internal feature responses for different pooling operations. Our proposed pooling operations are easy to implement and can be applied within various deep neural network architectures. [ABSTRACT FROM PUBLISHER]

Advanced Metering Infrastructure (AMI) realizes a two-way communication of electricity data through by interconnecting with a computer network as the core component of the smart grid. Meanwhile, it brings many new security threats and the traditional intrusion detection method can’t satisfy the security requirements of AMI. In this paper, an intrusion detection system based on Online Sequence Extreme Learning Machine (OS-ELM) is established, which is used to detecting the attack in AMI and carrying out the comparative analysis with other algorithms. Simulation results show that, compared with other intrusion detection methods, intrusion detection method based on OS-ELM is more superior in detection speed and accuracy. [ABSTRACT FROM AUTHOR]

An Acute Hypotensive Episode (AHE) is the sudden onset of a sustained period of low blood pressure and is one among the most critical conditions in Intensive Care Units (ICU). Without timely medical care, it can lead to an irreversible organ damage and death. By identifying patients at risk for AHE early, adequate medical intervention can save lives and improve patient outcomes. In this paper, we design a novel dual–boundary classification based approach for identifying patients at risk for AHE. Our algorithm uses only simple summary statistics of past Blood Pressure measurements and can be used in an online environment facilitating real–time updates and prediction. We perform extensive experiments with more than 4,500 patient records and demonstrate that our method outperforms the previous best approaches of AHE prediction. Our method can identify AHE patients two hours in advance of the onset, giving sufficient time for appropriate clinical intervention with nearly 80% sensitivity and at 95% specificity, thus having very few false positives. [ABSTRACT FROM AUTHOR]