Search

This paper serves as a survey of recent advances in large margin training and its theoretical foundations, mostly for (nonlinear) deep neural networks (DNNs) that are probably the most prominent machine learning models for large-scale data in the community over the past decade. We generalize the formulation of classification margins from classical research to latest DNNs, summarize theoretical connections between the margin, network generalization, and robustness, and introduce recent efforts in enlarging the margins for DNNs comprehensively. Since the viewpoint of different methods is discrepant, we categorize them into groups for ease of comparison and discussion in the paper. Hopefully, our discussions and overview inspire new research work in the community that aim to improve the performance of DNNs, and we also point to directions where the large margin principle can be verified to provide theoretical evidence why certain regularizations for DNNs function well in practice. We managed to shorten the paper such that the crucial spirit of large margin learning and related methods are better emphasized. [ABSTRACT FROM AUTHOR]

With the rapid development of Internet-of-Things and big data, health assessment of equipment is receiving more attention in recent years. It is critical to bridge the gap between real-time production data and health status evaluation, which helps maintenance team understand the health status of equipment exactly, and then make rational maintenance plans. For this purpose, this paper proposes a framework to realize real-time equipment health assessment with health status quantitatively characterized by health degree (HD). The proposed framework begins with removing redundant features using a principal component analysis (PCA) method. Then, to represent the optimal operation status, a support vector data description (SVDD) algorithm is employed for extracting normal observations in the offline part. Thereafter, HD is introduced based on the Euclidean distance between current observation and the normal sample set. In order to achieve online updating of the normal sample set, and promote accuracy and computational efficiency of the offline part, an improved incremental SVDD algorithm based on adaptive threshold N (NISVDD) is proposed. A case study is used to demonstrate the effectiveness of the proposed framework and model using a benchmark dataset of rolling bearing. Results suggest that the proposed framework is effective, and PCA shows good potential to extract features and keep most of the original information. The proposed NISVDD model is able to trace the dynamics of equipment health status for whole run-to-failure process, and outperforms other models in both accuracy and computational efficiency. [ABSTRACT FROM AUTHOR]

We present an approach to extend the functionality and the use of kernel machines in image processing applications. We introduce a novel way to design spatial kernel machines with spatial properties and demonstrate how those newly introduced spatial properties enhance the possibilities of the use of kernel machines in image processing applications as a proof of concept. In this paper, we demonstrate four particular extensions: 1) how to model shapes efficiently with spatially computed kernel parameters in a geometrically scalable way; 2) how to visualize the kernel parameters precisely and intuitively on binary 2D shapes; 3) how to construct a one-class classifier from the binary classifier in a straightforward manner without re-training; and 4) how to use the computed kernel parameters for filtering. The existing literature on kernel machines mostly focuses on estimating the optimal kernel parameters via additional cost function(s). In this paper, instead of employing an additional cost function to estimate the kernel-related parameters, we investigate on an analytical solution to predict the actual kernel parameters locally and show how to build a spatial kernel machine with our analytical approach. Classical kernel machines do not perform well on precise shape modeling with a low number of support vectors as demonstrated in this paper. However, we demonstrate and visualize that our analytical approach provides a natural means to relate the kernel parameters to the 2D shapes for sparse shape modeling, where the shape boundary represents the decision boundary. For that, we incorporate the selected kernel function’s geometric properties as an additional constraint into the classifier’s optimization problem by defining an easy-to-explain and intuitive concept: similarity domains. In our experiments, we study and demonstrate how the resulting new kernel machine enhances the capabilities of the classical kernel machines with applications on shape modeling, (geometrically) scaling the non-linear decision boundary at various scales and precise visualization of the kernel parameters in 2D images. [ABSTRACT FROM AUTHOR]

In active and assisted living environments, a major service that can be provided is the automated assessment of elderly people’s well-being. Therefore, activity recognition is required to detect what types of help disabled persons need to support them in their daily life activities. Unfortunately, it is still a difficult task to estimate the size of the required window for online sensor data streams to recognize a specific activity, especially when new sensor events are recorded. This paper proposes a windowing algorithm, which presents promising results to recognize complex human activities for multi-resident homes. The approach is based on the analysis of the sensor data to identify the best fitting sensors that should be considered in a specified window. Moreover, the second part of this paper proposes a set of different statistical spatio-temporal features to recognize human activities. In order to check the overall performance, this approach is tested using the CASAS data set and artificially generated laboratory data using our HBMS simulator. The results show high performance based on different evaluation metrics compared to other approaches. We believe that the proposed windowing approach provides a good approximation of the required window size in order to robustly detect human activities in comparison to other windowing approaches. [ABSTRACT FROM AUTHOR]

Using high-level synthesis techniques, this paper proposes an adaptable high-performance streaming dataflow engine for sparse matrix dense vector multiplication (SpMV) suitable for embedded FPGAs. As the SpMV is a memory-bound algorithm, this engine combines the three concepts of loop pipelining, dataflow graph, and data streaming to utilize most of the memory bandwidth available to the FPGA. The main goal of this paper is to show that FPGAs can provide comparable performance for memory-bound applications to that of the corresponding CPUs and GPUs but with significantly less energy consumption. The experimental results indicate that the FPGA provides higher performance compared to that of embedded GPUs for small and medium-size matrices by an average factor of 3.25 whereas the embedded GPU is faster for larger size matrices by an average factor of 1.58. In addition, the FPGA implementation is more energy efficient for the range of considered matrices by an average factor of 8.9 compared to the embedded CPU and GPU. A case study based on adapting the proposed SpMV optimization to accelerate the support vector machine (SVM) algorithm, one of the successful classification techniques in the machine learning literature, justifies the benefits of utilizing the proposed FPGA-based SpMV compared to that of the embedded CPU and GPU. The experimental results show that the FPGA is faster by an average factor of 1.7 and consumes less energy by an average factor of 6.8 compared to the GPU. [ABSTRACT FROM AUTHOR]

Cold load pick-up (CLPU) has been a critical concern to utilities. Researchers and industry practitioners have underlined the impact of CLPU on distribution system design and service restoration. The recent large-scale deployment of smart meters has provided the industry with a huge amount of data that are highly granular, both temporally and spatially. In this paper, a data-driven framework is proposed for assessing CLPU demand of residential customers using smart meter data. The proposed framework consists of two interconnected layers: 1) At the feeder level, a nonlinear autoregression model is applied to estimate the diversified demand during the system restoration and calculate the CLPU demand ratio. 2) At the customer level, Gaussian mixture models and probabilistic reasoning are used to quantify the CLPU demand increase. The proposed methodology has been verified using real smart meter data and outage cases. [ABSTRACT FROM AUTHOR]

Machine learning has become an emerging method for next-generation smart technologies. It also promotes a development-paradigm shift in sensing technologies, which are essential in various smart applications. In this paper, we develop a data-driven method for a monolithic ion-sensitive field-effect transistor (ISFET) to have both photon and pH bi-detection capabilities simultaneously. The proposed methods are executed based on sequential-bias-reconfiguration of the ISFET. Utilizing support vector machine and back-propagation neural network, the photocurrent and ion-induced current can be calculated and decoupled. To evaluate the proposed method, semi-quantification by classification methods and quantification by regression methods are both examined. This paper has experimentally demonstrated the dual-detection capability of a pH range from 5 to 9 and an intensity range from 0 to $760~\mu \text{W}$ /cm2 with prediction error less than 1.5%. To fulfill low-computation requirement for applications, we further optimize the proposed algorithm by feature reduction to balance performances between accuracy, complexity of data acquisition, and data processing. With the developed machine-learning sensing device, therefore, we successfully demonstrate potentials of data-driven sensing devices in future applications. [ABSTRACT FROM AUTHOR]

During group interactions, we react and modulate our emotions and behaviour to the group through phenomena including emotion contagion and physiological synchrony. Previous work on emotion recognition through video/image has shown that group context information improves the classification performance. However, when using physiological data, literature mostly focuses on intrapersonal models that leave-out group information, while interpersonal models are unexplored. This paper introduces a new interpersonal Weighted Group Synchrony approach, which relies on Electrodermal Activity (EDA) and Heart-Rate Variability (HRV). We perform an analysis of synchrony metrics applied across diverse data representations (EDA and HRV morphology and features, recurrence plot, spectrogram), to identify which metrics and modalities better characterise physiological synchrony for emotion recognition. We explored two datasets (AMIGOS and K-EmoCon), covering different group sizes (4 vs dyad) and group-based activities (video-watching vs conversation). The experimental results show that integrating group information improves arousal and valence classification, across all datasets, with the exception of K-EmoCon on valence. The proposed method was able to attain mean M-F1 of $\approx$ ≈ 72.15% arousal and 81.16% valence for AMIGOS, and M-F1 of $\approx$ ≈ 52.63% arousal, 65.09% valence for K-EmoCon, surpassing previous work results for K-EmoCon on arousal, and providing a new baseline on AMIGOS for long-videos. [ABSTRACT FROM AUTHOR]

An autonomous framework to evaluate layout of a piping design in the form of piping and instrumentation diagram (P&ID) according to a set of standards of marine and offshore industry is proposed. The method starts with transforming a P&ID into a vector x in R^d . Transformation is done based on a concept introduced for piping known as Histogram of Connectivity. The proposed descriptor captures two essential properties of P&ID: attributes of each component and connectivity among the components. Next, linear support vector machine (SVM) is used to learn a classifier from existing compliant and noncompliant designs. Subsequently, the linear classifier can be used to check if an unseen design complies with the standards. In addition, to enable follow up on noncompliant design including correction or modification, a method to analyze the reason of noncompliance prediction by the learned SVM model is introduced. The method has demonstrated encouraging performance in two challenging data sets of designs created with advice from experienced engineers in the industry, based on International Convention for the Prevention of Pollution from Ships (MARPOL) and Rules for Classification of Ships of Lloyd’s Register. Note to Practitioners—This paper is motivated by need of marine and offshore industry for automated solution for design appraisal. This paper aims to address this issue by using a machine learning-based approach. Some compliant and noncompliant designs are provided to a developed algorithm for a machine (or computer) to learn. After learning is completed, the machine is able to classify unseen designs as compliant or noncompliant. As highlighted in this paper, the developed method has demonstrated encouraging performance in two case studies, including specific parts in MARPOL and Rules of Lloyd’s Register. For adoption by industry, necessary steps include collecting some designs (compliant and noncompliant) available in an organization and feeding these into the developed method for learning by machine before it can predict. With ability of highlighting possible connections that cause noncompliance, follow up and correction on a noncompliant design is made possible. [ABSTRACT FROM PUBLISHER]

Condition monitoring of power transformers, which are key components of electrical power systems, is essential to identify incipient faults and avoid catastrophic failures. In this paper machine learning algorithms, i.e., nonlinear autoregressive neural networks and support vector machines, are proposed to model the transformer thermal behavior for the purpose of monitoring. The thermal models are developed based on the historical measurements from nine transformers comprised of two 180-MVA units, four 240-MVA units and three 1000-MVA units. The data consist of load profile, tap position, winding indicator temperature (WTI) measurement, ambient temperature, wind speed and solar radiation. The results are validated against field measurements, and it is clearly demonstrated that the alternative algorithms surpass the IEEE Annex G thermal model. An incipient thermal fault identification algorithm is then proposed and successfully used to identify an issue using measurements taken in the field. This algorithm could be used to alert the operator and plan intervention accordingly. [ABSTRACT FROM AUTHOR]

In this paper, a machine learning (ML) approach is presented that exploits accelerometers data to deal with gesture recognition (GR) problems. The proposed methodology aims at providing high accuracy classification for home automation systems, which are generally user independent, device independent, and device orientation independent, an heterogeneous scenario that has not been fully investigated in previous GR literature. The approach illustrated in this paper is composed of three main steps: event identification; feature extraction; and ML-based classification. The elements of the novelty of the proposed approach are 1) a preprocessing phase based on principal component analysis to increase the performance in real-world scenario conditions and 2) the development of parsimonious novel classification techniques based on sparse Bayesian learning. This methodology is tested on two datasets of four gesture classes (horizontal, vertical, circles, and eight-shaped movements) and on a further dataset with eight classes. In order to authentically describe a real-world home automation environment, the gesture movements are collected from more than 30 people who freely perform any gesture. It results in a dictionary of 12 and 20 different movements, respectively, in the case of the four-class and the eight-class databases. [ABSTRACT FROM AUTHOR]

Feature selection is an optional preprocessing procedure and is frequently used to improve the classification accuracy of a machine learning algorithm by removing irrelevant and/or redundant features. However, in many real-world applications, the test cost is also required for making optimal decisions, in addition to the classification accuracy. To the best of our knowledge, thus far, few studies have been conducted on test-cost-sensitive feature selection (TCSFS). In TCSFS, the objectives are twofold: 1) to improve the classification accuracy and 2) to decrease the test cost. Therefore, in fact, it constitutes a multiobjective optimization problem. In this paper, we transformed this multiobjective optimization problem into a single-objective optimization problem by utilizing a new evaluation function and in this paper, we propose a new general wrapper framework for TCSFS. Specifically, in our proposed framework, we add a new term to the evaluation function of a wrapper feature selection method so that the test cost of measuring features is taken into account. We experimentally tested our proposed framework, using 36 classification problems from the University of California at Irvine (UCI) repository, and compared it to some other state-of-the-art feature selection frameworks. The experimental results showed that our framework allows users to select an optimal feature subset with the minimal test cost, while simultaneously maintaining a high classification accuracy. [ABSTRACT FROM AUTHOR]

The extreme learning machine (ELM) is famous for its single hidden-layer feed-forward neural network which results in much faster learning speed comparing with traditional machine learning techniques. Moreover, extensions of ELM achieve stable classification performances for imbalanced data. In this paper, we introduce a hybrid method combining the extended Kalman filter (EKF) with cost-sensitive dissimilar ELM (CS-D-ELM). The raw data are preprocessed by EKF to produce inputs for the CS-D-ELM classifier. Experimental results show that the proposed method is more suitable for real-time fault diagnosis of air handling units than traditional approaches. [ABSTRACT FROM AUTHOR]

In this paper, we investigate stable patterns of electroencephalogram (EEG) over time for emotion recognition using a machine learning approach. Up to now, various findings of activated patterns associated with different emotions have been reported. However, their stability over time has not been fully investigated yet. In this paper, we focus on identifying EEG stability in emotion recognition. We systematically evaluate the performance of various popular feature extraction, feature selection, feature smoothing and pattern classification methods with the DEAP dataset and a newly developed dataset called SEED for this study. Discriminative Graph regularized Extreme Learning Machine with differential entropy features achieves the best average accuracies of 69.67 and 91.07 percent on the DEAP and SEED datasets, respectively. The experimental results indicate that stable patterns exhibit consistency across sessions; the lateral temporal areas activate more for positive emotions than negative emotions in beta and gamma bands; the neural patterns of neutral emotions have higher alpha responses at parietal and occipital sites; and for negative emotions, the neural patterns have significant higher delta responses at parietal and occipital sites and higher gamma responses at prefrontal sites. The performance of our emotion recognition models shows that the neural patterns are relatively stable within and between sessions. [ABSTRACT FROM AUTHOR]

Hyperspectral imagery (HSI) has shown promising results in real-world applications. However, the technological evolution of optical sensors poses two main challenges in HSI classification: 1) the spectral band is usually redundant and noisy and 2) HSI with millions of pixels has become increasingly common in real-world applications. Motivated by the recent success of hybrid huberized support vector machines (HHSVMs), which inherit the benefits of both lasso and ridge regression, this paper first investigates the advantages of HHSVM for HSI applications. Unfortunately, the existing HHSVM solvers suffer from prohibitive computational costs on large-scale data sets. To solve this problem, this paper proposes simple and effective stochastic HHSVM algorithms for HSI classification. In the stochastic settings, we show that with a probability of at least $1-\varrho $ , our algorithms find an $\epsilon $ -accurate solution using $\tilde {O}({1}/{\lambda _{2}\epsilon })$ iterations. Since the convergence rate of our algorithms does not depend on the size of the training set, our algorithms are suitable for handling large-scale problems. We demonstrate the superiority of our algorithms by conducting experiments on large-scale binary and multiclass classification problems, comparing to the state-of-the-art HHSVM solvers. Finally, we apply our algorithms to real HSI classification and achieve promising results. [ABSTRACT FROM AUTHOR]

In future 5G user-centric ultra-dense networks (UUDN), demands of high data rate and high spectrum efficiency are effectively met by dual connectivity (DC) technology. However, due to huge increase of base stations (BSs) and mobile users (MUs), it becomes difficult for BSs to quickly and precisely select the codeword and provide DC to MUs. Hence, different from some traditional methods, this correspondence paper aims to improve the network performance using the method of machine learning. First, we model the random distribution of BSs by homogeneous Poisson point processes, where each MU is served by millimeter-wave channel. Second, the probabilities that macro cell BS or small cell BS serves the MU are further derived to get the average sum rate (ASR) in UUDN. Third, inspired by ML, we utilize an iterative support vector machine (SVM) classifier to select the codewords of BSs, with sequential minimal optimization (SMO) algorithm used for training all link samples in UUDN. Then, an iterative SVM-SMO classification algorithm is proposed to achieve a highly efficient performance of DC, where the convergence and complexity are also discussed. The sample training and simulation at last are evaluated by Google TensorFlow. The simulation verified that our proposed algorithm gets a higher ASR than the traditional channel estimation based algorithm. In addition, the results also show a lower computational complexity can be achieved by the proposed algorithm as well. [ABSTRACT FROM AUTHOR]

Run-time solutions based on online monitoring and adaptation are required for resilience in nanoscale integrated circuits, as design-time solutions and guard bands are no longer sufficient. Bias temperature instability-induced transistor aging, one of the major reliability threats in nanoscale very large scale integration, degrades path delay over time and may lead to timing failures. Chip health monitoring is, therefore, necessary to track delay changes on a per-chip basis over the chip lifetime operation. However, direct monitoring based on actual measurement of path delays can only track a coarse-grained aging trend in a reactive manner, not suitable for proactive fine-grain adaptations. In this paper, we propose a low cost and fine-grained workload-induced stress monitoring approach, based on machine learning techniques, to accurately predict aging-induced delay. We integrate space and time sampling of selective flip-flops into the runtime monitoring infrastructure in order to reduce the cost of monitoring the workload. The prediction model is trained offline using support-vector regression and implemented in software. This approach can leverage proactive adaptation techniques to mitigate further aging of the circuit by monitoring aging trends. Simulation results for realistic open-source benchmark circuits highlight the accuracy of the proposed approach. [ABSTRACT FROM PUBLISHER]

Supervised approaches classify input data using a set of representative samples for each class, known as training samples. The collection of such samples is expensive and time demanding. Hence, unsupervised feature learning, which has a quick access to arbitrary amounts of unlabeled data, is conceptually of high interest. In this paper, we propose a novel network architecture, fully Conv-Deconv network, for unsupervised spectral-spatial feature learning of hyperspectral images, which is able to be trained in an end-to-end manner. Specifically, our network is based on the so-called encoder-decoder paradigm, i.e., the input 3-D hyperspectral patch is first transformed into a typically lower dimensional space via a convolutional subnetwork (encoder), and then expanded to reproduce the initial data by a deconvolutional subnetwork (decoder). However, during the experiment, we found that such a network is not easy to be optimized. To address this problem, we refine the proposed network architecture by incorporating: 1) residual learning and 2) a new unpooling operation that can use memorized max-pooling indexes. Moreover, to understand the "black box," we make an in-depth study of the learned feature maps in the experimental analysis. A very interesting discovery is that some specific "neurons" in the first residual block of the proposed network own good description power for semantic visual patterns in the object level, which provide an opportunity to achieve "free" object detection. This paper, for the first time in the remote sensing community, proposes an end-to-end fully Conv-Deconv network for unsupervised spectral-spatial feature learning. Moreover, this paper also introduces an in-depth investigation of learned features. Experimental results on two widely used hyperspectral data, Indian Pines and Pavia University, demonstrate competitive performance obtained by the proposed methodology compared with other studied approaches. [ABSTRACT FROM AUTHOR]

Alcoholism is a widely affected disorder that leads to critical brain deficiencies such as emotional and behavioural impairments. One of the prominent sources to detect alcoholism is by analysing Electroencephalogram (EEG) signals. Previously, most of the works have focused on detecting alcoholism using various machine and deep learning algorithms. This paper has used a novel algorithm named Sliding Singular Spectrum Analysis (S-SSA) to decompose and de-noise the EEG signals. We have considered independent component analysis (ICA) to select the prominent alcoholic and non-alcoholic components from the preprocessed EEG data. Later, these components were used to train and test various machine learning models like SVM, KNN, ANN, GBoost, AdaBoost and XGBoost to classify alcoholic and non-alcoholic EEG signals. The sliding SSA-ICA algorithm helps in reducing the computational time and complexity of the machine learning models. To validate the performance of the ICA algorithm, we have compared the computational time and accuracy of ICA with its counterpart, like principal component analysis (PCA) and linear discriminant analysis (LDA). The proposed algorithm is tested on a publicly available UCI alcoholic EEG dataset. To verify the performance of machine learning models, we have calculated various metrics like accuracy, precision, recall and F1 score. Our work reported the highest accuracy of 98.97% with the XGBoost classifier. The validation of the proposed method is done by comparing the classification metrics with the latest state-of-the-art works. [ABSTRACT FROM AUTHOR]

In this paper, we first derive the channel capacity of the full-duplex spatial modulation (FD-SM) system and its upper and lower bounds. Furthermore, different from the traditional optimization-driven decision, we use the data-driven prediction method to solve the transmit antenna selection (TAS) problem in the FD-SM system. Specifically, two novel TAS methods based on the support vector machine (SVM) and deep neural network (DNN) are proposed for reducing the effect of residual self-interference (RSI) on the FD-SM system performance. In our design, we propose a novel feature extraction method based on the principal component analysis (PCA) to help the proposed classifiers improve training. Our simulation results show that our data-driven TAS schemes can approach the optimal performance achieved by exhaustive search while significantly reducing complexity. [ABSTRACT FROM AUTHOR]

This paper presents a system for electrocardiogram (ECG) delineation and arrhythmia classification. The proposed system consists of a front-end integrated circuit, a delineation algorithm implemented on an FPGA board, and an arrhythmia classification algorithm. The front-end circuit applies a ternary second-order Delta modulator to measure the slope variation of the input analog ECG signal. The circuit converts the analog inputs into a pulse density modulated bitstream, whose pulse density is proportional to the slope variation of the input analog signal regardless of the instantaneous amplitude. The front-end chip can detect the minimum slope variation of 3.2 mV/ms2 within a 3 ms timing error. The front-end integrated circuit was fabricated with a 180 nm CMOS process occupying a 0.25 mm2 area with a 151 nW power consumption at the sampling rate of 1 kS/s. Based on the slope variation obtained from the front-end circuit, a delineation algorithm is designed to detect fiducial points in the ECG waveform. The delineation algorithm was tested on a Spartan-6 FPGA. The delineation system can detect the intervals, slopes, and morphology of the QRS/PT waves and form a feature set that contains 22 features. Based on these features, a rotate linear kernel support vector machine (SVM) is applied for patient-specific arrhythmia classification of the ventricular ectopic beat (VEB), supraventricular ectopic beat (SVEB), and heartbeats originating in sinus node. The performance of the proposed system is comparable to the recently published methods while providing a promising solution for the low-complexity implementation of future wearable ECG monitoring systems. [ABSTRACT FROM AUTHOR]

In this paper, we present a framework for performing automatic analysis of Land Use Zones based on Location-Based Social Networks (LBSNs). We model city areas using a hierarchical structure of POIs extracted from foursquare. We encode such structures in kernel machines, e.g., Support Vector Machines, using a new Tree Kernel, i.e., the Hierarchical POI Kernel (HPK), which can take the importance of the individual POIs into account during the substructure matching. This way, HPK projects structures in the space of all their possible substructures such that each dimension corresponds to a semantic structural feature, weighted according to the discriminative power of POIs. We generated four different datasets for the following cities: Barcelona, Lisbon, Amsterdam and Milan, where we trained and tested our models. The results show that our approach largely outperforms previous work and standard baseline built on simple features, such as counts of different POIs. Finally, we apply a mining algorithm to extract the most relevant features (tree fragments) from the implicit TK space according to the weights the kernel machine assigned to them. Our approach can produce an explicit set of representative features that can be used to classify and characterize urban areas. [ABSTRACT FROM AUTHOR]

Structural flexibility plays an essential role in many biological processes. B-factor is an important indicator to measure the flexibility of protein or RNA structures. Many methods were developed to predict protein B-factors, but few studies have been done for RNA B-factor prediction. In this paper, we proposed a new method RNAbval to predict RNA B-factors using random forest. The method was developed using a comprehensive set of features, including the sequence profile and predicted solvent accessibility. RNAbval achieved an improvement of 9.2-20.5 percent over the state-of-the-art method on two benchmark test datasets. The proposed method is available at http://yanglab.nankai.edu.cn/RNAbval/. [ABSTRACT FROM AUTHOR]

The advertising budget allocation problem for financial service is dealt with based on statistical learning and evolutionary computation in this paper. Taking the carry-over effects of the advertising into account, the least squares support vector machine regression (LS-SVMR) is used to construct the response model. A comparison between the proposed response model and traditional regression method based market response models is implemented. The results show the effectiveness and validity of the former model. Taking the budgets allocated to every month in the planning horizon as decision variables, the budget allocation optimization model is built and an improved differential evolution algorithm is used to find the optimal solutions. Finally, the proposed budget allocation method is illustrated by a practical problem. [ABSTRACT FROM AUTHOR]

In this paper, a novel Takagi-Sugeno–Kang (TSK) fuzzy classification system (FCS) is firstly presented for pattern classification tasks. It is distinguished by having the large margin criterion properly integrated into its objective function. In order to exploit the applicability of fuzzy systems in multiview scenarios, the proposed TSK-FCS is extended to a two-view version, called two-view TSK-FCS (TwoV-TSK-FCS), by using a collaborative learning mechanism. The adopted collaborative learning mechanism not only fully considers the independent information of each view, but also effectively discovers the correlation information hidden in the two views. Thus, the performance of TwoV-TSK-FCS can be enhanced accordingly. Comprehensive experiments on two-view synthetic and UCI datasets demonstrate the effectiveness of the proposed two-view FCS. [ABSTRACT FROM AUTHOR]

The core of many large-scale machine learning (ML) applications, such as neural networks (NN), support vector machine (SVM), and convolutional neural network (CNN), is the training algorithm that iteratively updates model parameters by processing massive datasets. From a plethora of studies aiming at accelerating ML, being data parallelization and parameter server, the prevalent assumption is that all data points are equivalently relevant to model parameter updating. In this article, we challenge this assumption by proposing a criterion to measure a data point's effect on model parameter updating, and experimentally demonstrate that the majority of data points are non-critical in the training process. We develop a slim learning framework, termed SlimML, which trains the ML models only on the critical data and thus significantly improves training performance. To such an end, SlimML efficiently leverages a small number of aggregated data points per iteration to approximate the criticalness of original input data instances. The proposed approach can be used by changing a few lines of code in a standard stochastic gradient descent (SGD) procedure, and we demonstrate experimentally, on NN regression, SVM classification, and CNN training, that for large datasets, it accelerates model training process by an average of 3.61 times while only incurring accuracy losses of 0.37 percent. [ABSTRACT FROM AUTHOR]

Available data in machine learning applications is becoming increasingly complex, due to higher dimensionality and difficult classes. There exists a wide variety of approaches to measuring complexity of labeled data, according to class overlap, separability or boundary shapes, as well as group morphology. Many techniques can transform the data in order to find better features, but few focus on specifically reducing data complexity. Most data transformation methods mainly treat the dimensionality aspect, leaving aside the available information within class labels which can be useful when classes are somehow complex. This paper proposes an autoencoder-based approach to complexity reduction, using class labels in order to inform the loss function about the adequacy of the generated variables. This leads to three different new feature learners, Scorer, Skaler and Slicer. They are based on Fisher’s discriminant ratio, the Kullback-Leibler divergence and least-squares support vector machines, respectively. They can be applied as a preprocessing stage for a binary classification problem. A thorough experimentation across a collection of 27 datasets and a range of complexity and classification metrics shows that class-informed autoencoders perform better than 4 other popular unsupervised feature extraction techniques, especially when the final objective is using the data for a classification task. [ABSTRACT FROM AUTHOR]

Automatic assessment of sleep apnea/ hypopnea syndrome (SAHS) based on fewer physiological signals is critical for the success of healthcare at home. However, previous studies that use such settings only achieve a lower assessment accuracy, causing fewer syndromes to be separated for effective diagnosis. This paper presents a 3-stage support vector machines (SVM)-based algorithm for SAHS assessment using a single-channel nasal pressure (NP) signal. In this work, NP signal is utilized for feature extraction. Amplitude features, as well as those extracted using discrete Fourier transform and discrete wavelet transform, are used for machine learning. A total of 58 sets of polysomnography recordings, each with approximately 7 h in duration, were analyzed. This work achieves a sensitivity of 95.7% and a positive predictive value of 90.9%, outperforming previous works using NP signal. Compared with prior studies using only SpO2 signal, this work still achieves better performance and supports more classification levels. Thanks to the low-complexity settings based only on the NP signal, the proposed approach provides a promising solution to SAHS assessment for remote healthcare. [ABSTRACT FROM AUTHOR]

Wi-Fi fingerprint-based indoor localization has recently attracted significant research interest since Wi-Fi devices are widely deployed and practical, and no additional infrastructure is required. However, the Received Signal Strength (RSS) is significantly different on heterogeneous devices, and this difference has a negative impact on localization results. In this paper, we propose a multi-fingerprint and multi-classifier fusion (MFMCF) localization method to improve the localization accuracy and solve the problem of heterogeneous hardware. First, the individual feature set of original RSS fingerprint, signal strength difference (SSD) fingerprint and hyperbolic location fingerprint (HLF) are fused as a composite fingerprint set (CFS), and then the data dimension is reduced by linear discriminant analysis (LDA). Second, three representative machine learning algorithms including K-Nearest Neighbor (KNN), Support Vector Machine (SVM) and Random Forest (RF) are selected and trained to construct an integrated fusion model. Finally, in order to get more accurate predictions, in the online phase, a selective strategy based on entropy is proposed by calculating the entropy of each classifier’s prediction result. Experiments show that MFMCF is an effective scheme to solve the localization problem of heterogeneous devices and improve the localization accuracy. [ABSTRACT FROM AUTHOR]

The additive kernel least squares support vector machine (AK-LS-SVM) has been well used in classification tasks due to its inherent advantages. For example, additive kernels work extremely well for some specific tasks, such as computer vision classification, medical research, and some specialized scenarios. Moreover, the analytical solution using AK-LS-SVM can formulate leave-one-out cross-validation error estimates in a closed form for parameter tuning, which drastically reduces the computational cost and guarantee the generalization performance especially on small and medium datasets. However, AK-LS-SVM still faces two main challenges: 1) improving the classification performance of AK-LS-SVM and 2) saving time when performing a grid search for model selection. Inspired by the stacked generalization principle and the transfer learning mechanism, a layer-by-layer combination of AK-LS-SVM classifiers embedded with transfer learning is proposed in this paper. This new classifier is called deep transfer additive kernel least square support vector machine (DTA-LS-SVM) which overcomes these two challenges. Also, considering that imbalanced datasets are involved in many real-world scenarios, especially for medical data analysis, the deep-transfer element is extended to compensate for this imbalance, thus leading to the development of another new classifier iDTA-LS-SVM. In the hierarchical structure of both DTA-LS-SVM and iDTA-LS-SVM, each layer has an AK-LS-SVM and the predictions from the previous layer act as an additional input feature for the current layer. Importantly, transfer learning is also embedded to guarantee generalization consistency between the adjacent layers. Moreover, both iDTA-LS-SVM and DTA-LS-SVM can ensure the minimal leave-one-out error by using the proposed fast leave-one-out cross validation strategy on the training set in each layer. We compared the proposed classifiers DTA-LS-SVM and iDTA-LS-SVM with the traditional LS-SVM and SVM using additive kernels on seven public UCI datasets and one real world dataset. The experimental results show that both DTA-LS-SVM and iDTA-LS-SVM exhibit better generalization performance and faster learning speed. [ABSTRACT FROM AUTHOR]

This paper aims at learning discriminative part detectors with only image-level labels. To this end, we need to develop effective technologies for both pattern mining and detection learning. Different from previous methods, which train part detectors in one step, we divide the detector learning process into two stages and formulate it as a weak to strong learning framework. In particular, we first learn exemplar detectors from the unaligned patterns and perform a detector-based spectral clustering to produce weak detectors that are only responsible for a few discriminative patterns. In this way, the weak detectors are able to offer right initial patterns for strong detector learning. Second, we learn strong detectors with patterns discovered from the weak detectors, which we formulate as a confidence-loss sparse multiple instance learning (cls-MIL) task. The cls-MIL considers the diversity of positive samples while avoiding drifting away from the well localized ones by assigning a confidence value to each positive sample. The responses of the learned detectors produce an effective mid-level image representation for both image classification and object localization. Experiments conducted on benchmark data sets well demonstrate the superiority of our method over existing approaches. [ABSTRACT FROM AUTHOR]

Falls are a leading health risk for the elderly. Various wearable-based fall detection systems based on machine learning models have been developed to provide emergency alarms and services to improve safety and health-related quality of life. To support long-term healthcare services, the appropriate sampling rate, which plays a vital role in a fall detection system, must be investigated to guarantee the performance accuracy and energy efficiency. Intuitively, decreasing the sampling rate of sensor nodes can provide computing and energy savings. However, there is a lack of research exploring the performance accuracy of fall detection systems in terms of sampling rate, especially for machine learning models. In this paper, the effects of decreasing sampling rates (ranging from 200/128 to 3 Hz) on wearable-based fall detection systems are investigated based on four machine learning models: support vector machine (SVM), $k$ -nearest neighbor, Naïve Bayes, and decision tree. Two emulated fall data sets, the SisFall public data set and the proposed data set of this paper, are used to allow an objective investigation of sampling rates. The findings show that fall detection systems based on SVM modeling and a radial basis function could achieve at least 98% and 97% accuracy, with sampling rates of 11.6 and 5.8 Hz, respectively. Overall, the experimental results demonstrate that a sampling rate of 22 Hz is sufficient for most machine learning models to support wearable-based fall detection systems (accuracy ≥97%). [ABSTRACT FROM AUTHOR]

To protect underage people from accessing objectionable videos in the Internet, an effective objectionable video recognition algorithm is necessary for web filtering. Recently, the multi-instance learning has been introduced for objectionable video recognition and achieves impressive results. However, hand-crafted features as well as redundant and noisy frames in objectionable videos become an intractable problem that inevitably degrades the recognition performance. In this paper, we propose a novel representative prototype selection algorithm embedding deep multi-instance representation learning. In the proposed method, an improved convolutional neural network is designed for multimodal multi-instance feature learning and a self-expressive dictionary learning model based on sparse and low rank constraint is designed to select the representative prototypes from each subspace of instances. Then the bag-level feature is constructed via mapping the bag to the selected prototypes. Experiments on three objectionable video sets show the effectiveness of our method for objectionable video recognition. [ABSTRACT FROM AUTHOR]

In the past few years, kernel-based one-class extreme learning machine (ELM) receives quite a lot of attention by researchers for offline/batch learning due to its noniterative and fast learning capability. This paper extends this concept for adaptive online learning with regularized kernel-based one-class ELM classifiers for detection of outliers, and are collectively referred to as ORK-OCELM. Two frameworks, viz., boundary and reconstruction, are presented to detect the target class in ORK-OCELM. The kernel hyperplane-based baseline one-class ELM model considers whole data in a single chunk, however, the proposed one-class classifiers are adapted in an online fashion from the stream of training samples. The performance of ORK-OCELM is evaluated on a standard benchmark as well as synthetic datasets for both types of environments, i.e., stationary and nonstationary. While evaluating on stationary datasets, these classifiers are compared against batch learning-based one-class classifiers. Similarly, while evaluating on nonstationary datasets, the comparison is done with incremental learning-based online one-class classifiers. The results indicate that the proposed classifiers yield better or similar outcomes for both. In the nonstationary dataset evaluation, adaptability of the proposed classifiers in a changing environment is also demonstrated. It is further shown that the proposed classifiers have large stream data handling capability even under limited system memory. Moreover, the proposed classifiers gain significant time improvement compared to traditional online one-class classifiers (in all aspects of training and testing). A faster learning ability of the proposed classifiers makes them more suitable for real-time anomaly detection. [ABSTRACT FROM AUTHOR]

Road accidents wreck lives. Could technology stop them from happening? Driving better road safety with technology and artificial intelligence are the key elements considered by several carmakers. The key aspect of transportation in the future is to build an ecosystem comprising autonomous, connected, electric and shared mobility. The evolution of autonomous vehicles (AVs) can potentially aid transportation to people and be deployed to resolve mobility-related pain for drivers and safety on roads while changing lanes. Thus, the intelligent assistance system should be smart enough to track such vehicles while deviating into another lane. In this paper, we propose a lane change prediction framework for feature learning, with the aim to have a deep and comprehensive understanding of lane change behaviors, meanwhile, reach a high performance based on the selected features. A time-step dataset with more than 1000 features is constructed from vehicle trajectory data. To identify the key features involved in the original feature set, an XGBoost-based three-step feature learning algorithm is proposed, which integrates the feature importance ranking, metric selection and recursive feature elimination. After analyzing the accuracy of test data from different time segment positions, the sliding window method is applied on a time-step dataset with filtered features to properly select time segments, which are flattened into corresponding time-series dataset for model prediction. In our case studies, a publicly available dataset, Next Generation SIMulation (NGSIM), is adopted to conduct experiments of feature learning and lane change prediction, where we achieved a new state-of-art accuracy with 97.6% under the time-series data of 75 selected features and 1-second window size with predictor XGBoost after adopting proposed three-step method, which is superior to the other state-of-the-art feature selection methods. [ABSTRACT FROM AUTHOR]

Real-time driver distraction detection is the core to many distraction countermeasures and fundamental for constructing a driver-centered driver assistance system. While data-driven methods demonstrate promising detection performance, a particular challenge is how to reduce the considerable cost for collecting labeled data. This paper explored semi-supervised methods for driver distraction detection in real driving conditions to alleviate the cost of labeling training data. Laplacian support vector machine and semi-supervised extreme learning machine were evaluated using eye and head movements to classify two driver states: attentive and cognitively distracted. With the additional unlabeled data, the semi-supervised learning methods improved the detection performance ( $G$-mean) by 0.0245, on average, over all subjects, as compared with the traditional supervised methods. As unlabeled training data can be collected from drivers' naturalistic driving records with little extra resource, semi-supervised methods, which utilize both labeled and unlabeled data, can enhance the efficiency of model development in terms of time and cost. [ABSTRACT FROM PUBLISHER]

Due to design and fabrication outsourcing to foundries, the problem of malicious modifications to integrated circuits (ICs), also known as hardware Trojans (HTs), has attracted attention in academia as well as industry. To reduce the risks associated with Trojans, researchers have proposed different approaches to detect them. Among these approaches, test-time detection approaches have drawn the greatest attention. Many test-time approaches assume the existence of a Trojan-free (TF) chip/model also known as “golden model.” Prior works suggest using reverse engineering (RE) to identify such TF ICs for the golden model. However, they did not state how to do this efficiently. In fact, RE is a very costly process which consumes lots of time and intensive manual effort. It is also very error prone. In this paper, we propose an innovative and robust RE scheme to identify the TF ICs. We reformulate the Trojan-detection problem as clustering problem. We then adapt a widely used machine learning method, K -means clustering, to solve our problem. Simulation results using state-of-the-art tools on several publicly available circuits show that the proposed approach can detect HTs with high accuracy rate. A comparison of this approach with our previously proposed approach [1] is also conducted. Both the limitations and application scenarios of the two methods are discussed in detail. [ABSTRACT FROM PUBLISHER]

Since the brain lesion detection and classification is a vital diagnosis task, in this paper, the problem of brain magnetic resonance imaging (MRI) classification is investigated. Recent advantages in machine learning and deep learning allows the researchers to develop the robust computer-aided diagnosis (CAD) tools for classification of brain lesions. Feature extraction is an essential step in any machine learning scheme. Time-frequency analysis methods provide localized information that makes them more attractive for image classification applications. Owing to the advantages of two-dimensional discrete orthonormal Stockwell transform (2D DOST), we propose to use it to extract the efficient features from brain MRIs and obtain the feature matrix. Since there are some irrelevant features, two-directional two-dimensional principal component analysis ((2D)2PCA) is used to reduce the dimension of the feature matrix. Finally, convolution neural networks (CNNs) are designed and trained for MRI classification. Simulation results indicate that the proposed CAD tool outperforms the recently introduced ones and can efficiently diagnose the MRI scans. [ABSTRACT FROM AUTHOR]

Nonlinear characteristics of solar cells and changes in environmental conditions, such as temperature, and in particular, the intensity of daytime irradiation, make it difficult to identify faults by the conventional means of protection. Therefore, a variety of machine learning techniques are proposed for fault detection in photovoltaic (PV) arrays. In this regard, classifying and identifying the location of a fault event is essential. In addition to fault recognition, selecting the method of fault correction is another issue to be addressed. However, there are scarce investigations in this field. In this paper, a comprehensive method for identifying, classifying, locating, and correcting faults is introduced. The proposed method is assessed with the expansion of the diagnostic space of the graph-based semisupervised learning algorithm and an increased number of class labels. After identifying the type and location of a fault, the system temporarily isolates the fault to function without interruption until it is fully corrected. The problem of overlapping cell data in normal and fault-prone modes is resolved by applying different methods of normalization. The results show that all faults including unlearned and learned in a wide range of environmental conditions, where possible PV arrays are experienced, are properly identified and corrected. Moreover, our studies demonstrate that the proposed system mitigates the output voltage variations over a fault-prone mode. [ABSTRACT FROM AUTHOR]

The ever-increasing amount of data in cellular networks poses challenges for network operators to monitor the quality of experience (QoE). Traditional key quality indicators (KQIs)-based hard decision methods are difficult to undertake the task of QoE anomaly detection in the case of big data. To solve this problem, in this paper, we propose a KQIs-based QoE anomaly detection framework using semi-supervised machine learning algorithm, i.e., iterative positive sample aided one-class support vector machine (IPS-OCSVM). There are four steps for realizing the proposed method while the key step is combining machine learning with the network operator's expert knowledge using OCSVM. Our proposed IPS-OCSVM framework realizes QoE anomaly detection through soft decision and can easily fine-tune the anomaly detection ability on demand. Moreover, we prove that the fluctuation of KQIs thresholds based on expert knowledge has a limited impact on the result of anomaly detection. Finally, experiment results are given to confirm the proposed IPS-OCSVM framework for QoE anomaly detection in cellular networks. [ABSTRACT FROM AUTHOR]

Physical unclonable function (PUF) is an advanced hardware security technology. Most conventional encryption approaches rely on the secure keys stored in nonvolatile memory, which are vulnerable to physical attacks. In contrast, PUF exploits the hardware fabrication variations to generate secure keys. As there are significant fabrication induced variations in carbon nanotube (CNT)-based circuits, they are natural candidates for building highly secure PUFs. However, existing PUFs are reported to be vulnerable to machine learning modeling attacks. In this paper, we develop a novel CNT PUF design through leveraging Lorenz chaotic system. Lorenz chaotic system magnifies the differences among responses of similar challenges. This salient feature gives the superior security performance, making the PUF design resistant to machine learning modeling attacks. Our experimental results demonstrate that these attacking methods can achieve very high bit-wise prediction rates for the standard CNT PUF. In contrast, when hacking the proposed Lorenz chaotic system-based CNT PUF, they become much less effective, where only up to 55% bit-wise prediction rates can be achieved. [ABSTRACT FROM AUTHOR]

This paper aims at evaluating the capabilities to detect road traversability in urban and extra-urban scenarios of support vector machine-based classifiers that use local descriptors extracted from point cloud data. The evaluation of the proposed classifiers is carried out by using four different kernels and comparing five point descriptors obtained from geometric and appearance-based features. A comparison among the performance of descriptors individually has demonstrated that the normal vector-based descriptor achieves an accuracy of 88%, outperforming by about 6%–15% all the other considered ones. To further improve the interpretation capabilities, the space of features is augmented by merging the components of each point descriptor, reaching 92% classification accuracy. A set of test scenarios have been acquired during an extensive experimental campaign using an all-terrain vehicle. Tests on real data show high classification performance for road scenarios and rural environments; the generality of the method makes it applicable for different types of mobile robots including, but not limited to, autonomous vehicles. [ABSTRACT FROM PUBLISHER]

With the rapid development of machine learning in the medical cloud system, cloud-assisted medical computing provides a concrete platform for remote rapid medical diagnosis services. Support vector machine (SVM), as one of the important algorithms of machine learning, has been widely used in the field of medical diagnosis for its high classification accuracy and efficiency. In some existing schemes, healthcare providers train diagnostic models with SVM algorithms and provide online diagnostic services to doctors. Doctors send the patient’s case report to the diagnostic models to obtain the results and assist in clinical diagnosis. However, case report involves patients’ privacy, and patients do not want their sensitive information to be leaked. Therefore, the protection of patient’s privacy has become an important research direction in the field of online medical diagnosis. In this paper, we propose a privacy-preserving medical diagnosis scheme based on multi-class SVMs. The scheme is based on the distributed two trapdoors public key cryptosystem (DT-PKC) and Boneh-Goh-Nissim (BGN) cryptosystem. We design a secure computing protocol to compute the core process of the SVM classification algorithm. Our scheme can deal with both linearly separable data and nonlinear data while protecting the privacy of user data and support vectors. The results show that our scheme is secure, reliable, scalable with high accuracy. [ABSTRACT FROM AUTHOR]