Search

Active learning method (ALM) is a powerful fuzzy–based soft computing methodology suitable for various applications such as function modeling, control systems, clustering and classification. Despite considerable advantages, the main computational engine of ALM, ink drop spread (IDS), is memory-intensive, which imposes significant area overheads in the hardware realization of the ALM for real–time applications. In this paper, we propose a compressed model for ALM which greatly alleviates the storage limitations. The proposed approach employs a distinct inference algorithm, enabling a significant reduction in memory utilization from $O(N^{2})$ to $O(2N)$ for a multi–input single–output (MISO) system. Also, the computational costs in both training and inference modes are decreased to only a few additions and multiplications. Furthermore, we develop a memory–efficient digital architecture for the proposed compressed ALM algorithm that can be leveraged for various computing systems through configuring a few registers. Finally, we assess the performance of the proposed approach using various function modeling and classification applications and provide a comparison with conventional ALM and some other well-know approaches. Simulation and hardware implementation results demonstrate that the proposed approach achieves reduced noise sensitivity with $128\times $ reduction in the average memory usage while realizing comparable accuracy compared to the other approaches studied herein. [ABSTRACT FROM AUTHOR]

To reduce multiplication operations in convolution of convolutional neural networks (CNNs), there are three widely used convolutional acceleration algorithms, i.e., Winograd, FFT and FFA. However, current accelerators based on these convolutional acceleration algorithms have issues on flexibility and efficiency. Firstly, some accelerators utilized a combination of these acceleration algorithms and employed multiple types of computational units to achieve their respective advantages. As a result, some computational units are left unused when the best-performing unit is working, which causes much area inefficiency. Secondly, current accelerators tend to choose small parameters of these convolutional acceleration algorithms to avoid unacceptable precision loss, as a result, they are hardly to support large kernel sizes and lack of flexibility. Thirdly, these acceleration algorithms are typically presented for 1-stride convolutions, consequently, few implementation considers the acceleration of large-stride convolutions, which is a major restriction to hardware flexibility. This paper proposed a stride-based convolution decomposition method (SCDM) to reform different convolution shapes (i.e., kernel sizes & strides) to an identical pattern. With the aid of SCDM, a Winograd-stretched and hardware-efficient design (WHD) is presented to utilize one uniform computational unit for the acceleration of different convolution shapes, which combines complementary performance advantages on both Winograd F(4,3)andF(4,2) units. Compared to current FFT-based or FFA-based works, WHD can stretch the use range of Winograd and simplify implementation, thereby achieving hardware flexibility and efficiency. Evaluation results show that 34.08%~55.41% operation reduction were achieved on six CNN models, while incurring a slight hardware overhead. [ABSTRACT FROM AUTHOR]

In this paper, we study how to utilize forecasts to design online electrical vehicle (EV) charging algorithms that can attain strong performance guarantees. We consider the scenario of an aggregator serving a large number of EVs together with its background load, using both its own renewable energy (for free) and the energy procured from the external grid. The goal of the aggregator is to minimize its peak procurement from the grid, subject to the constraint that each EV has to be fully charged before its deadline. Further, the aggregator can predict the future demand and the renewable energy supply with some levels of uncertainty. We show that such prediction can be very effective in reducing the competitive ratios of online control algorithms, and even allow online algorithms to achieve close-to-offline-optimal peak. Specifically, we first propose a 2-level increasing precision model (2-IPM), to model forecasts with different levels of accuracy. We then develop a powerful computational approach that can compute the optimal competitive ratio under 2-IPM over any online algorithm, and also online algorithms that can achieve the optimal competitive ratio. Simulation results show that, even with up to 20% day-ahead prediction errors, our online algorithms still achieve competitive ratios fairly close to 1, which are much better than the classic results in the literature with a competitive ratio of $e$ . The second contribution of this paper is that we solve a dilemma for online algorithm design, e.g., an online algorithm with good competitive ratio may exhibit poor average-case performance. We propose a new Algorithm-Robustification procedure that can convert an online algorithm with good average-case performance to one with both the optimal competitive ratio and good average-case performance. We demonstrate via trace-based simulations the superior performance of the robustified version of a well-known heuristic algorithm based on model predictive control. [ABSTRACT FROM PUBLISHER]

The traditional multilayer perceptron (MLP) using a McCulloch–Pitts neuron model is inherently limited to a set of neuronal activities, i.e., linear weighted sum followed by nonlinear thresholding step. Previously, generalized operational perceptron (GOP) was proposed to extend the conventional perceptron model by defining a diverse set of neuronal activities to imitate a generalized model of biological neurons. Together with GOP, a progressive operational perceptron (POP) algorithm was proposed to optimize a predefined template of multiple homogeneous layers in a layerwise manner. In this paper, we propose an efficient algorithm to learn a compact, fully heterogeneous multilayer network that allows each individual neuron, regardless of the layer, to have distinct characteristics. Based on the complexity of the problem, the proposed algorithm operates in a progressive manner on a neuronal level, searching for a compact topology, not only in terms of depth but also width, i.e., the number of neurons in each layer. The proposed algorithm is shown to outperform other related learning methods in extensive experiments on several classification problems. [ABSTRACT FROM AUTHOR]

Feature selection (FS) is an important component of many pattern recognition tasks. In these tasks, one is often confronted with very high-dimensional data. FS algorithms are designed to identify the relevant feature subset from the original features, which can facilitate subsequent analysis, such as clustering and classification. Structured sparsity-inducing feature selection (SSFS) methods have been widely studied in the last few years, and a number of algorithms have been proposed. However, there is no comprehensive study concerning the connections between different SSFS methods, and how they have evolved. In this paper, we attempt to provide a survey on various SSFS methods, including their motivations and mathematical representations. We then explore the relationship among different formulations and propose a taxonomy to elucidate their evolution. We group the existing SSFS methods into two categories, i.e., vector-based feature selection (feature selection based on lasso) and matrix-based feature selection (feature selection based on lr,p -norm). Furthermore, FS has been combined with other machine learning algorithms for specific applications, such as multitask learning, multilabel learning, multiview learning, classification, and clustering. This paper not only compares the differences and commonalities of these methods based on regression and regularization strategies, but also provides useful guidelines to practitioners working in related fields to guide them how to do feature selection. [ABSTRACT FROM AUTHOR]

Conjugate gradient (CG) methods are a class of important methods for solving linear equations and nonlinear optimization problems. In this paper, we propose a new stochastic CG algorithm with variance reduction and we prove its linear convergence with the Fletcher and Reeves method for strongly convex and smooth functions. We experimentally demonstrate that the CG with variance reduction algorithm converges faster than its counterparts for four learning models, which may be convex, nonconvex or nonsmooth. In addition, its area under the curve performance on six large-scale data sets is comparable to that of the LIBLINEAR solver for the $L2$ -regularized $L2$ -loss but with a significant improvement in computational efficiency. 1 CGVR algorithm is available on github: https://github.com/xbjin/cgvr [ABSTRACT FROM AUTHOR]

This paper addresses the robust gradient learning (RGL) problem. Gradient learning models aim at learning the gradient vector of some target functions in supervised learning problems, which can be further used to applications, such as variable selection, coordinate covariance estimation, and supervised dimension reduction. However, existing GL models are not robust to outliers or heavy-tailed noise. This paper provides an RGL framework to address this problem in both regression and classification. This is achieved by introducing a robust regression loss function and proposing a robust classification loss. Moreover, our RGL algorithm works in an instance-based kernelized dictionary instead of some fixed reproducing kernel Hilbert space, which may provide more flexibility. To solve the proposed nonconvex model, a simple computational algorithm based on gradient descent is provided and the convergence of the proposed method is also analyzed. We then apply the proposed RGL model to applications, such as nonlinear variable selection and coordinate covariance estimation. The efficiency of our proposed model is verified on both synthetic and real data sets. [ABSTRACT FROM AUTHOR]

The semisupervised least squares support vector machine (LS-S3VM) is an important enhancement of least squares support vector machines in semisupervised learning. Given that most data collected from the real world are without labels, semisupervised approaches are more applicable than standard supervised approaches. Although a few training methods for LS-S3VM exist, the problem of deriving the optimal decision hyperplane efficiently and effectually has not been solved. In this paper, a fully weighted model of LS-S3VM is proposed, and a simple integer programming (IP) model is introduced through an equivalent transformation to solve the model. Based on the distances between the unlabeled data and the decision hyperplane, a new indicator is designed to represent the possibility that the label of an unlabeled datum should be reversed in each iteration during training. Using the indicator, we construct an extended candidate set consisting of the indices of unlabeled data with high possibilities, which integrates more information from unlabeled data. Our algorithm is degenerated into a special scenario of the previous algorithm when the extended candidate set is reduced into a set with only one element. Two strategies are utilized to determine the descent directions based on the extended candidate set. Furthermore, we developed a novel method for locating a good starting point based on the properties of the equivalent IP model. Combined with the extended candidate set and the carefully computed starting point, a fast algorithm to solve LS-S3VM quasi-optimally is proposed. The choice of quasi-optimal solutions results in low computational cost and avoidance of overfitting. Experiments show that our algorithm equipped with the two designed strategies is more effective than other algorithms in at least one of the following three aspects: 1) computational complexity; 2) generalization ability; and 3) flexibility. However, our algorithm and other algorithms have similar levels of performance in the remaining aspects. [ABSTRACT FROM AUTHOR]

We consider the problem of designing static feedback gains subject to a priori structural constraints, which is a nonconvex problem in general. Previous work has focused on either characterizing special structures that result into convex formulations, or employing certain techniques to allow convex relaxations of the original problem. In this paper, by exploiting the underlying sparsity properties of the problem, and using chordal decomposition, we propose a scalable algorithm to obtain structured feedback gains to stabilize a large-scale system. We first extend the chordal decomposition theorem for positive semidefinite matrices to the case of matrices with block-chordal sparsity. Then, a block-diagonal Lyapunov matrix assumption is used to convert the design of structured feedback gains into a convex problem, which inherits the sparsity pattern of the original problem. Combining these two results, we propose a sequential design method to obtain structured feedback gains clique-by-clique over a clique tree of the block-chordal matrix, which only needs local information and helps ensure privacy of model data. Several illustrative examples demonstrate the efficiency and scalability of the proposed sequential design method. [ABSTRACT FROM PUBLISHER]

Linear discriminant analysis (LDA) as a well-known supervised dimensionality reduction method has been widely applied in many fields. However, the lack of sparsity in the LDA solution makes interpretation of the results challenging. In this paper, we propose a new model for sparse uncorrelated LDA (ULDA). Our model is based on the characterization of all solutions of the generalized ULDA. We incorporate sparsity into the ULDA transformation by seeking the solution with minimum \ell 1 -norm from all minimum dimension solutions of the generalized ULDA. The problem is then formulated as an \ell 1 -minimization problem with orthogonality constraint. To solve this problem, we devise two algorithms: 1) by applying the linearized alternating direction method of multipliers and 2) by applying the accelerated linearized Bregman method. Simulation studies and high-dimensional real data examples demonstrate that our algorithms not only compute extremely sparse solutions but also perform well in classification. [ABSTRACT FROM AUTHOR]

As a special class of recurrent neural network, Zhang neural network (ZNN) has been recently proposed since 2001 for solving various time-varying problems, and has shown high efficiency and excellent performance for solving the problems in the real domain. In this paper, to solve online the time-varying complex generalized inverse (in most cases, the pseudoinverse) problem in the complex domain, a new type of complex-valued ZNN is further proposed and investigated. The design of such a complex ZNN is based on a complex Zhang function (ZF) which is indefinite and quite different from the usual error function (specially, the scalar-valued energy function) in the studies of conventional algorithms. By introducing five different complex ZFs, five different complex ZNN models (termed complex ZNN-I, ZNN-II, ZNN-III, ZNN-IV, and ZNN-V models) are proposed, developed, and investigated for the online solution of the time-varying complex generalized inverse matrices. Theoretical results of convergence analysis are presented to show the desirable properties of complex ZNN models. In addition, we discover the link between the proposed complex ZNN models and the Getz–Marsden dynamic system in the complex domain. Computer-simulation results further demonstrate the effectiveness of complex ZNN models based on different complex ZFs for the time-varying complex generalized inverse matrices. [ABSTRACT FROM AUTHOR]

Multi-process networks figure in many engineering applications. Parameterized discrete event systems provide a convenient means of modeling such networks when the number of subprocesses is arbitrary, unknown or time-varying. Unfortunately, some key properties of these networks, such as nonblocking and deadlock-freedom, are undecidable. Moreover, mathematical tools supporting analysis of these networks are limited. This paper introduces a novel mathematical notion, weak invariant simulation, which is adapted to the analysis of synchronous products of discrete event systems. Furthermore, it proposes an efficient method to check whether a finite-state generator weakly invariantly simulates another finite-state generator with respect to a specific subalphabet. This new simulation relation is used to define a tractable subclass of parameterized ring networks of isomorphic subprocesses in which deadlock-freedom is decidable. Within this framework, a procedure is given to determine the reachable deadlocked states of the network. The effectiveness of the procedure is demonstrated by the deadlock analysis of a version of the dining philosophers problem. [ABSTRACT FROM AUTHOR]

The inverse of a recently proposed hysteresis and creep circuit model is proposed and discussed. The model is particularly suitable for piezoelectric actuators and its inverse can be used for open-loop compensation of the undesired nonlinearities in high-precision applications. The inverse model is defined, analyzed in terms of conditions ensuring a correct compensation, and discretized to provide a digital compensation algorithm suitable for implementation in low-cost programmable devices. Quantitative results on experimental data are provided and discussed, including the compensation on an atomic force microscope. [ABSTRACT FROM PUBLISHER]