Search

Fast-growing scientific papers pose the problem of rapidly and accurately finding a list of reference papers for a given manuscript. Citation recommendation is an indispensable technique to overcome this obstacle. In this paper, we propose a citation recommendation approach via mutual reinforcement on a three-layered graph, in which each paper, author or venue is represented as a vertex in the paper layer, author layer, and venue layer, respectively. For personalized recommendation, we initiate the random walk separately for each query researcher. However, this has a high computational complexity due to the large graph size. To solve this problem, we apply a three-layered interactive clustering approach to cluster related vertices in the graph. Personalized citation recommendations are then made on the subgraph, generated by the clusters associated with each researcher’s needs. When evaluated on the ACL anthology network, DBLP, and CiteSeer ML data sets, the performance of our proposed model-based citation recommendation approach is comparable with that of other state-of-the-art citation recommendation approaches. The results also demonstrate that the personalized recommendation approach is more effective than the nonpersonalized recommendation approach. [ABSTRACT FROM AUTHOR]

In this paper, a Takenaka-Malmquist (TM) basis function based equivalent data-model is established by an adaptive rational decomposition for the finite-time interval trajectory tracking control and real-time control of a class of nonlinear systems in the frequency domain. This data model can adaptively learn and match the control process of nonlinear systems. As a result, the proposed trajectory tracking as well as real-time control method can reflect the feature of adaptive learning in order-by-order decomposition, and the feasibility of the proposed method is guaranteed by the convergence of adaptive decomposition by TM basis function under the maximum selection principle (MSP) in Hardy space $H^{2}(\mathbb {D})$. Compared with the traditional model-free control method, this data learning model which matches the control process has obvious advantages in the system model expression and control accuracy. Simulation results at the end of this paper show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper presents a sequential learning framework for sensors in a network, where a few sensors assume the role of an instructor to train other sensors in the network. The instructors provide estimated labels for measurements of new sensors. These labels are possibly noisy, because a classifier of the instructor may not be perfect. A recursive density estimator is proposed to obtain the true measurement model (i.e., the observation density conditioned on the label) in spite of the training with noisy labels. Specifically, this paper answers the question “Can a sensor train other sensors?”, provides necessary conditions for sensors to act as instructors, presents a sequential learning framework using recursive nonparametric kernel density estimation, and provides a convergence rate for the expected error in an observation density. The underlying concepts are illustrated and validated with simulation results. [ABSTRACT FROM AUTHOR]

Recent years have witnessed an increasing need for the automated classification of sequential data, such as activities of daily living, social media interactions, financial series, and others. With the continuous flow of new data, it is critical to classify the observations on-the-fly and without being limited by a predetermined number of classes. In addition, a model should be able to update its parameters in response to a possible evolution in the distributions of the classes. This compelling problem, however, does not seem to have been adequately addressed in the literature, since most studies focus on offline classification over predefined class sets. In this paper, we present a principled solution for this problem based on an adaptive online system leveraging Markov switching models and hierarchical Dirichlet process priors. This adaptive online approach is capable of classifying the sequential data over an unlimited number of classes while meeting the memory and delay constraints typical of streaming contexts. In this paper, we introduce an adaptive “learning rate” that is responsible for balancing the extent to which the model retains its previous parameters or adapts to new observations. Experimental results on stationary and evolving synthetic data and two video data sets, TUM Assistive Kitchen and collated Weizmann, show a remarkable performance in terms of segmentation and classification, particularly for sequences from evolutionary distributions and/or those containing previously unseen classes. [ABSTRACT FROM AUTHOR]

Sparse nonnegative matrix factorization (SNMF) aims to factorize a data matrix into two optimized nonnegative sparse factor matrices, which could benefit many tasks, such as document-word co-clustering. However, the traditional SNMF typically assumes the number of latent factors (i.e., dimensionality of the factor matrices) to be fixed. This assumption makes it inflexible in practice. In this paper, we propose a doubly sparse nonparametric NMF framework to mitigate this issue by using dependent Indian buffet processes (dIBP). We apply a correlation function for the generation of two stick weights associated with each column pair of factor matrices while still maintaining their respective marginal distribution specified by IBP. As a consequence, the generation of two factor matrices will be columnwise correlated. Under this framework, two classes of correlation function are proposed: 1) using bivariate Beta distribution and 2) using Copula function. Compared with the single IBP-based NMF, this paper jointly makes two factor matrices nonparametric and sparse, which could be applied to broader scenarios, such as co-clustering. This paper is seen to be much more flexible than Gaussian process-based and hierarchial Beta process-based dIBPs in terms of allowing the two corresponding binary matrix columns to have greater variations in their nonzero entries. Our experiments on synthetic data show the merits of this paper compared with the state-of-the-art models in respect of factorization efficiency, sparsity, and flexibility. Experiments on real-world data sets demonstrate the efficiency of this paper in document-word co-clustering tasks. [ABSTRACT FROM AUTHOR]

Large-scale neuromorphic computing requires the multi-chip network to provide high computing power. Efficient routing schemes and on-chip router design are necessary for handling various inter-chip transmission patterns. In this paper, we propose a hybrid-mode on-chip router that supports both multicast and unicast routing for the large-scale neuromorphic simulation. Two routing schemes, namely Cache-like Spike Weight Indexing and General Unicast Flow Control, are proposed to accommodate the chip-to-chip transmission of spike and non-spike data. This work is evaluated on a neuromorphic platform built with an $8\times 8$ FPGA chips array. Running a simulation of 1M neurons at 200MHz, the proposed router achieves a processing latency of 25ns and a chip-to-chip latency of 287ns. Working in the unicast mode, the router can synchronize status flags of all chips within $5 ~\mu \text{s}$. Moreover, it reduces the peak spike traffic by 25.65% with the help of Load-aware Multicast Routing, compared with other multicast routing strategies. [ABSTRACT FROM AUTHOR]

This paper proposes a Bayesian nonparametric framework for clustering axially symmetric data. Our approach is based on a Dirichlet processes mixture model with Watson distributions, which can also be considered as the infinite Watson mixture model. In this paper, first, we extend the finite Watson mixture model into its infinite counterpart based on the framework of truncated Dirichlet process mixture model with a stick-breaking representation. Second, we propose a coordinate ascent mean-field variational inference algorithm that can effectively learn the parameters of our model with closed-form solutions; Third, to cope with a massive data set, we develop a stochastic variational inference algorithm to learn the proposed model through the method of stochastic gradient ascent; Finally, the proposed nonparametric Bayesian model is evaluated through simulated axially symmetric data sets and a real-world application, namely, gene expression data clustering. [ABSTRACT FROM AUTHOR]

Matrix factorization (MF) has been widely used to discover the low-rank structure and to predict the missing entries of data matrix. In many real-world learning systems, the data matrix can be very high dimensional but sparse. This poses an imbalanced learning problem since the scale of missing entries is usually much larger than that of the observed entries, but they cannot be ignored due to the valuable negative signal. For efficiency concern, existing work typically applies a uniform weight on missing entries to allow a fast learning algorithm. However, this simplification will decrease modeling fidelity, resulting in suboptimal performance for downstream applications. In this paper, we weight the missing data nonuniformly, and more generically, we allow any weighting strategy on the missing data. To address the efficiency challenge, we propose a fast learning method, for which the time complexity is determined by the number of observed entries in the data matrix rather than the matrix size. The key idea is twofold: 1) we apply truncated singular value decomposition on the weight matrix to get a more compact representation of the weights and 2) we learn MF parameters with elementwise alternating least squares (eALS) and memorize the key intermediate variables to avoid repeating computations that are unnecessary. We conduct extensive experiments on two recommendation benchmarks, demonstrating the correctness, efficiency, and effectiveness of our fast eALS method. [ABSTRACT FROM AUTHOR]

The computation-intensive circuit simulation makes the analog circuit sizing challenging for large-scale/complicated analog/RF circuits. A Bayesian optimization approach has been proposed recently for the optimization problems involving the evaluations of black-box functions with high computational cost in either objective functions or constraints. In this paper, we propose a weighted expected improvement-based Bayesian optimization approach for automated analog circuit sizing. Gaussian processes (GP) are used as the online surrogate models for circuit performances. Expected improvement is selected as the acquisition function to balance the exploration and exploitation during the optimization procedure. The expected improvement is weighted by the probability of satisfying the constraints. In this paper, we propose a complete Bayesian optimization framework for the optimization of analog circuits with constraints for the first time. The existing GP model-based optimization methods for analog circuits take the GP models as either offline models or as assistance for the evolutionary algorithms. We also extend the Bayesian optimization algorithm to handle multi-objective optimization problems. Compared with the state-of-the-art approaches listed in this paper, the proposed Bayesian optimization method achieves better optimization results with significantly less number of simulations. [ABSTRACT FROM PUBLISHER]

This paper proposes a data-driven learning control method for stochastic nonlinear systems under random communication conditions, including data dropouts, communication delays, and packet transmission disordering. A renewal mechanism is added to the buffer to regulate the arrived packets, and a recognition mechanism is introduced to the controller for the selection of suitable update packets. Both intermittent and successive update schemes are proposed based on the conventional P-type iterative learning control algorithm, and are shown to converge to the desired input with probability one. The convergence and effectiveness of the proposed algorithms are verified by means of illustrative simulations. [ABSTRACT FROM AUTHOR]

This paper presents a 7-bit digital to phase converter (DPC) for high speed clock and data recovery (CDR) applications which is capable of generating multi-phase clocks at 7-GHz frequency. Fabricated in a standard 65-nm CMOS technology, the design introduces a modified phase interpolator (PI) and a quadrature phase corrector (QPC) to reduce the effect of the circuit imperfections on the DPC’s resolution and linearity. Employing a 14-GHz quadrature reference clock, the DPC achieves DNL/INL of 0.7/6 LSB respectively while consuming 40.5 mW power from 1.05 V supply. [ABSTRACT FROM AUTHOR]

This paper describes the learning and control capabilities of a biologically constrained bottom-up model of the mammalian cerebellum. Results are presented from six tasks: 1) eyelid conditioning; 2) pendulum balancing; 3) proportional–integral–derivative control; 4) robot balancing; 5) pattern recognition; and 6) MNIST handwritten digit recognition. These tasks span several paradigms of machine learning, including supervised learning, reinforcement learning, control, and pattern recognition. Results over these six domains indicate that the cerebellar simulation is capable of robustly identifying static input patterns even when randomized across the sensory apparatus. This capability allows the simulated cerebellum to perform several different supervised learning and control tasks. On the other hand, both reinforcement learning and temporal pattern recognition prove problematic due to the delayed nature of error signals and the simulator’s inability to solve the credit assignment problem. These results are consistent with previous findings which hypothesize that in the human brain, the basal ganglia is responsible for reinforcement learning, while the cerebellum handles supervised learning. [ABSTRACT FROM PUBLISHER]

In recent years, the performance of semisupervised learning (SSL) has been theoretically investigated. However, most of this theoretical development has focused on binary classification problems. In this paper, we take it a step further by extending the work of Castelli and Cover to the multiclass paradigm. In particular, we consider the key problem in SSL of classifying an unseen instance x into one of $K$ different classes, using a training data set sampled from a mixture density distribution and composed of $l$ labeled records and $u$ unlabeled examples. Even under the assumption of identifiability of the mixture and having infinite unlabeled examples, labeled records are needed to determine the $K$ decision regions. Therefore, in this paper, we first investigate the minimum number of labeled examples needed to accomplish that task. Then, we propose an optimal multiclass learning algorithm, which is a generalization of the optimal procedure proposed in the literature for binary problems. Finally, we make use of this generalization to study the probability of error when the binary class constraint is relaxed. [ABSTRACT FROM PUBLISHER]

The paper introduces a new concept of a true random number generator (TRNG). Most metastability-based solutions operate on the uncertainty of a logical output state of a device (flip-flop, D-latch) aimed to be resolved from an exact metastable point. However, it has been shown that the metastable point of a bistable circuit (which is practically impossible to reach) does not guarantee absolute randomness or sufficient entropy. We propose the concept of a device in which the direct proximity of the metastable point is not mandatory. In our concept the transition times of two devices are compared. Such construction is less sensitive to the proximity of the metastable point, temperature fluctuations, and power supply instabilities. The paper briefly describes the metastability phenomena in general and other known metastability-based TRNG concepts. A new concept of a dual-metastability time-competitive generator is presented, analyzed both numerically and theoretically, and verified based on the sample circuit's implementation. Empirical and statistical test results are presented. [ABSTRACT FROM AUTHOR]

In this paper, we propose a label-less learning for emotion cognition (LLEC) to achieve the utilization of a large amount of unlabeled data. We first inspect the unlabeled data from two perspectives, i.e., the feature layer and the decision layer. By utilizing the similarity model and the entropy model, this paper presents a hybrid label-less learning that can automatically label data without human intervention. Then, we design an enhanced hybrid label-less learning to purify the automatic labeled data. To further improve the accuracy of emotion detection model and increase the utilization of unlabeled data, we apply enhanced hybrid label-less learning for multimodal unlabeled emotion data. Finally, we build a real-world test bed to evaluate the LLEC algorithm. The experimental results show that the LLEC algorithm can improve the accuracy of emotion detection significantly. [ABSTRACT FROM AUTHOR]

The space of graphs is often characterized by a nontrivial geometry, which complicates learning and inference in practical applications. A common approach is to use embedding techniques to represent graphs as points in a conventional Euclidean space, but non-Euclidean spaces have often been shown to be better suited for embedding graphs. Among these, constant-curvature Riemannian manifolds (CCMs) offer embedding spaces suitable for studying the statistical properties of a graph distribution, as they provide ways to easily compute metric geodesic distances. In this paper, we focus on the problem of detecting changes in stationarity in a stream of attributed graphs. To this end, we introduce a novel change detection framework based on neural networks and CCMs, which takes into account the non-Euclidean nature of graphs. Our contribution in this paper is twofold. First, via a novel approach based on adversarial learning, we compute graph embeddings by training an autoencoder to represent graphs on CCMs. Second, we introduce two novel change detection tests operating on CCMs. We perform experiments on synthetic data, as well as two real-world application scenarios: the detection of epileptic seizures using functional connectivity brain networks and the detection of hostility between two subjects, using human skeletal graphs. Results show that the proposed methods are able to detect even small changes in a graph-generating process, consistently outperforming approaches based on Euclidean embeddings. [ABSTRACT FROM AUTHOR]

Voltage source converters (VSCs) have been widely used in modern power systems. Consequently, reliable operations of VSCs are essential for the power system to operate properly. A parameter estimator (PE) for a VSC system acts as a useful tool for problem diagnosis and preventive maintenance. Generally speaking, depending on how the measurement data are attained, a parameter estimation method can be classified into invasive and non-invasive. Non-invasive methods, which do not require any external excitation, are generally preferred for parameter estimation. However, most of the PEs can only estimate some parameters of the converters while treating others constant. This paper proposes a non-invasive parameter estimation method, which is able to predict simultaneously the circuit and control parameters of a VSC. To achieve this, the accurate open-loop and closed-loop steady-state models of the VSC are first derived. These models will be used to generate the training data sets to obtain the artificial neural network models to account for various uncertainties in a practical circuit. All the training data are generated by the derived VSC models, which are constructed in MATLAB and run off-line. The proposed method has been verified via PSCAD/EMTDC and validated with the experiment. [ABSTRACT FROM AUTHOR]

In complex-valued Hopfield neural networks (CHNNs), the neuron states are complex numbers whose amplitudes are: 1) they can also be described in special orthogonal matrices of order and 2) here, we propose a new Hopfield model, the $O(2)$ -valued Hopfield neural network [ $O(2)$ -HNN], whose neuron states are extended to orthogonal matrices. Its neuron states are embedded in 4-D space, while those of CHNNs are embedded in 2-D space. Computer simulations were conducted to compare the noise tolerance (NT) and storage capacity (SC) of CHNNs, $O(2)$ -HNNs, and rotor Hopfield neural networks. In terms of SC, $O(2)$ -HNNs outperformed the others, while in NT, they outdid CHNNs. [ABSTRACT FROM AUTHOR]

This paper presents a family of methods for the design of adaptive kernels for tree-structured data that exploits the summarization properties of hidden states of hidden Markov models for trees. We introduce a compact and discriminative feature space based on the concept of hidden states multisets and we discuss different approaches to estimate such hidden state encoding. We show how it can be used to build an efficient and general tree kernel based on Jaccard similarity. Furthermore, we derive an unsupervised convolutional generative kernel using a topology induced on the Markov states by a tree topographic mapping. This paper provides an extensive empirical assessment on a variety of structured data learning tasks, comparing the predictive accuracy and computational efficiency of state-of-the-art generative, adaptive, and syntactical tree kernels. The results show that the proposed generative approach has a good tradeoff between computational complexity and predictive performance, in particular when considering the soft matching introduced by the topographic mapping. [ABSTRACT FROM AUTHOR]

Incremental learning with concept drift has often been tackled by ensemble methods, where models built in the past can be retrained to attain new models for the current data. Two design questions need to be addressed in developing ensemble methods for incremental learning with concept drift, i.e., which historical (i.e., previously trained) models should be preserved and how to utilize them. A novel ensemble learning method, namely, Diversity and Transfer-based Ensemble Learning (DTEL), is proposed in this paper. Given newly arrived data, DTEL uses each preserved historical model as an initial model and further trains it with the new data via transfer learning. Furthermore, DTEL preserves a diverse set of historical models, rather than a set of historical models that are merely accurate in terms of classification accuracy. Empirical studies on 15 synthetic data streams and 5 real-world data streams (all with concept drifts) demonstrate that DTEL can handle concept drift more effectively than 4 other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new end-to-end deep neural network model for time-series classification (TSC) with emphasis on both the accuracy and the interpretation. The proposed model contains a convolutional network component to extract high-level features and a recurrent network component to enhance the modeling of the temporal characteristics of TS data. In addition, a feedforward fully connected network with the sparse group lasso (SGL) regularization is used to generate the final classification. The proposed architecture not only achieves satisfying classification accuracy, but also obtains good interpretability through the SGL regularization. All these networks are connected and jointly trained in an end-to-end framework, and it can be generally applied to TSC tasks across different domains without the efforts of feature engineering. Our experiments in various TS data sets show that the proposed model outperforms the traditional convolutional neural network model for the classification accuracy, and also demonstrate how the SGL contributes to a better model interpretation. [ABSTRACT FROM AUTHOR]

The centers and radii of radial basis functions (RBFs) greatly affect the approximation capability of RBF networks (RBFNs). Traditional statistics-based approaches are widely used, but they may lack adaptivity to different data structures. Quantum clustering (QC), derived from quantum mechanics and the Schrödinger equation, demonstrates excellent capability in finding the structure and conformity toward data distribution. In this paper, a novel neural networks model called quantum local potential function networks (QLPFNs) is proposed. The QLPFN inherits the outstanding properties of QC by constructing the waves and the potential functions, and the level of data concentration can be discovered to obtain the inherent structures of the given data set. The local potential functions form the basic components of the QLPFN structure, which are automatically generated from the subsets of training data following specific subspace division procedures. Therefore, the QLPFN model in fact incorporates the level of data concentration as a computation technique, which is different from the classical RBFN model that exhibits radial symmetry toward specific centers. Some application examples are given in this paper to show the effectiveness of the QLPFN model. [ABSTRACT FROM AUTHOR]

Cluster validation, which is the process of evaluating the quality of clustering results, plays an important role for practical machine learning systems. Categorical sequences, such as biological sequences in computational biology, have become common in real-world applications. Different from previous studies, which mainly focused on attribute-value data, in this paper, we work on the cluster validation problem for categorical sequences. The evaluation of sequences clustering is currently difficult due to the lack of an internal validation criterion defined with regard to the structural features hidden in sequences. To solve this problem, in this paper, a novel cluster validity index (CVI) is proposed as a function of clustering, with the intracluster structural compactness and intercluster structural separation linearly combined to measure the quality of sequence clusters. A partition-based algorithm for robust clustering of categorical sequences is also proposed, which provides the new measure with high-quality clustering results by the deterministic initialization and the elimination of noise clusters using an information theoretic method. The new clustering algorithm and the CVI are then assembled within the common model selection procedure to determine the number of clusters in categorical sequence sets. A case study on commonly used protein sequences and the experimental results on some real-world sequence sets from different domains are given to demonstrate the performance of the proposed method. [ABSTRACT FROM PUBLISHER]

Checking validity of a model is a crucial step in the process of system identification. This is especially true when dealing with switched affine systems since, in this case, the problem of system identification from noisy data is known to be generically NP-Hard and can only be solved in practice by using heuristics and relaxations. Therefore, before the identified models can be used for instance for controller design, they should be systematically validated against additional experimental data. In this paper we address the problem of model (in)validation for multi-input multi-output switched affine systems in output error form with unknown switches. As a first step, we prove that necessary and sufficient invalidation certificates can be obtained by solving a sequence of convex optimization problems. In principle, these problems involve increasingly large matrices. However, as we show in the paper by exploiting recent results from semialgebraic geometry, the proposed algorithm is guaranteed to stop after a finite number of steps that can be be explicitly computed from the a priori information. In addition, this algorithm exploits the sparse structure of the underlying optimization problem to substantially reduce the computational burden. The effectiveness of the proposed method is illustrated using both academic examples and a non-trivial problem arising in computer vision: activity monitoring. [ABSTRACT FROM AUTHOR]

Video captioning, in essential, is a complex natural process, which is affected by various uncertainties stemming from video content, subjective judgment, and so on. In this paper, we build on the recent progress in using encoder-decoder framework for video captioning and address what we find to be a critical deficiency of the existing methods that most of the decoders propagate deterministic hidden states. Such complex uncertainty cannot be modeled efficiently by the deterministic models. In this paper, we propose a generative approach, referred to as multimodal stochastic recurrent neural networks (MS-RNNs), which models the uncertainty observed in the data using latent stochastic variables. Therefore, MS-RNN can improve the performance of video captioning and generate multiple sentences to describe a video considering different random factors. Specifically, a multimodal long short-term memory (LSTM) is first proposed to interact with both visual and textual features to capture a high-level representation. Then, a backward stochastic LSTM is proposed to support uncertainty propagation by introducing latent variables. Experimental results on the challenging data sets, microsoft video description and microsoft research video-to-text, show that our proposed MS-RNN approach outperforms the state-of-the-art video captioning benchmarks. [ABSTRACT FROM AUTHOR]

This paper presents a new time-varying long-term Synaptic Efficacy Function-based leaky-integrate-and-fire neuRON model, referred to as SEFRON and its supervised learning rule for pattern classification problems. The time-varying synaptic efficacy function is represented by a sum of amplitude modulated Gaussian distribution functions located at different times. For a given pattern, the SEFRON’s learning rule determines the changes in the amplitudes of weights at selected presynaptic spike times by minimizing a new error function reflecting the differences between the desired and actual postsynaptic firing times. Similar to the gamma-aminobutyric acid-switch phenomenon observed in a biological neuron that switches between excitatory and inhibitory postsynaptic potentials based on the physiological needs, the time-varying synapse model proposed in this paper allows the synaptic efficacy (weight) to switch signs in a continuous manner. The computational power and the functioning of SEFRON are first illustrated using a binary pattern classification problem. The detailed performance comparisons of a single SEFRON classifier with other spiking neural networks (SNNs) are also presented using four benchmark data sets from the UCI machine learning repository. The results clearly indicate that a single SEFRON provides a similar generalization performance compared to other SNNs with multiple layers and multiple neurons. [ABSTRACT FROM AUTHOR]

This paper deals with the quantized control problem for nonlinear semi-Markov jump systems subject to singular perturbation under a network-based framework. The nonlinearity of the system is well solved by applying Takagi–Sugeno (T-S) fuzzy theory. The semi-Markov jump process with the memory matrix of transition probability is introduced, for which the obtained results are more reasonable and less limiting. In addition, the packet dropouts governed by a Bernoulli variable and the signal quantization associated with a logarithmic quantizer are deeply studied. The major goal is to devise a fuzzy controller, which not only assures the mean-square $\bar { \sigma }$ -error stability of the corresponding system but also allows a higher upper bound of the singularly perturbed parameter. Sufficient conditions are developed to make sure that the applicable controller could be found. The further examination to demonstrate the feasibility of the presented method is given by designing a controller of a series DC motor model. [ABSTRACT FROM AUTHOR]

This paper addresses the duality between the deterministic feed-forward neural networks (FF-NNs) and linear Bayesian networks (LBNs), which are the generative stochastic models representing probability distributions over the visible data based on a linear function of a set of latent (hidden) variables. The maximum entropy principle is used to define a unique generative model corresponding to each FF-NN, called projected belief network (PBN). The FF-NN exactly recovers the hidden variables of the dual PBN. The large- $N$ asymptotic approximation to the PBN has the familiar structure of an LBN, with the addition of an invertible nonlinear transformation operating on the latent variables. It is shown that the exact nature of the PBN depends on the range of the input (visible) data details for the three cases of input data range are provided. The likelihood function of the PBN is straightforward to calculate, allowing it to be used as a generative classifier. An example is provided in which a generative classifier based on the PBN has comparable performance to a deep belief network in classifying handwritten characters. In addition, several examples are provided that demonstrate the duality relationship, for example, by training networks from either side of the duality. [ABSTRACT FROM AUTHOR]

Finite mixtures of skew distributions provide a flexible tool for modeling heterogeneous data with asymmetric distributional features. However, parameter estimation via the Expectation–Maximization (EM) algorithm can become very time consuming due to the complicated expressions involved in the E-step that are numerically expensive to evaluate. While parallelizing the EM algorithm can offer considerable speedup in time performance, current implementations focus almost exclusively on distributed platforms. In this paper, we consider instead the most typical operating environment for users of mixture models—a standalone multicore machine and the R programming environment. We develop a block implementation of the EM algorithm that facilitates the calculations on the E- and M-steps to be spread across a number of threads. We focus on the fitting of finite mixtures of multivariate skew normal and skew $t$ distributions, and show that both the E- and M-steps in the EM algorithm can be modified to allow the data to be split into blocks. Our approach is easy to implement and provides immediate benefits to users of multicore machines. Experiments were conducted on two real data sets to demonstrate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Venture capital market is one of the most important financial markets, which plays an important role in promoting industrial innovation and economic development. In this study, the complete data set of all venture capital events occurred in China from 2009 to 2020 was used to construct two kinds of complex networks, whose topological property and dynamic evolution trend of the network are studied. Specifically, we construct the co-investment network among investor, as well as the binary complex network of investor-entrepreneur, and propose the evaluation method of the importance and performance of venture capital institutions. Based on the proposed method, we identify some leading venture capital institutions with high performance and importance. Furthermore, this paper enlightens venture capital practitioners by discovering the investment behavior and preference of these leading institutions. [ABSTRACT FROM AUTHOR]

Modeling and prediction of temporal sequences is central to many signal processing and machine learning applications. Prediction based on sequence history is typically performed using parametric models, such as fixed-order Markov chains ( $n$ -grams), approximations of high-order Markov processes, such as mixed-order Markov models or mixtures of lagged bigram models, or with other machine learning techniques. This paper presents a method for sequence prediction based on sparse hyperdimensional coding of the sequence structure and describes how higher order temporal structures can be utilized in sparse coding in a balanced manner. The method is purely incremental, allowing real-time online learning and prediction with limited computational resources. Experiments with prediction of mobile phone use patterns, including the prediction of the next launched application, the next GPS location of the user, and the next artist played with the phone media player, reveal that the proposed method is able to capture the relevant variable-order structure from the sequences. In comparison with the $n$ -grams and the mixed-order Markov models, the sparse hyperdimensional predictor clearly outperforms its peers in terms of unweighted average recall and achieves an equal level of weighted average recall as the mixed-order Markov chain but without the batch training of the mixed-order model. [ABSTRACT FROM AUTHOR]

In this paper, a novel framework to address the problem of parametric estimation for continuous-time linear time-invariant dynamic systems is dealt with. The proposed methodology entails the design of suitable kernels of non-anticipative linear integral operators thus obtaining estimators showing, in the ideal case, “non-asymptotic” (i.e., “finite-time”) convergence. The analysis of the properties of the kernels guaranteeing such a convergence behaviour is addressed and a novel class of admissible kernel functions is introduced. The operators induced by the proposed kernels admit implementable (i.e., finite-dimensional and internally stable) state-space realizations. Extensive numerical results are reported to show the effectiveness of the proposed methodology. Comparisons with some existing continuous-time estimators are addressed as well and insights on the possible bias affecting the estimates are provided. [ABSTRACT FROM PUBLISHER]

Training of Deep Neural Network (DNN) at the edge faces the challenge of high energy consumption due to the requirements of a large number of memory accesses for gradient calculations. Therefore, it is necessary to minimize data fetches to perform training of a DNN model on the edge. In this paper, a novel technique has been proposed to reduce the memory access for the training of fully connected layers in transfer learning. By analyzing the memory access patterns in the backpropagation phase in fully connected layers, the memory access can be optimized. We introduce a new method to update the weights by introducing the delta term for every node of output and fully connected layer. Delta term aims to reduce memory access for the parameters which are required to access repeatedly during the training process of fully connected layers. The proposed technique shows 0.13x-13.93x energy savings for the training of fully connected layers for famous DNN architectures on multiple processor architectures. The proposed technique can be used to perform transfer learning on-chip to reduce energy consumption as well as memory access. [ABSTRACT FROM AUTHOR]

This paper considers the problem of selecting the top-m designs using simulation with input uncertainty. The performance of each design is measured by its worst case performance. The objective of this paper is to maximize the probability of correctly selecting the top-m designs given a fixed simulation budget. Due to the complexity of probability of correct selection (PCS), we develop a lower bound for the PCS and derive an asymptotically optimal budget allocation rule. Useful insights on characterizing the efficient budget allocation rule with input uncertainty are provided. Meanwhile, a sequential simulation procedure is suggested to implement the allocation rule. A series of numerical experiments indicate that the proposed simulation budget allocation rule can outperform all existing selection rules. [ABSTRACT FROM AUTHOR]

Multiview learning reveals the latent correlation among different modalities and utilizes the complementary information to achieve a better performance in many applications. In this paper, we propose a novel multiview learning model based on the Gaussian process latent variable model (GPLVM) to learn a set of nonlinear and nonparametric mapping functions and obtain a shared latent variable in the manifold space. Different from the previous work on the GPLVM, the proposed shared autoencoder Gaussian process (SAGP) latent variable model assumes that there is an additional mapping from the observed data to the shared manifold space. Due to the introduction of the autoencoder framework, both nonlinear projections from and to the observation are considered simultaneously. Additionally, instead of fully connecting used in the conventional autoencoder, the SAGP achieves the mappings utilizing the GP, which remarkably reduces the number of estimated parameters and avoids the phenomenon of overfitting. To make the proposed method adaptive for classification, a discriminative regularization is embedded into the proposed method. In the optimization process, an efficient algorithm based on the alternating direction method and gradient decent techniques is designed to solve the encoder and decoder parts alternatively. Experimental results on three real-world data sets substantiate the effectiveness and superiority of the proposed approach as compared with the state of the art. [ABSTRACT FROM AUTHOR]

This paper proposes a Bayesian nonparametric regression model of panel data for sequential pattern classification. The proposed method provides a flexible and parsimonious model that allows both time-independent spatial variables and time-dependent exogenous variables to be predictors. Not only this method improves the accuracy of parameter estimation for limited data, but also it facilitates model interpretation by identifying statistically significant predictors with hypothesis testing. Moreover, as the data length approaches infinity, posterior consistency of the model is guaranteed for general data-generating processes under regular conditions. The resulting model of panel data can also be used for sequential classification. The proposed method has been tested by numerical simulation, then validated on an econometric public data set, and subsequently validated for detection of combustion instabilities with experimental data that have been generated in a laboratory environment. [ABSTRACT FROM AUTHOR]

The vast majority of dimensionality reduction (DR) techniques rely on the second-order statistics to define their optimization objective. Even though this provides adequate results in most cases, it comes with several shortcomings. The methods require carefully designed regularizers and they are usually prone to outliers. In this paper, a new DR framework that can directly model the target distribution using the notion of similarity instead of distance is introduced. The proposed framework, called similarity embedding framework (SEF), can overcome the aforementioned limitations and provides a conceptually simpler way to express optimization targets similar to existing DR techniques. Deriving a new DR technique using the SEF becomes simply a matter of choosing an appropriate target similarity matrix. A variety of classical tasks, such as performing supervised DR and providing out-of-sample extensions, as well as, new novel techniques, such as providing fast linear embeddings for complex techniques, are demonstrated in this paper using the proposed framework. Six data sets from a diverse range of domains are used to evaluate the proposed method and it is demonstrated that it can outperform many existing DR techniques. [ABSTRACT FROM AUTHOR]

Analyzing data with latent spatial and/or temporal structure is a challenge for machine learning. In this paper, we propose a novel nonlinear model for studying data with latent dependence structure. It successfully combines the concepts of Markov random fields, transductive learning, and regression, making heavy use of the notion of joint feature maps. Our transductive conditional random field regression model is able to infer the latent states by combining limited labeled data of high precision with unlabeled data containing measurement uncertainty. In this manner, we can propagate accurate information and greatly reduce uncertainty. We demonstrate the usefulness of our novel framework on generated time series data with the known temporal structure and successfully validate it on synthetic as well as real-world offshore data with the spatial structure from the oil industry to predict rock porosities from acoustic impedance data. [ABSTRACT FROM AUTHOR]

Typical dimensionality reduction (DR) methods are data-oriented, focusing on directly reducing the number of random variables (or features) while retaining the maximal variations in the high-dimensional data. Targeting unsupervised situations, this paper aims to address the problem from a novel perspective and considers model-oriented DR in parameter spaces of binary multivariate distributions. Specifically, we propose a general parameter reduction criterion, called confident-information-first (CIF) principle, to maximally preserve confident parameters and rule out less confident ones. Formally, the confidence of each parameter can be assessed by its contribution to the expected Fisher information distance within a geometric manifold over the neighborhood of the underlying real distribution. Then, we demonstrate two implementations of CIF in different scenarios. First, when there are no observed samples, we revisit the Boltzmann machines (BMs) from a model selection perspective and theoretically show that both the fully visible BM and the BM with hidden units can be derived from the general binary multivariate distribution using the CIF principle. This finding would help us uncover and formalize the essential parts of the target density that BM aims to capture and the nonessential parts that BM should discard. Second, when there exist observed samples, we apply CIF to the model selection for BM, which is in turn made adaptive to the observed samples. The sample-specific CIF is a heuristic method to decide the priority order of parameters, which can improve the search efficiency without degrading the quality of model selection results as shown in a series of density estimation experiments. [ABSTRACT FROM AUTHOR]

In this paper, we extend the exponentially embedded family (EEF), a new approach to model order estimation and probability density function construction originally proposed by Kay in 2005, to multivariate pattern recognition. Specifically, a parametric classifier rule based on the EEF is developed, in which we construct a distribution for each class based on a reference distribution. The proposed method can address different types of classification problems in either a data-driven manner or a model-driven manner. In this paper, we demonstrate its effectiveness with examples of synthetic data classification and real-life data classification in a data-driven manner and the example of power quality disturbance classification in a model-driven manner. To evaluate the classification performance of our approach, the Monte-Carlo method is used in our experiments. The promising experimental results indicate many potential applications of the proposed method. [ABSTRACT FROM PUBLISHER]

Modern data acquisition routinely produces massive amounts of network data. Though many methods and models have been proposed to analyze such data, the research of network data is largely disconnected with the classical theory of statistical learning and signal processing. In this paper, we present a new framework for modeling network data, which connects two seemingly different areas: network data analysis and compressed sensing. From a nonparametric perspective, we model an observed network using a large dictionary. In particular, we consider the network clique detection problem and show connections between our formulation with a new algebraic tool, namely Randon basis pursuit in homogeneous spaces. Such a connection allows us to identify rigorous recovery conditions for clique detection problems. Though this paper is mainly conceptual, we also develop practical approximation algorithms for solving empirical problems and demonstrate their usefulness on real-world datasets. [ABSTRACT FROM AUTHOR]

Sparsified kernel adaptive filters (SKAFs) is an attractive filtering solution with low memory and computational complexity. Most of existing SKAFs are based on the mean square error (MSE) criterion under Gaussian noise assumption for its simplicity and convenience. When the assumption deviates largely from the underlying truth, the performance of these methods could degrade significantly. In this paper, we propose a novel SKAF, named as projected kernel least mean p-power algorithm (PKLMP), based on the mean p-power error (MPE) criterion and vector projection (VP) method. We provide convergence analyses in terms of the stead-state MSE, based on a Taylor expansion method, and derive the lower and upper bounds for the steady-state excess MSE. We also conduct mean convergence analysis for PKLMP, and derive convergence conditions. To exploit the information in the desired outputs, we further derive a modified PKLMP by smoothing the desired signal. Finally, a simple and effective online variable kernel centers strategy is proposed to improve the filtering performance of the proposed KAFs. Simulation results under a static function estimation, a chaotic time-series prediction, and two real-world time-series predictions are conducted and validate the effectiveness of the proposed PKLMP algorithms. [ABSTRACT FROM AUTHOR]

Piecewise affine (PWA) models are attractive frameworks that can represent various hybrid systems with local affine submodels and polyhedral regions due to their universal approximation properties. The PWA identification problem amounts to estimating both the submodel parameters and the polyhedral partitions from data. In this paper, we propose a novel approach to address the identification problem of PWA systems such that the number of submodels, parameters of submodels, and the polyhedral partitions are obtained. In particular, a cluster-based algorithm is designed to acquire the number of submodels, the initial labeled data set, and initial parameters corresponding to each submodel. Additionally, we develop a modified self-training support vector machine algorithm to simultaneously identify the hyperplanes and parameter of each submodel with the ouputs of the cluster-based algorithm. The proposed algorithm is computationally efficient for region estimation and able to accomplish this task with only a small quantity of classified regression vectors. The effectiveness of the proposed identification approach is illustrated via simulation results. [ABSTRACT FROM AUTHOR]

In this paper, $H_\infty $ control problem is investigated by off-policy integral reinforcement learning (IRL) method for the nonlinear systems with completely unknown dynamics, disturbances, and constrained-input. Firstly, according to a model-based policy iteration (PI) algorithm, a model-free algorithm is proposed based on the derived iterative equation, and the equivalence of model-based PI algorithm and model-free algorithm is proven. Then, the model-free algorithm is implemented by off-policy IRL technology to solve the Hamilton-Jacobi-Isaacs (HJI) equation with the collected system data by the least-square approach, where three neural networks (NNs) are constructed to approximate the value function, control and the disturbance. Finally, our proposed methods are applied to stabilize an autonomous third-order Chua’s chaotic circuit system and a non-autonomous second-order memristive chaotic circuit system to illustrate the efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, a novel data-driven predictive iterative learning control (DDPILC) scheme based on a radial basis function neural network (RBFNN) is proposed for a class of repeatable nonaffine nonlinear discrete-time systems subjected to nonrepetitive external disturbances. First, by utilizing the dynamic linearization technique (DLT) with a newly introduced and unknown system parameter pseudopartial derivative (PPD) and designing a new RBFNN estimation algorithm along the iterative learning axis for addressing the unknown PPD and the unknown nonrepetitive external disturbances, a data-driven prediction model is established. It is theoretically shown that by constructing a composite energy function (CEF) with respect to the modeling error for the first time, the convergence of the modeling error via the proposed DLT-based RBFNN modeling method can be guaranteed, and the convergence speed is tunable. Then, a DDPILC with a disturbance compensation term is designed, and the convergence of the tracking control error is analyzed. Finally, simulations of a train operation system reveal that even if the train suffers from randomly varying load disturbances and nonlinear running resistance, the proposed scheme can make both the modeling error and the tracking control error decrease successively with increasing operation number. [ABSTRACT FROM AUTHOR]

Federated learning is an emerging technique used to prevent the leakage of private information. Unlike centralized learning that needs to collect data from users and store them collectively on a cloud server, federated learning makes it possible to learn a global model while the data are distributed on the users’ devices. However, compared with the traditional centralized approach, the federated setting consumes considerable communication resources of the clients, which is indispensable for updating global models and prevents this technique from being widely used. In this paper, we aim to optimize the structure of the neural network models in federated learning using a multi-objective evolutionary algorithm to simultaneously minimize the communication costs and the global model test errors. A scalable method for encoding network connectivity is adapted to federated learning to enhance the efficiency in evolving deep neural networks. Experimental results on both multilayer perceptrons and convolutional neural networks indicate that the proposed optimization method is able to find optimized neural network models that can not only significantly reduce communication costs but also improve the learning performance of federated learning compared with the standard fully connected neural networks. [ABSTRACT FROM AUTHOR]

Intraclass compactness and interclass separability are crucial indicators to measure the effectiveness of a model to produce discriminative features, where intraclass compactness indicates how close the features with the same label are to each other and interclass separability indicates how far away the features with different labels are. In this paper, we investigate intraclass compactness and interclass separability of features learned by convolutional networks and propose a Gaussian-based softmax G-softmax) function that can effectively improve intraclass compactness and interclass separability. The proposed function is simple to implement and can easily replace the softmax function. We evaluate the proposed G-softmax function on classification data sets (i.e., CIFAR-10, CIFAR-100, and Tiny ImageNet) and on multilabel classification data sets (i.e., MS COCO and NUS-WIDE). The experimental results show that the proposed G-softmax function improves the state-of-the-art models across all evaluated data sets. In addition, the analysis of the intraclass compactness and interclass separability demonstrates the advantages of the proposed function over the softmax function, which is consistent with the performance improvement. More importantly, we observe that high intraclass compactness and interclass separability are linearly correlated with average precision on MS COCO and NUS-WIDE. This implies that the improvement of intraclass compactness and interclass separability would lead to the improvement of average precision. [ABSTRACT FROM AUTHOR]

This paper develops an effective distributed strategy for the solution of constrained multiagent stochastic optimization problems with coupled parameters across the agents. In this formulation, each agent is influenced by only a subset of the entries of a global parameter vector or model, and is subject to convex constraints that are only known locally. Problems of this type arise in several applications, most notably in disease propagation models, minimum-cost flow problems, distributed control formulations, and distributed power system monitoring. This paper focuses on stochastic settings, where a stochastic risk function is associated with each agent and the objective is to seek the minimizer of the aggregate sum of all risks subject to a set of constraints. Agents are not aware of the statistical distribution of the data and, therefore, can only rely on stochastic approximations in their learning strategies. We derive an effective distributed learning strategy that is able to track drifts in the underlying parameter model. A detailed performance and stability analysis is carried out showing that the resulting coupled diffusion strategy converges at a linear rate to an $O(\mu)$ neighborhood of the true penalized optimizer. [ABSTRACT FROM AUTHOR]

Semisupervised text classification has attracted much attention from the research community. In this paper, a novel model, the semisupervised sequential variational autoencoder (SSVAE), is proposed to tackle this problem. By treating the categorical label of unlabeled data as a discrete latent variable, the proposed model maximizes the variational evidence lower bound of the data likelihood, which implicitly derives the underlying label distribution for the unlabeled data. Analytical work indicates that the autoregressive nature of the sequential model is the crucial issue that renders the vanilla model ineffective. To remedy this, two types of decoders are investigated in the SSVAE model and verified. In addition, a reweighting approach is proposed to circumvent the credit assignment problem that occurs during the reconstruction procedure, which can further improve performance for sparse text data. Experimental results show that our method significantly improves the classification accuracy compared with other modern methods. [ABSTRACT FROM AUTHOR]

In many practical transfer learning scenarios, the feature distribution is different across the source and target domains (i.e., nonindependent identical distribution). Maximum mean discrepancy (MMD), as a domain discrepancy metric, has achieved promising performance in unsupervised domain adaptation (DA). We argue that the MMD-based DA methods ignore the data locality structure, which, up to some extent, would cause the negative transfer effect. The locality plays an important role in minimizing the nonlinear local domain discrepancy underlying the marginal distributions. For better exploiting the domain locality, a novel local generative discrepancy metric-based intermediate domain generation learning called Manifold Criterion guided Transfer Learning (MCTL) is proposed in this paper. The merits of the proposed MCTL are fourfold: 1) the concept of manifold criterion (MC) is first proposed as a measure validating the distribution matching across domains, and DA is achieved if the MC is satisfied; 2) the proposed MC can well guide the generation of the intermediate domain sharing similar distribution with the target domain, by minimizing the local domain discrepancy; 3) a global generative discrepancy metric is presented, such that both the global and local discrepancies can be effectively and positively reduced; and 4) a simplified version of MCTL called MCTL-S is presented under a perfect domain generation assumption for more generic learning scenario. Experiments on a number of benchmark visual transfer tasks demonstrate the superiority of the proposed MC guided generative transfer method, by comparing with the other state-of-the-art methods. The source code is available in https://github.com/wangshanshanCQU/MCTL. [ABSTRACT FROM AUTHOR]