Search

Deep neural network (DNN) has been widely studied due to its high performance and usability for various applications such as image classification, detection, segmentation, translation, and action recognition. Thanks to the universal applications and high performance of DNN algorithm, DNN is adopted for various AI platforms, including edge/mobile devices as well as cloud servers. However, high-performance DNN requires a large amount of computation and memory access, making it challenging to implement DNN operation on edge/mobile. There have been several ways to solve these problems, including algorithms as well as hardware for DNN. Algorithms that help accelerate DNN in hardware enable much more efficient operation of high-performance AI. This article aims to provide an overview of the recent hardware and algorithm co-design schemes enabling efficient processing of DNNs. Specifically, it will provide algorithm optimization methods for DNN structure, neurons, synapses, and data types. This paper also introduces optimization methods for hardware architectures, PE array, data-path control, and microarchitecture of PE. And we will also show examples of DNN algorithm and hardware co-designed ASICs. [ABSTRACT FROM AUTHOR]

In this paper, we consider the composition optimization with two expected-value functions in the form of $({1}/{n})\sum _{i = 1}^{n} F_{i}\left({({1}/{m})\sum _{j = 1}^{m} G_{j}(x)}\right)+R(x)$ , which formulates many important problems in statistical learning and machine learning such as solving Bellman equations in reinforcement learning and nonlinear embedding. Full gradient- or classical stochastic gradient descent-based optimization algorithms are unsuitable or computationally expensive to solve this problem due to the inner expectation $({1}/{m})\sum _{j = 1}^{m} G_{j}(x)$. We propose a dualityfree-based stochastic composition method that combines the variance reduction methods to address the stochastic composition problem. We apply the stochastic variance reduction gradient- and stochastic average gradient algorithm-based methods to estimate the inner function and the dualityfree method to estimate the outer function. We prove the linear convergence rate not only for the convex composition problem but also for the case that the individual outer functions are nonconvex, while the objective function is strongly convex. We also provide the results of experiments that show the effectiveness of our proposed methods. [ABSTRACT FROM AUTHOR]

The paper addresses analytical and practical aspects of integration of equation-based (classical) and signal-derived [artificial neural network (ANN)] dynamic models for transient analysis of large-scale dynamical systems. Our signal-based part is based on two ANNs, and is derived from measurements at boundary points. In this paper, we describe this hybrid modeling technique, and focus on: 1) a least square-based mechanism for on-line correction of dynamic variable predictions that is based on actual operating conditions; 2) the resilience of the algorithm to missing measurements due to failed communication links; and 3) a complete two-way interaction between the differential-algebraic equation based subsystem and the ANN-based subsystem. The paper demonstrates the feasibility of implementing our approach in standard power system software by integrating the ANN-based model with the transient analysis toolbox from Matlab. We illustrate capabilities of the proposed approach for transient analysis on a benchmark multi-machine example derived from the New England/New-York interconnected power system. [ABSTRACT FROM AUTHOR]

In this paper, novel call admission control (CAC) algorithms are developed based on cellular neural networks. These algorithms can achieve high network utilization by performing CAC in real-time, which is imperative in supporting quality of service (QoS) communication over packet-switched networks. The proposed solutions are of basic significance in access technology where a subscriber population (connected to the Internet via an access module) needs to receive services. In this case, QoS can only be preserved by admitting those user configurations which will not overload the access module. The paper treats CAC as a set separation problem where the separation surface is approximated based on a training set. This casts CAC as an image processing task in which a complex admission pattern is to be recognized from a couple of initial points belonging to the training set. Since CNNs can implement any propagation models to explore complex patterns, CAC can then be carried out by a CNN. The major challenge is to find the proper template matrix which yields high network utilization. On the other hand, the proposed method is also capable of handling three-dimensional separation surfaces, as in a typical access scenario there are three traffic classes (e.g., two type of Internet access and one voice over asymmetric digital subscriber line. [ABSTRACT FROM AUTHOR]

In this paper, a biologically inspired, CNN-based, multi-channel, texture boundary detection technique is presented. The proposed approach is similar to human vision system. The algorithm is simple and straightforward such that it can be implemented on the cellular neural networks (CNNs). CNN contains several important advantages, such as efficient real-time processing capability and feasible very large-scale integration (VLSI) implementation. The proposed algorithm also had been widely tested on synthetic texture images. Those texture images are randomly selected from the Brodatz textures database [1]. According to our simulation results, the boundaries of uniform textures can be detected quite successfully. For the nonuniform or nonregular textures, the results also indicate meaningful properties, and the properties also are consistent to the human visual sensation. The proposed algorithm also has been implemented on the CNN Universal Machine (CNN-UM), and yields similar results as the simulation on the PC. Based on the efficient performance of CNN-UM, the algorithm becomes very fast. [ABSTRACT FROM AUTHOR]

In this paper, discrete-time blind-source separation (BSS) of instantaneous mixtures is studied. Decorrelation-based sufficient criteria for BSS of stationary and nonstionary sources are derived based on nonstationarity and nonwhiteness. A gradient algorithm is proposed based on these criteria. A batch-data algorithm and an on-line algorithm are developed based on the corollaries of the BSS criteria. These algorithms are especially useful for the separation of nonstationary sources. They are robust to additive white noises if the time-delayed decorrelation and the nonstationarity of the sources are considered simultaneously in the algorithms. Experiment results show the effectiveness and performance of the proposed algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we continue to explore application of nonsmooth analysis to the study of global asymptotic robust stability (GARS) of delayed neural networks. In combination with Lyapunov theory, our approach gives several new types of sufficient conditions ensuring GARS. A significant common aspect of our results is their low computational complexity. It is demonstrated that the reported results can be verified either by conducting spectral decompositions of symmetric matrices associated with the uncertainty sets of network parameters, or by solving a semidefinite programming problem. Nontrivial examples are constructed to compare with some closely related existing results. [ABSTRACT FROM AUTHOR]

Blind source separation (BSS) aims at recovering statistically independent source signals from their linear mixtures without knowing the mixing coefficients. Besides independent component analysis, nonlinear principal component analysis (NPCA) is shown to be another useful tool for solving this problem, but it requires that the measured data be prewhitened. By taking into account the autocorrelation matrix of the measured data, we present in this paper a modified NPCA criterion, and develop a least-mean-square (LMS) algorithm and a recursive least-squares algorithm. They can perform the online BSS using directly the unwhitened observations. Since a natural gradient learning is applied and the prewhitening process is removed, the proposed algorithms work more efficiently than the existing NPCA algorithms, as verified by computer simulations on man-made sources as well as practical speech signals. [ABSTRACT FROM AUTHOR]

This paper proposes a new image-descreening technique based on texture classification using a cellular neural network (CNN) with template trained by genetic algorithm (GA), called GA-CNN. Instead of using the fixed filters for image descreening, we are equipped with a more pliable mechanism for classifications in screening patterns. Using CNN makes it possible to get an accurate texture classification result in a faster speed by its superiority of implementable hardware and the flexible choices of templates. The use of the GA here helps us to look for the most appropriate template for CNNs more adaptively and methodically. The evolved parameters in the template for CNNs can not only provide a quicker classification mechanism but also help us with a better texture classification for screening patterns. After the class of screening patterns in the querying images is determined by the trained GA-CNN-based texture classification system, the recommendatory filters are induced to solve the screening problems. The induction of the classification in screening patterns has simplified the choke of filters and made it valueless to determine a new structured filter. Eventually, our comprehensive methodology is going to be topped off with more desirable results and the indication for the decrease in time complexity. [ABSTRACT FROM AUTHOR]

The promising potential of cellular neural networks (CNN) has resulted in the development of several template design methods. The CNN universal machine (CNN-UM), a programmable CNN, has made it possible to create image-processing algorithms that run on this platform. However, very large-scale integration implementations of CNN-UMs presented parameter deviations that do not occur on ideal CNN structures. Consequently, new design methods were developed aiming at more robust templates. Although these new templates were indeed more robust, erroneous behavior can still be observed. An alternative for chip-independent robustness is chip-specific optimization, where the template is targeted to an individual chip. This paper describes a solution proposal in this sense to automatically tune templates in order to make the chip react as an ideal CNN structure. The approach uses measurements of actual CNN-UM chips as part of the cost function for a global optimization method to find an optimal template given an initial approximation. Further improvements are achieved by generating chip-specific robust templates by doing a search for the best template among the optimal ones. The tuned templates are therefore customized versions that are expected to be much less sensitive to imperfections on the operation of CNN-UM chips. Results are presented for the binary and grayscale cases, including the case of grayscale output. It is expected that as this technique matures, it will give CNN-UM chips enough reliability to compete with digital systems in terms of robustness in addition to advantages of speed. [ABSTRACT FROM AUTHOR]

In this paper, an on-chip implementation of the active contour technique called pixel-level snakes is proposed. This is based on an optimized cellular neural network (CNN) algorithm with capabilities to support changes in the contour topology. The entire algorithm has been implemented on a 64 × 64 CNN universal machine chip-set architecture for which the results of the time performance measurements are given. To illustrate the validity and capabilities of the proposed implementation some on-chip experiments are also included. [ABSTRACT FROM AUTHOR]

As the most important property of neural networks (NNs), the universal approximation capability of NNs is widely used in many applications. However, this property is generally proven for continuous systems. Most industrial systems are hybrid systems (e.g., piecewise continuous), which is a significant limitation for real applications. Recently, many identification methods have been proposed for hybrid system approximation; however, these methods only operate in linear hybrid systems. In this paper, the progressive learning machine—a new learning algorithm based on multi-NNs—is proposed for general hybrid nonlinear/linear system approximation. This algorithm classifies hybrid systems into several continuous systems and can approximate any hybrid system with zero output error. The performance of the proposed learning method is demonstrated via numerical examples and with experimental data from real applications. [ABSTRACT FROM AUTHOR]