Search

We make two related contributions motivated by the challenge of training stochastic neural networks, particularly in a PAC–Bayesian setting: (1) we show how averaging over an ensemble of stochastic neural networks enables a new class of partially-aggregated estimators, proving that these lead to unbiased lower-variance output and gradient estimators, (2) we reformulate a PAC–Bayesian bound for signed-output networks to derive in combination with the above a directly optimisable, differentiable objective and a generalisation guarantee, without using a surrogate loss or loosening the bound. We show empirically that this leads to competitive generalisation guarantees and compares favourably to other methods for training such networks. Finally, we note that the above leads to a simpler PAC–Bayesian training scheme for sign-activation networks than previous work.

Asymmetric investors’ sentiments on returns and risks play an important role in updating the portfolio strategies in multi-period portfolio selection problems. By introducing the Prospect Theory to measure the asymmetric investors’ sentiments, a dynamic sentiment-adjusted model (DSAM) is proposed to sparse portfolio selection problem over multiple periods, in which the objective is to minimize the risk of the portfolio. As we focus on the sparse portfolio, a l 0 constraint is added to our model. The l 0 constraint represents that we can only purchase at most k securities from N candidate securities, in which k is a small number compared to N. Since the objective function of the sparse portfolio with l 0 constraint is NP-hard, and could not be solved by the Deep Learning algorithms. The stochastic neural networks algorithm with re-parametrisation trick (SNNrP) is introduced to solve the DSAM. The back-testing framework of our paper includes a multi-period portfolio selection model, in which asymmetric investors’ sentiments are modeled to iterate investors’ expected return level each period. In the back-testing framework, we conduct the experiments for different investment periods with different investors’ sentiments. The experimental results for the Nasdaq and CSI 300 data sets show that, on average, compared with the traditional Mean–variance model, the terminal return and risk obtained by the DSAM model outperforms by 9% and 11.75%.

This paper studies the issue of the exponential synchronization for coupled reaction-diffusion stochastic neural networks with time-varying delay (TVD). Two new delay-dependent impulsive pinning control mechanisms are presented, where distributed and discrete TVDs are both considered. Through utilizing the Lyapunov function approach, several sufficient criteria under the developed control strategies are established. Our results display that exponential synchronization of the coupled reaction-diffusion stochastic neural networks can be realized via controlling a small number of the network nodes with delayed impulses. Then, the effectiveness and feasibility of the proposed method are demonstrated by several numerical examples.

This paper is concerned with the periodic stochastic differential equations with nonlinear impulses. By using the properties of periodic Markov processes, the existence of periodic measures for the impulsive stochastic equations is established. As applications, we study the existence of periodic measures of impulsive periodic stochastic logistic equations and impulsive periodic stochastic neural networks, respectively.

Although high accuracies were achieved by artificial neural network (ANN), determining the optimal number of neurons in the hidden layer and the activation function is still an open issue. In this paper, the applicability of assigning the number of neurons in the hidden layer and the activation function randomly was investigated. Based on the findings, two novel versions of randomized ANNs, which are stochastic, and pruned stochastic, were proposed to achieve a higher accuracy without any time-consuming optimization stage. The proposed approaches were evaluated and validated by the basic versions of the popular randomized ANNs [1] are the random weight neural network [2], the random vector functional links [3] and the extreme learning machine [4] methods. In the stochastic version of randomized ANNs, not only the weights and biases of the neurons in the hidden layer but also the number of neurons in the hidden layer and each activation function were assigned randomly. In pruned stochastic version of these methods, the winner networks were pruned according to a novel strategy in order to produce a faster response. Proposed approaches were validated via 60 datasets (30 classification and 30 regression datasets). Obtained accuracies and time usages showed that both versions of randomized ANNs can be employed for classification and regression.

This paper is concerned with the stochastic stabilization for genetic regulatory networks. Based on the Lyapunov stability theory in combination with certain convex algorithm, we obtain the sufficient condition under which the unstable genetic regulatory network can be stabilized by using Brownian motion. Finally, a numerical illustrative example is provided to show the effectiveness and correctness of the proposed method.

An approach for the stochastic reconstruction of petroleum fractions based on the joint use of artificial neural networks and genetic algorithms was developed. This hybrid approach reduced the time required for optimization of the composition of the petroleum fraction without sacrificing accuracy. A reasonable initial structural parameter set in the optimization space was determined using an artificial neural network. Then, the initial parameter set was optimized using a genetic algorithm. The simulations show that the time savings were between 62 and 74% for the samples used. This development is critical, considering that the characteristic time required for the optimization procedure is hours or even days for stochastic reconstruction. In addition, the standalone use of the artificial neural network step that produces instantaneous results may help where it is necessary to make quick decisions.

Pseudoconvex optimization problem, as an important nonconvex optimization problem, plays an important role in scientific and engineering applications. In this paper, a recurrent one-layer neural network is proposed for solving the pseudoconvex optimization problem with equality and inequality constraints. It is proved that from any initial state, the state of the proposed neural network reaches the feasible region in finite time and stays there thereafter. It is also proved that the state of the proposed neural network is convergent to an optimal solution of the related problem. Compared with the related existing recurrent neural networks for the pseudoconvex optimization problems, the proposed neural network in this paper does not need the penalty parameters and has a better convergence. Meanwhile, the proposed neural network is used to solve three nonsmooth optimization problems, and we make some detailed comparisons with the known related conclusions. In the end, some numerical examples are provided to illustrate the effectiveness of the performance of the proposed neural network.

The delay-dependent state estimation problem for Takagi-Sugeno fuzzy stochastic neural networks with time-varying delays is considered in this paper. We aim to design state estimators to estimate the network states such that the dynamics of the estimation error systems are guaranteed to be exponentially stable in the mean square. Both fuzzy-rule-independent and the fuzzy-rule-dependent state estimators are designed. Delay-dependent sufficient conditions are presented to guarantee the existence of the desired state estimators for the fuzzy stochastic neural networks. Finally, two numerical examples demonstrate that the proposed approaches are effective.

In this paper, synchronization of stochastic reaction-diffusion neural networks with Dirichlet boundary conditions and unbounded discrete time-varying delays is investigated. By virtue of theories of partial differential equations, inequality methods, and stochastic analysis techniques, pth moment exponential synchronization and almost sure exponential synchronization of the underlying neural networks are developed. The obtained results in this study enhance and generalize some earlier ones. The effectiveness and merits of the theoretical criteria are substantiated by two numerical simulations.

In this study, the adaptive tracking control is investigated for a class of stochastic pure-feedback non-linear time-delay systems with output constraint and asymmetric input saturation non-linearity. First, the Gaussian error function is employed to represent a continuous differentiable asymmetric saturation model, and the barrier Lyapunov function is designed to cope with the output constraints. Then, the appropriate Lyapunov–Krasovskii functional and the property of hyperbolic tangent functions are used to address the effects of the unknown time-delay terms, and the neural network is employed to approximate the unknown non-linearities. At last, based on Lyapunov stability theory, a robust adaptive neural controller is proposed, which decreases the number of learning parameters and thus avoids the over-estimation problem. Under the designed neural controller, all the closed-loop signals are guaranteed to be 4-moment (or 2 moment) semi-globally uniformly ultimately bounded and the tracking error converges to a small neighbourhood of the origin for bounded initial conditions. Two simulation examples are presented to further illustrate the effectiveness of the designed method.

Stability evaluation of a weight-update system of higher-order neural units (HONUs) with polynomial aggregation of neural inputs (also known as classes of polynomial neural networks) for adaptation of both feedforward and recurrent HONUs by a gradient descent method is introduced. An essential core of the approach is based on spectral radius of a weight-update system, and it allows stability monitoring and its maintenance at every adaptation step individually. Assuring stability of the weight-update system (at every single adaptation step) naturally results in adaptation stability of the whole neural architecture that adapts to target data. As an aside, the used approach highlights the fact that the weight optimization of HONU is a linear problem, so the proposed approach can be generally extended to any neural architecture that is linear in its adaptable parameters., Comment: 2016, 13 pages

Summary This paper is concerned with the exponential ultimate boundedness problems for the impulsive stochastic delay difference systems. Several sufficient conditions on the global pth moment exponential ultimate boundedness are presented by using the Lyapunov methods and the algebraic inequality techniques, and the estimated exponential convergence rate and the ultimate bound are provided as well. As an application, the boundedness criteria are applied to a class of discrete impulsive stochastic neural networks with delays. The obtained results show that the impulses not only can stabilize an unstable stochastic difference delay system but also can make an unbounded stochastic difference delay system into a bounded system. Examples and simulations are also provided to demonstrate the effectiveness of the derived theoretical results.

Distributed cooperative control of multiple stochastic high-order nonlinear systems has not been addressed in literature. This paper presents an approach to design of distributed cooperative controllers for multiple stochastic high-order nonlinear systems under directed leader–followers type network topology via the so-called distributed integrator backstepping method. By using the algebraic graph theory and stochastic analysis method, it is shown that the output tracking errors between the followers and the leader can be tuned arbitrarily small while all the states of the closed-loop system remain bounded in probability. Finally, the effectiveness of the proposed control approach is illustrated on a stochastic underactuated mechanical system.

This paper investigates a kind of switched discrete-time neural network. Such neural network is composed of multiple sub-networks and switched different sub-networks according to the states of neural network. There is no common equilibrium for all of sub-networks, i.e., multiple equilibria coexist. Firstly, a bounded condition is presented for the switched discrete-time neural network. And then sufficient conditions are derived to ensure region stability of the equilibrium points of such neural network by mathematical analysis and nonsingular M-matrix theory. Four examples are presented to verify the validity of our results.

In this paper, the existence, uniqueness and asymptotic behavior of mild solutions of stochastic neural network systems driven by fractional Brownian motion are investigated. By applying the Banach fixed point theorem, the existence and uniqueness of mild solution are analytically proved in a Hilbert space. Based on the moment inequality of wick-type integral analysis technique, the p-th moment exponential convergence condition of the mild solution is presented. Finally, two numerical examples are presented to demonstrate the validity of the theoretical results.

Stochastic graph as a graph model for complex social networks.Four sampling algorithms for stochastic graphs in which edge weights are random variables.Analyze complex networks using stochastic network measures and sampling algorithms.Study the performance of the sampling algorithms on the stochastic networks. Recently, there has been growing interest in social network analysis. Graph models for social network analysis are usually assumed to be a deterministic graph with fixed weights for its edges or nodes. As activities of users in online social networks are changed with time, however, this assumption is too restrictive because of uncertainty, unpredictability and the time-varying nature of such real networks. The existing network measures and network sampling algorithms for complex social networks are designed basically for deterministic binary graphs with fixed weights. This results in loss of much of the information about the behavior of the network contained in its time-varying edge weights of network, such that is not an appropriate measure or sample for unveiling the important natural properties of the original network embedded in the varying edge weights. In this paper, we suggest that using stochastic graphs, in which weights associated with the edges are random variables, can be a suitable model for complex social network. Once the network model is chosen to be stochastic graphs, every aspect of the network such as path, clique, spanning tree, network measures and sampling algorithms should be treated stochastically. In particular, the network measures should be reformulated and new network sampling algorithms must be designed to reflect the stochastic nature of the network. In this paper, we first define some network measures for stochastic graphs, and then we propose four sampling algorithms based on learning automata for stochastic graphs. In order to study the performance of the proposed sampling algorithms, several experiments are conducted on real and synthetic stochastic graphs. The performances of these algorithms are studied in terms of Kolmogorov-Smirnov D statistics, relative error, Kendall's rank correlation coefficient and relative cost.

A boosting-based method of learning a feed-forward artificial neural network (ANN) with a single layer of hidden neurons and a single output neuron is presented. Initially, an algorithm called Boostron is described that learns a single-layer perceptron using AdaBoost and decision stumps. It is then extended to learn weights of a neural network with a single hidden layer of linear neurons. Finally, a novel method is introduced to incorporate non-linear activation functions in artificial neural network learning. The proposed method uses series representation to approximate non-linearity of activation functions, learns the coefficients of nonlinear terms by AdaBoost. It adapts the network parameters by a layer-wise iterative traversal of neurons and an appropriate reduction of the problem. A detailed performances comparison of various neural network models learned the proposed methods and those learned using the least mean squared learning (LMS) and the resilient back-propagation (RPROP) is provided in this paper. Several favorable results are reported for 17 synthetic and real-world datasets with different degrees of difficulties for both binary and multi-class problems.

This paper focuses on the global mean square exponential stability of stochastic neural networks with retarded and advanced argument. By employing the theory of differential equations with piecewise constant argument of generalized type, several sufficient conditions in form of algebraic inequalities are proposed to ensure the existence and uniqueness of solution. Considering that the piecewise alternately retarded and advanced argument exists, we estimate dynamic effect of system status in the current time and in the deviating function. Theoretical analysis of global mean square exponential stability is carried out by the stability theory of stochastic differential equations. Finally, numerical examples are exploited to illustrate the effectiveness of the results established.

A quaternionic extension of feed forward neural network, for processing multi-dimensional signals, is proposed in this paper. This neural network is based on the three layered network with random weights, called Extreme Learning Machines (ELMs), in which iterative least-mean-square algorithms are not required for training networks. All parameters and variables in the proposed network are encoded by quaternions and operations among them follow the quaternion algebra. Neurons in the proposed network are expected to operate multi-dimensional signals as single entities, rather than real-valued neurons deal with each element of signals independently. The performances for the proposed network are evaluated through two types of experiments: classifications and reconstructions for color images in the CIFAR-10 dataset. The experimental results show that the proposed networks are superior in terms of classification accuracies for input images than the conventional (real-valued) networks with similar degrees of freedom. The detailed investigations for operations in the proposed networks are conducted. HighlightsA feedforward neural network for accepting three- or four-dimension is proposed.Quaternions, which are a four-dimensional hypercomlex numbers, are used for encoding neuronal parameters.Its performances are evaluated through the classification and autoencoding for CIFAR-10 dataset.

This paper investigates the adaptive neural network optimal output feedback control design problem for nonlinear continuous-time systems with actuator saturation. The system dynamics and states of the controlled system are unknown. A neural network state observer is constructed to estimate the system states. This paper uses two neural networks, one is used to construct the neural network state observer, the other (critic neural network) is used to approximate the cost functions, which comprise an observer-critic architecture. In this architecture, the critic neural network weights are tuned based on both the current data and the previous data, thus the conditions of the persistent excitation in the previous literatures are relaxed. By utilizing adaptive dynamic programming approach, a new observer-based optimal control scheme is developed. It is proved that the proposed adaptive neural network output feedback optimal control scheme can ensure that the whole closed-loop system is stable. Moreover, the estimate errors of the critic neural network weights are asymptotically stable. A simulation example is given to validate the effectiveness of the proposed method.

This paper deals with the stochastic exponential synchronization problem of memristor-based neural networks with time-varying delay. Firstly, considering the state-dependent properties of the memristor, less conservative of model is constructed to analyze the complicated memristor-based neural networks. Then, by applying the stochastic differential inclusions theory and Lyapunov functional approach, sufficient verifiable conditions that depend on the time-varying delay and stochastic perturbation are obtained. It is shown that synchronization can be realized by linear feedback control and adaptive feedback control. The derived results complement and improve the previously known results. Finally, a numerical example is given to illustrate the effectiveness of the theoretical results.

This paper investigates the problem of distributed stochastic approximation in multiagent systems. The algorithm under study consists of two steps: A local stochastic approximation step and a diffusion step, which drives the network to a consensus. The diffusion step uses row-stochastic matrices to weight the network exchanges. As opposed to previous works, exchange matrices are not supposed to be doubly stochastic, and may also depend on the past estimate. We prove that nondoubly stochastic matrices generally influence the limit points of the algorithm. Nevertheless, the limit points are not affected by the choice of the matrices provided that the latter are doubly stochastic in expectation. This conclusion legitimates the use of broadcast-like diffusion protocols, which are easier to implement. Next, by means of a central limit theorem, we prove that doubly stochastic protocols perform asymptotically as well as centralized algorithms and we quantify the degradation caused by the use of nondoubly stochastic matrices. Throughout this paper, a special emphasis is put on the special case of distributed nonconvex optimization as an illustration of our results.

The random neural network (RNN) is a probabilitsic queueing theory-based model for artificial neural networks, and it requires the use of optimization algorithms for training. Commonly used gradient descent learning algorithms may reside in local minima, evolutionary algorithms can be also used to avoid local minima. Other techniques such as artificial bee colony (ABC), particle swarm optimization (PSO), and differential evolution algorithms also perform well in finding the global minimum but they converge slowly. The sequential quadratic programming (SQP) optimization algorithm can find the optimum neural network weights, but can also get stuck in local minima. We propose to overcome the shortcomings of these various approaches by using hybridized ABC/PSO and SQP. The resulting algorithm is shown to compare favorably with other known techniques for training the RNN. The results show that hybrid ABC learning with SQP outperforms other training algorithms in terms of mean-squared error and normalized root-mean-squared error.

This study is concerned with the output-feedback control problem for a class of stochastic high-order non-linear systems with time-varying delays. A distinctive feature of the control scheme is that the restrictions on delay-dependent drift and diffusion terms are greatly relaxed by using radial basis function neural network (NN) approximation approach. Furthermore, with the approach, the specific knowledge of NN nodes and weights is not required. Under some weaker conditions, by combining dynamic surface control technique with proper Lyapunov–Krasovskii functional, an adaptive NN output-feedback controller is designed constructively such that the closed-loop system is 4-moment (or mean square) semi-globally uniformly ultimately bounded. Finally, the control scheme is applied to both a practical stochastic robot system and a numerical system to demonstrate the effectiveness of the proposed approach.

In this paper, the passivity analysis problem is investigated for a class of discrete-time stochastic neural networks (DSNNs) with randomly occurring mixed time delays (ROMDs). The mixed delays comprise time-varying discrete delays, infinite-distributed delays as well as finite-distributed delays. A set of Bernoulli-distributed white sequences is used to account for the random nature of the occurrence of the mixed time delays. In addition, stochastic disturbances are taken into consideration to describe the state-dependent noises caused possibly by electronic devices and hardware implementation of neural networks. By using a combination of Lyapunov-Krasovskii functional, free-weighting matrix approach and stochastic analysis technique, we establish sufficient conditions guaranteeing the passivity performance of the underlying DSNNs. Furthermore, a delay-dependent robust passivity criterion is presented to deal with the parameter uncertainties in the DSNNs with ROMDs. A simulation example is provided to verify the effectiveness of the proposed approach.

It has been reported that training deep neural networks is more difficult than training shallow neural networks. Hinton et al. proposed deep belief networks with a learning algorithm that trains one layer at a time. A much better generalization can be achieved when pre-training each layer with an unsupervised learning algorithm. Since then, deep neural networks have been extensively studied. On the other hand, it has been revealed that singular points affect the training dynamics of the learning models such as neural networks and cause a standstill of training. Naturally, training deep neural networks suffer singular points. As described in this paper, we present a deep neural network model that has fewer singular points than the usual one. First, we demonstrate that some singular points in the deep real-valued neural network, which is equivalent to a deep complex-valued neural network, have been resolved as its inherent property. Such deep neural networks are less likely to become trapped in local minima or plateaus caused by critical points. Results of experiments on the two spirals problem, which has an extreme nonlinearity, support our theory. Copyright © 2017 John Wiley & Sons, Ltd.

In this paper, adaptive synchronization of stochastic memristor-based neural networks with mixed delays is investigated. By using the differential inclusions theory, adaptive control technique and stochastic Lyapunov method, two adaptive updated laws are designed and two synchronization criteria are derived for stochastic memristor-based neural networks with mixed delays. The derived criteria complement and improve the previously known results since stochastic perturbations and mixed delays are considered. Finally, two numerical examples are provided to illustrate the effectiveness of the theoretical results.

System identification problems are generally inverse vibration problems. Sometimes it is difficult to handle the inverse problems by traditional methods and classical artificial neural network. As such, the objective of this paper is to identify structural parameters by developing a novel functional link neural network (FLNN) model. FLNN model is more efficient than multi-layer neural network (MNN) as computation is less because hidden layer is not required. Here, single-layer neural network with multi-input and multi-output with feed-forward neural network model and principle of error back propagation has been used to identify structural parameters. The hidden layer is excluded by enlarging the input patterns with the help of Legendre and Hermite polynomials. Comparison of results among MNN, Legendre neural network, Hermite neural network and desired is considered and it is found that FLNN models are more effective than MNN.

This paper concerns with dynamical behaviors for a class of impulsive BAM neural networks with stochastic effects and mixed delays. By establishing integral-differential inequalities with time-varying inputs, we give the pth moment state estimation and obtain some sufficient conditions ensuring pth asymptotical input-to-state stability and pth exponential input-to-state stability with variable gains for the impulsive stochastic neural networks with delays. The present approach can remove some conservative and restrictive conditions on input-to-state stability given in existing publications and extend to more general stochastic delayed systems with nonlinear impulses.

Differences between the practical suspended-dome and the corresponding numerical model are inevitable. To reduce the existing discrepancy, model updating of a suspended-dome was investigated using the back-propagation network in the article. The article first proposed a method to increase the prediction precision of back-propagation network: reducing the range of the training data for the back-propagation network according to the previous prediction results continuously. Then, some parameters that can be measured are updated by the corresponding measured values directly, and other parameters that cannot be directly measured are updated by the corresponding prediction values from back-propagation network. The results indicate that the updated model can predict the experimental model perfectly, and back-propagation network is effective and accurate to predict the given parameters that cannot be described by an algorithm. The results also confirm that the proposed method to increase the prediction precision of back-propagation network is valid.

We introduce the problem of locating facilities over a finite time horizon with the goal of intercepting stochastic traffic flows that exhibit evasive behavior, which arises when locating weigh-in-motion systems, tollbooths, vehicle inspection stations, or other fixed flow-capturing facilities used for law enforcement. The problem can be formulated as a multi-stage, mixed-integer stochastic program; however, under certain independence assumptions, this can be reformulated as a large two-stage stochastic program, enabling us to solve much larger instances. We additionally propose an algorithm based on Lagrangian relaxation that separates the reformulated stochastic program into a variant of a deterministic knapsack problem and a sum of time-decoupled single-period stochastic programs that can be solved independently. The model and algorithm are tested on instances involving road networks of Nevada and Vermont. A comparison with the previously studied single-period stochastic programming approach shows that the newly proposed multi-period model substantially reduces the expected cost.

Recently, projection neural network (PNN) was proposed for solving monotone variational inequalities (VIs) and related convex optimization problems. In this paper, considering the inertial term into first order PNNs, an inertial PNN (IPNN) is also proposed for solving VIs. Under certain conditions, the IPNN is proved to be stable, and can be applied to solve a broader class of constrained optimization problems related to VIs. Compared with existing neural networks (NNs), the presence of the inertial term allows us to overcome some drawbacks of many NNs, which are constructed based on the steepest descent method, and this model is more convenient for exploring different Karush-Kuhn-Tucker optimal solution for nonconvex optimization problems. Finally, simulation results on three numerical examples show the effectiveness and performance of the proposed NN.

The finite-time stochastic synchronization of time-delay neural networks with noise disturbance is investigated according to finite-time stability theory of stochastic differential equation. Via constructing suitable Lyapunov function and controllers, finite-time stochastic synchronization is realized and sufficient conditions are derived. By analyzing the synchronization progress, factors affecting the convergence speed are given and feasible suggestions are proposed to improve the convergence rate. Finally, numerical simulations are given to verify the theoretical results.

We study coarse pattern formation in a cellular automaton modelling a spatially-extended stochastic neural network. The model, originally proposed by Gong and Robinson (Phys Rev E 85(5):055,101(R), 2012), is known to support stationary and travelling bumps of localised activity. We pose the model on a ring and study the existence and stability of these patterns in various limits using a combination of analytical and numerical techniques. In a purely deterministic version of the model, posed on a continuum, we construct bumps and travelling waves analytically using standard interface methods from neural field theory. In a stochastic version with Heaviside firing rate, we construct approximate analytical probability mass functions associated with bumps and travelling waves. In the full stochastic model posed on a discrete lattice, where a coarse analytic description is unavailable, we compute patterns and their linear stability using equation-free methods. The lifting procedure used in the coarse time-stepper is informed by the analysis in the deterministic and stochastic limits. In all settings, we identify the synaptic profile as a mesoscopic variable, and the width of the corresponding activity set as a macroscopic variable. Stationary and travelling bumps have similar meso- and macroscopic profiles, but different microscopic structure, hence we propose lifting operators which use microscopic motifs to disambiguate them. We provide numerical evidence that waves are supported by a combination of high synaptic gain and long refractory times, while meandering bumps are elicited by short refractory times.

For controlling an equilibrium of a nonlinear stochastic system, the problem of stabilization and synthesis with a required dispersion is studied. This problem is solved for the case where the feedback regulator uses noisy data. The new approach is based on an extension of the stochastic sensitivity synthesis method. Technically, this problem is reduced to the analysis of some quadratic matrix equations. A solution to the problem of minimizing the stochastic sensitivity is given. Details of such analysis are discussed for 2D and 3D nonlinear stochastic oscillators.

In this paper, we propose a novel framework for speeding up the test-time of traffic sign recognition, which is named Branch Convolution Neural Network. It is the first time to introduce a branch-output mechanism into a deep Convolution Neural Network. Our model has an accuracy as high as a deep convolution neural network model, while it performs faster at the same condition during test stage. It is a significantly accelerated framework for designing a real-time deep neural network system. We present a detail process to change a regular pre-trained Convolution Neural Network into a Branch Convolution Neural Network: train several simple branch classifiers, bias classifiers and optimize branches. Experiment applied on GTSRB shows that large number of traffic signs are unnecessary to go through all layers in a deep model and they can be separated out in a relative shallow neural network. This framework speeds up the recognition progress, while keeping the accuracy within an extremely minor drop.

This paper presents a dynamic fuzzy stochastic neural network model for nonparametric system identification using ambient vibration data. The model is developed to handle two types of imprecision in the sensed data: fuzzy information and measurement uncertainties. The dimension of the input vector is determined by using the false nearest neighbor approach. A Bayesian information criterion is applied to obtain the optimum number of stochastic neurons in the model. A fuzzy C-means clustering algorithm is employed as a data mining tool to divide the sensed data into clusters with common features. The fuzzy stochastic model is created by combining the fuzzy clusters of input vectors with the radial basis activation functions in the stochastic neural network. A natural gradient method is developed based on the Kullback–Leibler distance criterion for quick convergence of the model training. The model is validated using a power density pseudospectrum approach and a Bayesian hypothesis testing-based metric. The proposed methodology is investigated with numerically simulated data from a Markov Chain model and a two-story planar frame, and experimentally sensed data from ambient vibration data of a benchmark structure.