Search

• We propose fuzzy reinforced polynomial neural networks based on PNN architecture. • We propose fuzzy reinforced polynomial neurons (FRPNs) in the design of FRPNNs. • The proposed FRPNNs is a generalization of hybrid predictors (HPs). In the field of dynamic system identification and prediction, linear models (e.g., autoregressive models), nonlinear models (namely, neural networks models), and hybrid predictors (HPs) that are a hybridization of linear and nonlinear models have been proposed in the past. However, they are not completely free from limitations: they exhibit difficulties to describe high-order nonlinear relations between input and output variables. In this study, we propose fuzzy reinforced polynomial neural networks (FRPNNs), which are polynomial neural network architecture-based on fuzzy reinforced polynomial neurons (FRPNs) to overcome this limitation. The proposed FRPNs that consist of approximation part (AP) and compensation part (CP) arise as novel HPs. Here the CP for modeling nonlinear patterns can be regarded as forming the reinforced part for the AP that aims at capturing linear patterns, while AP is the linear polynomial neuron used in the conventional polynomial neural networks. In some sense, the overall FRPNNs are essentially generalized polynomial neural network architecture with novel HPs. The parameters considered in the design of the proposed fuzzy reinforced polynomial neural networks are optimized with the aid of the particle swarm optimization (PSO). The performance of FRPNNs is discussed involving time series and system identification datasets. Experimental results demonstrate that the proposed FRPNNs achieve at most the accuracy of 43.6% higher in comparison with the accuracy produced by some classical models reported in the literature. [ABSTRACT FROM AUTHOR]

• Deep neural networks (DNN) with polynomials activation functions, or polynomial activation neural networks (PANN), are introduced for the first time. Nonlinear difference equation modeling is contrived. Relations with common DNN are examined by Taylor series. • Stability analysis for feed-forward and recurrent PANN's are considered under small parametrization assumption, through trajectory bounds estimation and linear approximation. • The coverage back-propagation parameterization of PANN's is suggested, and pre-training algorithms with polynomial activation optimization or without are worked out, respectively. The pre-training algorithms are computationally efficient. • As application of PANN, nonlinear function fitting is numerically investigated. The results show high capability of PANN in functions fitting or features retrieving. In this paper, firstly, a class of deep (or multi-layer) neural networks with polynomial activation functions (or polynomial activation neural networks: PANN) are created, while feed-forward and recurrent architectures and their nonlinear difference modeling are explicated. PANN's relationships with conventional deep neural networks with sigmoid activation functions are discussed briefly by means of Taylor series. Secondly, numerical stability and stabilization of PANN's are examined, and the stability conditions are derived with bounded-state trajectory inequalities and small-state linear approximation, under small parametrization assumption; stability analysis implication coincides with what we already learnt about neural network pre-training. Thirdly, based on what we term the coverage back-propagation parametrization, pre-training algorithms for PANN with or without activation functions optimization are constructed; particularly, activation function optimization is a new concept of this study, which brings us with more learning flexibility in general neural networks. Finally, nonlinear function fitting is numerically illustrated to show application of PANN, which reveals high generalization capability of linear parameter-varying neural algorithms. [ABSTRACT FROM AUTHOR]