Search

For multilayer perceptron (MLP), the initial weights will significantly influence its performance. Based on the enhanced fractional derivative extend from convex optimization, this paper proposes a fractional gradient descent (RFGD) algorithm robust to the initial weights of MLP. We analyze the effectiveness of the RFGD algorithm. The convergence of the RFGD algorithm is also analyzed. The computational complexity of the RFGD algorithm is generally larger than that of the gradient descent (GD) algorithm but smaller than that of the Adam, Padam, AdaBelief, and AdaDiff algorithms. Numerical experiments show that the RFGD algorithm has strong robustness to the order of fractional calculus which is the only added parameter compared to the GD algorithm. More importantly, compared to the GD, Adam, Padam, AdaBelief, and AdaDiff algorithms, the experimental results show that the RFGD algorithm has the best robust performance for the initial weights of MLP. Meanwhile, the correctness of the theoretical analysis is verified. [ABSTRACT FROM AUTHOR]

This brief proposes an enhanced fractional derivative that can prevent the tap weight coefficients from destroying the gradient information, solve the problem caused by the fractional extreme point, and improve the convergence speed with the help of error estimation information and Sign function. Based on this fractional derivative, an enhanced fractional least mean square (EFLMS) filter algorithm is proposed. We analyze the influence of unknown system vector on the convergence performance of the EFLMS algorithm. The computational complexity of the proposed algorithm is also analyzed. Simulation experiments show that the EFLMS algorithm achieves better performance in system identification than the classic least mean square (LMS) algorithm and the existing algorithms based on fractional calculus. [ABSTRACT FROM AUTHOR]