Search

For multilayer perceptron (MLP), the initial weights will significantly influence its performance. Based on the enhanced fractional derivative extend from convex optimization, this paper proposes a fractional gradient descent (RFGD) algorithm robust to the initial weights of MLP. We analyze the effectiveness of the RFGD algorithm. The convergence of the RFGD algorithm is also analyzed. The computational complexity of the RFGD algorithm is generally larger than that of the gradient descent (GD) algorithm but smaller than that of the Adam, Padam, AdaBelief, and AdaDiff algorithms. Numerical experiments show that the RFGD algorithm has strong robustness to the order of fractional calculus which is the only added parameter compared to the GD algorithm. More importantly, compared to the GD, Adam, Padam, AdaBelief, and AdaDiff algorithms, the experimental results show that the RFGD algorithm has the best robust performance for the initial weights of MLP. Meanwhile, the correctness of the theoretical analysis is verified. [ABSTRACT FROM AUTHOR]

Find the global optimal solution of the model is one promising research topic in computational intelligent community. Dependent on analogies to natural processes, the evolutionary swarm intelligent algorithms are widely used for solving global optimization problems which directed by the fitness values. In this paper, we propose one efficient fractional global learning machine (Fragmachine) which includes two stages (descending and ascending) to determine the optimal search path. The neural network model is used to approach the given fitness value. Specifically, for the descending stage, the integer gradient of the network output with respect the current location is employed to find the next descending point, while for the ascending stage, the fractional gradient is implemented to climb and escape from the local optimal point. We further propose one adaptive learning rate during training which relies on both the current gradient (integer or fractional) information and the fitness value. Finally, a series of numerical experiments verify the effectiveness of the proposed algorithm, Fragmachine. [ABSTRACT FROM AUTHOR]