Search

Complex-valued neural network is a kind of learning model which can deal with problems in complex domain. Fully complex extreme learning machine (CELM) is a much faster training algorithm than the complex backpropagation (CBP) scheme. However, it is at the cost of using more hidden nodes to obtain the comparable performance. An upper-layer-solution-aware algorithm has been proposed for training single-hidden layer feedforward neural networks, which performs much better than its counterparts, pseudo-inverse learning (PIL)/extreme learning machine and gradient decent-based backpropagation neural networks. Consequently, there exist two challenges that need to be dealt with: 1) How to combine the advantages of CBP and CELM to develop a novel complex learning algorithm? and 2) What is the convergent behavior of the presented algorithm? In this article, an input weights dependent complex-valued (IWDCV) learning algorithm based on Wirtinger calculus has been proposed, which effectively solves the nonanalytic problem of the common activation functions during training neural networks. In addition, the monotonicity of the error function and the deterministic convergence of the proposed model have been strictly proved, which theoretically guarantee the efficiency and effectiveness of the given model, IWDCV. Finally, for real and complex-valued problems, a variety of simulations have been done to demonstrate the comparable performance of the proposed algorithm which support the theoretical observations as well. [ABSTRACT FROM AUTHOR]

Elman recurrent network is a representative model with feedback mechanism. Although gradient descent method has been widely used to train Elman network, it frequently leads to slow convergence. According to optimization theory, conjugate gradient method is an alternative strategy in searching the descent direction during training. In this paper, an efficient conjugate gradient method has been presented to reach the optimal solution in two ways: (1) constructing a more effective conjugate coefficient, (2) determining adaptive learning rates in terms of the generalized Armijo search method. Experiments show that the performance of the new algorithm is superior to traditional algorithms, such as gradient descent method and conjugate gradient method. In particular, the new algorithm has better performance than the evolutionary algorithm. Finally, we prove the weak and strong convergence of the presented algorithm, i.e., the gradient norm of the error function with respect to the weight vectors converges to zero and the weight sequence approaches a fixed optimal point. [ABSTRACT FROM AUTHOR]