Search

In this paper, we investigate the method of solving partial differential equations (PDEs) using artificial neural network (ANN) structures, which have been actively applied in artificial intelligence models. The ANN model for solving PDEs offers the advantage of providing explicit and continuous solutions. However, the ANN model for solving PDEs cannot construct a conventionally solvable linear system with known matrix solvers; thus, computational speed could be a significant concern. We study the implementation of the multigrid method, developing a general concept for a coarse-grid correction method to be integrated into the ANN-PDE architecture, with the goal of enhancing computational efficiency. By developing a reduced form of the multigrid method for ANN, we demonstrate that it can be interpreted as an equivalent representation of the classic multigrid expansion. We validated the applicability of the proposed method through rigorous experiments, which included analyzing loss decay and the number of iterations along with improvements in terms of accuracy, speed, and complexity. We accomplished this by employing the gradient descent method and the Broyden–Fletcher–Goldfarb–Shanno (BFGS) method to update the gradients while solving the given ANN systems of PDEs. [ABSTRACT FROM AUTHOR]

The BFGS method is a common and effective method for solving unconstrained optimization problems in quasi-Newton algorithm. However, many scholars have proven that the algorithm may fail in some cases for nonconvex problems under Wolfe conditions. In this paper, an adaptive projection BFGS algorithm is proposed naturally which can solve nonconvex problems, and the following properties are shown in this algorithm: ➀ The generation of the step size α j satisfies the popular Wolfe conditions; ➁ a specific condition is proposed which has sufficient descent property, and if the current point satisfies this condition, the ordinary BFGS iteration process proceeds as usual; ➂ otherwise, the next iteration point x j + 1 is generated by the proposed adaptive projection method. This algorithm is globally convergent for nonconvex problems under the weak-Wolfe-Powell (WWP) conditions and has a superlinear convergence rate, which can be regarded as an extension of projection BFGS method proposed by Yuan et al. (J. Comput. Appl. Math. 327:274-294, 2018). Furthermore, the final numerical results and the application of the algorithm to the Muskingum model demonstrate the feasibility and competitiveness of the algorithm. [ABSTRACT FROM AUTHOR]

In this article, we propose an adaptation of the Hyperbolic Augmented Lagrangian algorithm to solve multiobjective optimization problems. After conducting a literature review, we provide a definition of the Hyperbolic Augmented Lagrange method. Under clearly defined assumptions, we demonstrate that any limit point of a sequence generated by the proposed approach is feasible and constitutes a Pareto-optimal solution. To assess the effectiveness of the proposed algorithm, we incorporate two variants of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method, thereby offering two approaches for solving well-known tests problems from the literature. We employ metrics to evaluate the performance of these two approaches. [ABSTRACT FROM AUTHOR]

We present two quasi-Newton algorithms for solving unconstrained optimization problems based on two modified secant relations to get reliable approximations of the Hessian matrices of the objective function. The proposed methods make use of both gradient and function values, and utilize information from the two most recent steps in contrast to the the usual secant relation using only the latest step. We show that the modified BFGS methods based on the new secant relations are globally convergent and have a local superlinear rate of convergence. Computational experiments are made on problems from the CUTEst library. Comparative numerical results show competitiveness of the proposed methods in the sense of the Dolan–Moré performance profiles. [ABSTRACT FROM AUTHOR]

It is known that many tensor data have symmetric or partially symmetric structure and structural tensors have structure preserving Candecomp/Parafac (CP) decompositions. However, the well-known alternating least squares (ALS) method cannot realize structure preserving CP decompositions of tensors. Hence, in this paper, we consider numerical problems of structure preserving rank-R approximation and structure preserving CP decomposition of partially symmetric tensors. For the problem of structure preserving rank-R approximation, we derive the gradient formula of the objective function, obtain BFGS iterative formulas, propose a BFGS algorithm for positive partially symmetric rank-R approximation, and discuss the convergence of the algorithm. For the problem of structure preserving CP decomposition, we give a necessary condition for partially symmetric tensors with even orders to have positive partially symmetric CP decompositions, and design a general partially symmetric rank-R approximation algorithm. Finally, some numerical examples are given. Through numerical examples, we find that if a tensor has a positive partially symmetric CP decomposition then its partially symmetric rank CP decomposition must be a positive CP decomposition. In addition, we compare the BFGS algorithm proposed in this paper with the standard CP-ALS method. Numerical examples show that the BFGS algorithm has better stability and faster computing speed than CP-ALS algorithm. [ABSTRACT FROM AUTHOR]

This paper presents a modification of the q-BFGS method for nonlinear unconstrained optimization problems. For this modification, we use a simple symmetric positive definite matrix and propose a new q-quasi-Newton equation, which is close to the ordinary q-quasi-Newton equation in the limiting case. This method uses only first order q-derivatives to build an approximate q-Hessian over a number of iterations. The q-Armijo-Wolfe line search condition is used to calculate step length, which guarantees that the objective function value is decreasing. This modified q-BFGS method preserves the global convergence properties of the q-BFGS method, without the convexity assumption on the objective function. Numerical results on some test problems are presented, which show that an improvement has been achieved. Moreover, we depict the numerical results through the performance profiles. [ABSTRACT FROM AUTHOR]

This work implements the BFGS (Broyden-Fletcher-Goldfarb-Shanno) optimization method for training the type-1 and singleton fuzzy logic system applied to solve binary classification problems. The BFGS is a quasi-Newton method that approximates the second-order information using the gradient of the cost function. Additionally, the Golden Section method is used to obtain the step size for each line search in a descent direction. The effectiveness of the proposed method is demonstrated by using well-established classification metrics evaluated in popular datasets from the literature. Comparisons between the proposed approach and well-known gradient-based methods available are also provided, showing that the BFGS achieves improved performance in terms of accuracy, mean squared error, and the number of epoch demanded during the training phase. [ABSTRACT FROM AUTHOR]

The BFGS method, which has great numerical stability, is one of the quasi-Newton line search methods. However, the global convergence of the BFGS method with a Wolfe line search is still an open problem for general functions. Recently, Yuan et al. (Appl. Math. Mode. 47:811–825, 2017) presented a modified weak Wolfe-Powell (WWP) line search that globally converges for general functions; however, they only partially solved this open problem. In this paper, a further modified WWP line search is proposed, and the global convergence of the BFGS method is established under suitable conditions for general functions. The numerical results show that the new line search produces more interesting results than the normal line search. Moreover, a fact engineering problem is studied with the Muskingum model to show the performance of the presented algorithm. [ABSTRACT FROM AUTHOR]

We propose a BFGS method with Wolfe line searches for unconstrained multiobjective optimization problems. The algorithm is well defined even for general nonconvex problems. Global convergence and R-linear convergence to a Pareto optimal point are established for strongly convex problems. In the local convergence analysis, if the objective functions are locally strongly convex with Lipschitz continuous Hessians, the rate of convergence is Q-superlinear. In this respect, our method exactly mimics the classical BFGS method for single-criterion optimization. [ABSTRACT FROM AUTHOR]

Prediction of pollution in water flow is a very important task. To this end, it is imperative to be able to define the uncertainty in a model prediction. This is the purpose of sensitivity analysis whose role is to identify what uncertainty in the model outputs is attributable to the model inputs (parameters in this case). Traditionally, this is achieved by running the model perturbed by many random samples in the parameter space to determine their impact on the model outputs. It provides information on how much of the output variance is controlled by each parameter of the inputs. The theoretical results related to the procedure based adjoint approach for computing a sensitivity of the response function (RF) to changes in the input source are presented in the paper. It is shown that this approach allows to compute, by one single integration of the adjoint equation over a given time interval, a sensitivity of the RF to any source located in the domain of interest. The proposed approach is applied to the 2D Saint-Venant flow equations for modelling the water pollution problem. A numerical experiment is formulated and implemented for the Thanh Nhan Lake in Hanoi for studying a sensitivity of some RF to observations. The numerical model is constructed by applying the well-known finite-volume method. Two appropriate optimization problems are introduced and solved on the basis of the BFGS algorithm. The numerical results show the efficiency of the proposed method and confirm the theoretical findings. [ABSTRACT FROM AUTHOR]

In this paper, we investigate a class of interval-valued multiobjective optimization problems (IVMOP), and formulate a certain equivalent multiobjective optimization problem (MOP). We establish that any Pareto optimal solution of the equivalent MOP is also an effective solution of IVMOP. We introduce two variants of the quasi-Newton method to solve IVMOP when every component of the objective function has second-order partial generalized Hukuhara derivatives. In the first variant, we assume that the objective function of IVMOP is strongly convex, and approximate Hessian matrix of each component of the objective function of MOP by using the well-known BFGS method. The algorithm generated by this variant exhibits superlinear convergence to a locally effective solution of IVMOP. In the second variant, we approximate the Hessian matrix of each component of MOP by applying the Levenberg–Marquardt method. Moreover, this variant can be applied to non-convex functions, and the sequence generated by this variant converges to a locally effective solution of IVMOP as well. Finally, some numerical examples are furnished to illustrate the effectiveness of our developed methodology. [ABSTRACT FROM AUTHOR]