Search

Conventional multiobjective optimization algorithms (MOEAs) with or without preferences are successful in solving multi- and many-objective optimization problems. However, a strong hypothesis underlying their performance is that MOEAs are able to find a representative solution set to cover the entire Pareto-optimal front (PF) and decision makers are able to conveniently and precisely articulate their preference, which is not always easy to fulfill in practice. Accordingly, it is suggested that representative solutions in the naturally interesting regions of the PF rather than the whole PF should be targeted. A large body of research has been proposed to search or identify the knees or knee regions over the past decades. Therefore, this article aims to provide a comprehensive survey of the research on knee-oriented optimization. We start with a discussion of the importance and basic concepts of the knees, followed by a summary of knee-oriented benchmarks and indicators. After that, knee-oriented frameworks and techniques, and real-world applications are presented. Finally, potential challenges are pointed out and a few promising future lines of research are suggested. The survey offers a new perspective to develop MOEAs for solving multi- and many-objective optimization problems. [ABSTRACT FROM AUTHOR]

In recent years, most of the studies have shown that the generalized iterated shrinkage thresholdings (GISTs) have become the commonly used first-order optimization algorithms in sparse learning problems. The nonconvex relaxations of the $\ell _{0}$ -norm usually achieve better performance than the convex case (e.g., $\ell _{1}$ -norm) since the former can achieve a nearly unbiased solver. To increase the calculation efficiency, this work further provides an accelerated GIST version, that is, AGIST, through the extrapolation-based acceleration technique, which can contribute to reduce the number of iterations when solving a family of nonconvex sparse learning problems. Besides, we present the algorithmic analysis, including both local and global convergence guarantees, as well as other intermediate results for the GIST and AGIST, denoted as (A)GIST, by virtue of the Kurdyka-Łojasiewica (KŁ) property and some milder assumptions. Numerical experiments on both synthetic data and real-world databases can demonstrate that the convergence results of objective function accord to the theoretical properties and nonconvex sparse learning methods can achieve superior performance over some convex ones. [ABSTRACT FROM AUTHOR]

The critical step of learning the robust regression model from high-dimensional visual data is how to characterize the error term. The existing methods mainly employ the nuclear norm to describe the error term, which are robust against structure noises (e.g., illumination changes and occlusions). Although the nuclear norm can describe the structure property of the error term, global distribution information is ignored in most of these methods. It is known that optimal transport (OT) is a robust distribution metric scheme due to that it can handle correspondences between different elements in the two distributions. Leveraging this property, this article presents a novel robust regression scheme by integrating OT with convex regularization. The OT-based regression with $L_{2} $ norm regularization (OTR) is first proposed to perform image classification. The alternating direction method of multipliers is developed to handle the model. To further address the occlusion problem in image classification, the extended OTR (EOTR) model is then presented by integrating the nuclear norm error term with an OTR model. In addition, we apply the alternating direction method of multipliers with Gaussian back substitution to solve EOTR and also provide the complexity and convergence analysis of our algorithms. Experiments were conducted on five benchmark datasets, including illumination changes and various occlusions. The experimental results demonstrate the performance of our robust regression model on biometric image classification against several state-of-the-art regression-based classification methods. [ABSTRACT FROM AUTHOR]

In recent years, low-rank matrix recovery problems have attracted much attention in computer vision and machine learning. The corresponding rank minimization problems are both combinational and NP-hard in general, which are mainly solved by both nuclear norm and Schatten-p ($0 < {p} < 1$) norm based optimization algorithms. However, inspired by weighted nuclear norm and Schatten-p norm as the relaxations of rank function, the main merits of this work firstly provide a modified Schatten-p norm in the affine matrix rank minimization problem, denoted as the modified Schatten-p norm minimization (MSpNM). Secondly, its surrogate function is constructed and the equivalence relationship with the MSpNM is further achieved. Thirdly, the iterative singular value thresholding algorithm (ISVTA) is devised to optimize it, and its accelerated version, i.e., AISVTA, is also obtained to reduce the number of iterations through the well-known Nesterov’s acceleration strategy. Most importantly, the convergence guarantees and their relationship with objective function, stationary point and variable sequence generated by the proposed algorithms are established under some specific assumptions, e.g., Kurdyka-Łojasiewicz (K) property. Finally, numerical experiments demonstrate the effectiveness of the proposed algorithms in the matrix completion problem for image inpainting and recommender systems. It should be noted that the accelerated algorithm has a much faster convergence speed and a very close recovery precision when comparing with the proposed non-accelerated one. [ABSTRACT FROM AUTHOR]

Recently, nuclear norm-based low rank representation (LRR) methods have been popular in several applications, such as subspace segmentation. However, there exist two limitations: one is that nuclear norm as the relaxation of rank function will lead to the suboptimal solution since nuclear norm-based minimization subproblem tends to the over-relaxations of singular value elements and treats each of them equally; the other is that solving LRR problems may cause more time consumption due to involving singular value decomposition of the large scale matrix at each iteration. To overcome both disadvantages, this paper mainly considers two tractable variants of LRR: one is Schatten- ${p}$ norm minimization-based LRR (i.e., ${S}_{p}$ NM_LRR) and the other is Schatten- ${p}$ norm factorization-based LRR (i.e., ${S}_{p}$ NF_LRR) for ${p=1}$ , 2/3 and 1/2. By introducing two or more auxiliary variables in the constraints, the alternating direction method of multiplier (ADMM) with multiple updating variables can be devised to solve these variants of LRR. Furthermore, both computational complexity and convergence property are given to evaluate nonconvex multiblocks ADMM algorithms. Several experiments finally validate the efficacy and efficiency of our methods on both synthetic data and real world data. [ABSTRACT FROM AUTHOR]

Face recognition (FR) via regression analysis-based classification has been widely studied in the past several years. Most existing regression analysis methods characterize the pixelwise representation error via l1 -norm or l2 -norm, which overlook the 2D structure of the error image. Recently, the nuclear norm-based matrix regression model is proposed to characterize low-rank structure of the error image. However, the nuclear norm cannot accurately describe the low-rank structural noise when the incoherence assumptions on the singular values does not hold, since it overpenalizes several much larger singular values. To address this problem, this paper presents the robust nuclear norm to characterize the structural error image and then extends it to deal with the mixed noise. The majorization–minimization (MM) method is applied to derive a iterative scheme for minimization of the robust nuclear norm optimization problem. Then, an efficiently alternating direction method of multipliers (ADMM) method is used to solve the proposed models. We use weighted nuclear norm as classification criterion to obtain the final recognition results. Experiments on several public face databases demonstrate the effectiveness of our models in handling with variations of structural noise (occlusion, illumination, and so on) and mixed noise. [ABSTRACT FROM PUBLISHER]