Search

In this paper, we propose a distributed finite-time optimization protocol for single-integrator continuous-time multi-agent systems, which utilizes a modified Newton–Raphson method. The inverse of Hessian matrix, sign function and the gradient are adopted for the design of the algorithms. The proposed algorithms make agents converge to the network optimizer under any initial state and finite time, respectively. Lyapunov method and the properties of sign function are employed to verify the convergence of the proposed algorithms. Besides, the adaptive method is also considered to avoid complex parameter conditions and centralized parameters. Finally, numerical simulations are provided to testify the presented results. [ABSTRACT FROM AUTHOR]

Bioinformatics is a subject that studies life phenomena by using mathematical and information science theories and techniques. Its main tasks, such as DNA sequence comparison, protein structure prediction and cell metabolism analysis, can be regarded as complex optimization problems with different characteristics. Evolutionary computing is a kind of global optimization algorithm inspired by nature. Over the years, scholars have accumulated fruitful results in solving complex optimization problems such as large-scale, dynamic, multi-modal, multi-objective and multi-constrained problems by using Evolutionary Computation algorithms, and have successfully applied to the above optimization tasks in bioinformatics. This paper mainly summarizes the work of Evolutionary Computation technologies in bioinformatics from 2019 to 2023 at multiple levels, including Genomics, Proteomics, metabolomics and molecular networks related optimization tasks, as well as further applications in disease diagnosis and drug development. • Recent works on Evolutionary Computation technologies in Bioinformatics from 2019 to 2023. • Optimization tasks in Bioinformatics from genotype to phenotype. • Evolutionary Computation technologies from Genomics, Proteomics, Metabolomics and Molecular Networks level respectively. • Downstream applications such as disease diagnosis and drug discovery. [ABSTRACT FROM AUTHOR]

Natural Language Processing (NLP) models are known vulnerable to adversarial text attacks. Various word-level attacks have been proposed to modify input words by a pre-calculated word saliency order or the heuristic optimization algorithm. However, the predefined fixed order usually leads to a low attack success ratio, and the heuristic algorithm is often inefficient due to many model queries. In this paper, an Adaptive Gradient-based Word Saliency (AGWS) is proposed to improve the query-efficiency and simultaneously preserve a high attack success ratio for the word-level adversarial text attack. Firstly, the AGWS calculates the word saliency with a one-time gradient to all words instead of iterative queries, so the query-efficiency is enhanced. Secondly, the AGWS adaptively updates the word saliency via the variable neighborhood search algorithm to avoid the fixed modification order, which is significant in improving the attack success ratio. Additionally, the AGWS collect substitutes with a hybrid sememe and synonym strategy to enlarge adversary options. Extensive experiments and ablation studies manifest that the AGWS achieves the highest or second highest attack success ratio with lowest word perturbation percentage and improves the query-efficiency compared with baselines. Besides, the AGWS also shows superiorities in adversarial training and transferability. [ABSTRACT FROM AUTHOR]

Trust region policy optimization (TRPO) is one of the landmark policy optimization algorithms in deep reinforcement learning. Its purpose is to maximize a surrogate objective based on an advantage function, subject to the limited Kullback–Leibler (KL) divergence of two consecutive policies. Although there have been many successful applications of this algorithm in the literature, the approach has often been criticized for suppressing the exploration ability of some application environments due to its strict divergence constraint. As such, most researchers prefer to use entropy regularization, which is added to the expected discounted rewards or the surrogate objectives. That said, there is much debate about whether there might be an alternative strategy for regularizing TRPOs. In this paper, we present just that. Our strategy is to regularize the KL divergence-based constraint via Shannon entropy. This approach enlarges the difference between two consecutive policies and thus derives a new TRPO scheme with entropy regularization for use with KL divergence constraint. Next, the surrogate objective and Shannon entropy are approximated linearly, while the KL divergence is expanded quadratically. An efficient conjugate gradient optimization procedure then solves two sets of linear equations, providing a detailed code-level implementation that can be used for a fair experimental comparison. Extensive experiments within eight benchmark environments demonstrate that our proposed method is superior to both the original TRPO and the entropy regularized objective TRPO. Further, theoretical and experimental analysis shows that three TRPO-like methods have an equal time complexity and a close computational burden. [ABSTRACT FROM AUTHOR]

Combinatorial optimization problems (COPs) play an important role in both industrial production and theoretical research. As a classical class of optimization algorithms, evolutionary algorithms have shown good performance in COPs. However, as the size of the problem increases, it becomes difficult for evolutionary algorithms to produce satisfactory solutions within the specified time. In recent years, neural network algorithms have been applied to solve COPs and have shown superior effects. Inspired by this, this paper proposes a new algorithm based on graph structure for solving COPs. Through relaxation, COPs are transformed into their corresponding continuous optimization problems. The output state of our algorithm is converted to a discrete solution by approximation method. Compared with previous algorithms, our algorithm can not only solve COPs with general form such as binary quadratic programming (BQP), but also solve COPs with special form, which greatly expands the application scope of neural network algorithms. In addition, our algorithm can be embedded into other problem frameworks such as solving multi-objective combinatorial optimization problem (MOCOP) through scalarization method. A new penalty function is added to the objective function, which enables our algorithm to overcome the inconsistency between the optimal solution of the original problem and the corresponding continuous problem, so that our algorithm can perform well on various problems. Several numerical simulations are used to illustrate the effectiveness of our algorithm in solving COPs. Experimental results show that our algorithm can effectively solve COPs with general form and has a shorter solving time compared to traditional evolutionary algorithms. [ABSTRACT FROM AUTHOR]

The expensive optimization problem, widespread in real-world applications, poses great challenges to traditional evolutionary algorithms (EAs) due to its costly evaluations. A lot of investigations have proved that it is a practical way to incorporate surrogate models into EAs to reduce the number of time-consuming evaluations. Although various types of surrogate models from the community of machine learning, such as regression models that approximate the fitness values of candidate solutions or classification models that predict candidate solutions' category, have been widely applied separately, none of them consistently outperforms the other. Considering the intrinsic consistency of alleviating computational burden to improve the optimization efficiency, this paper introduces a two-layer architecture to integrate regression and classification models and synergize global and local evolution for high-dimensional expensive many-objective optimization problem (EMaOP). To implement this idea, the Kriging model based on incremental learning is utilized to explore the potential promising space. Further, multi local support vector machine (SVM) classifiers assisted improved differential evolution is run by turn with the global search to accelerate convergence. Accordingly, a novel surrogate-assisted EA, named Classification- and Regression-assisted evolutionary algorithm (CR-SAEA), is proposed on this basis. Experimental studies on several benchmark functions with up to 11 objectives and 200 dimensions confirm the effectiveness of the proposed algorithm and its competitiveness against state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

In this paper, a novel population robustness-based switching response framework (PR-SRF) is proposed to develop effective dynamic multi-objective optimization algorithm (DMOA), which integrates different response strategies to comprehensively cope with the dynamic behaviors. In particular, the population robustness is described by the quantification of how severely the environmental changes affect current population, which is timely graded as three levels of weak, strong, and normal to enable the adaptive switch of three different responses of diversity-enhancement, diversity-maintenance, and the knowledge-transfer, respectively. In this way, associations between the adopted responses and the changing environments are successfully established, thereby facilitating more intelligent decision when handling the dynamic behaviors. According to the benchmark evaluation results, the proposed PR-SRF-DMOA yields better comprehensive performance than several other DMOAs with popular response strategies, and it also outperforms another three DMOAs with hybrid responses, which demonstrates the great competitiveness of our algorithm. In addition, ablation study proves that the proposed PR-SRF can sufficiently exploit the merits of different responses, which effectively alleviates the negative knowledge transfer in extremely fluctuating environments, thereby providing valuable references for the development of evolutionary transfer optimization (ETO) algorithms. [ABSTRACT FROM AUTHOR]

Existing supervised Multi-Target Feature Selection (MTFS) methods seldom consider the nearest-neighbor relationship and statistical correlation of samples underlying the output space, which leads the result of feature selection to be easily interfered by the output noise, thus making it difficult to achieve satisfactory performance. This paper proposes a novel MTFS method to preserve both the global and local target correlations. Specifically, the low-rank constraint is introduced to achieve multi-layer regression structure to better decouple the inter-input and inter-target relationships. Moreover, the local nearest-neighbor relationships and variable correlations of the sample points in the output space are also explored through adaptive graph and manifold learning, to better utilize the target correlations to improve the MTFS performance. Following the above principle, the resulting objective function and the corresponding optimization algorithm are proposed. Extensive experiments on several public datasets show that the proposed method is superior to other state-of-the-art methods. • A new Embedded MTFS method is proposed to learn the feature redundancy and target correlations in MTR problems. • The nearest-neighbor relationships and target correlations are introduced to achieve superior performance in MTFS tasks. • Extensive experiments on several public datasets show that the proposed method is superior to the compared methods. [ABSTRACT FROM AUTHOR]

Adaptive operator selection is an online method that automatically adjusts the application rate of different operators based on their actual performance. This paper proposes an adaptive operator selection paradigm based on dueling deep Q-network (DDQN), aiming to improve the training efficiency of the Q-network for solving multi-objective optimization problems. The Q-network is decomposed into state value and action advantage networks, allowing the agent to learn state values more frequently and accurately. A novel state space and reward mechanism are designed to adapt to the characteristics of evolutionary multi-objective optimization. Combining adaptive operator selection with a multi-objective evolutionary algorithm with adaptive weights (AdaW), an AdaW-DDQN algorithm with high adaptability is proposed. Experimental results on three complex benchmark suites demonstrate that the proposed adaptive operator selection paradigm significantly improves the performance of multi-objective optimization algorithms. The AdaW-DDQN algorithm provides an effective and efficient solution for solving multi-objective optimization problems. • An adaptive operator selection paradigm based on dueling deep Q-network is proposed. • Ten effective crossover operators are designed as a pool of candidate operators. • A new state space and reward mechanism are designed for the multi-objective optimization problem. • The proposed adaptive operator selection paradigm is embedded into MOEA/D with adaptive weights. • Experimental results show that AdaW-DDQN has strong generalization ability. [ABSTRACT FROM AUTHOR]

• Teaching-learning-based optimization (TLBO) is one of population-based heuristic stochastic swarm intelligent algorithm. • In this paper, a comprehensive survey on the recent advances in TLBO is presented. • Literature survey reveals some interesting challenges and future research directions. Teaching-learning-based optimization (TLBO) algorithm which imitates the teaching–learning process in a classroom, is one of population-based heuristic stochastic swarm intelligent algorithms. TLBO executes through similar iterative evolution processes as utilized by a standard evolutionary algorithm. Unlike traditional evolutionary algorithms and swarm intelligent algorithms, the iterative computation process of teaching–learning-based optimization is divided into two phases and each phase executes iterative learning operation. In this paper, we present a comprehensive survey on the recent advances in TLBO. A review of the current literature reveals intriguing challenges and suggests potential future research directions. [ABSTRACT FROM AUTHOR]

Clustering is an unsupervised classical data processing technique, in which Fuzzy K-Means is extensively researched in practical application owing to its efficiency. However, common outliers in real-world always lead to clustering degradation. In this paper, we design an Enhanced Robust Fuzzy K-Means clustering method (ERFKM) joint ℓ 0 -norm sparse nearest neighbor constraint as a countermeasure to above challenge. On one hand, we adaptively weigh different samples by binary vector to measure each contribution for robustness, among which the outliers can be effectively recognized to eliminate their impact on clustering. That is, the weighted term is able to steadily correct the cumulative drift deviation of cluster centroid via ignoring extremely large sample-centroid residuals from outliers. On the other hand, the fuzzy regularizer with ℓ 0 -norm constraint aims to further enhance the segmentation ability of indicated normal samples through automatic cluster neighbor assignment. Finally, an optimization algorithm for our model is devised to simultaneously update the binary weight of each sample and sparse membership matrix for achieving robust clustering. Comprehensive experiments demonstrate the advantages and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Dynamic multi-objective optimization problems (DMOPs) involve several conflicting objectives, and these objective functions change over time. Therefore, addressing DMOPs necessitates an effective response to environmental changes. However, most existing algorithms only deal with DMOPs with one particular type of environmental changes, whereas real-world dynamic changes are more complicated. Therefore, this paper proposes a self-adaptive DMOEA based on transfer learning and elitism-based mutation (ATM-DMOEA), aiming to efficiently tackle DMOPs exhibiting complex environmental changes. Specifically, a change evaluation method is devised to gauge change intensity and discern whether a change is drastic or gentle. Subsequently, an adaptive change response strategy is implemented to accommodate varying environmental changes. For drastic changes, the algorithm employs an elitism-based manifold transfer learning method, while gentle changes are handled with a diversity enhancement strategy introduced by adaptive elitism-based mutations with a varying mutation probability. The experiments have validated the competitiveness of the proposed ATM-DMOEA on the majority of DMOP test instances with different levels of change severity. [ABSTRACT FROM AUTHOR]

With the increasing practicality of deep learning applications, practitioners are inevitably faced with datasets corrupted by noise from various sources such as measurement errors, mislabeling, and estimated surrogate inputs/outputs that can adversely impact the optimization results. It is a common practice to improve the optimization algorithm's robustness to noise, since this algorithm is ultimately in charge of updating the network parameters. Previous studies revealed that the first-order moment used in Adam-like stochastic gradient descent optimizers can be modified based on the Student's t-distribution. While this modification led to noise-resistant updates, the other associated statistics remained unchanged, resulting in inconsistencies in the assumed models. In this paper, we propose AdaTerm, a novel approach that incorporates the Student's t-distribution to derive not only the first-order moment but also all the associated statistics. This provides a unified treatment of the optimization process, offering a comprehensive framework under the statistical model of the t-distribution for the first time. The proposed approach offers several advantages over previously proposed approaches, including reduced hyperparameters and improved robustness and adaptability. AdaTerm achieves this by considering the interdependence of gradient dimensions. In particular, upon detection, AdaTerm excludes aberrant gradients from the update process and enhances its robustness for subsequent updates. Conversely, it performs normal parameter updates when the gradients are statistically valid, allowing for flexibility in adapting its robustness. This noise-adaptive behavior contributes to AdaTerm's exceptional learning performance, as demonstrated through various optimization problems with different and/or unknown noise ratios. Furthermore, we introduce a new technique for deriving a theoretical regret bound without relying on AMSGrad, providing a valuable contribution to the field. [ABSTRACT FROM AUTHOR]

Stochastic gradient descent (SGD), one of the most popular neural network optimization algorithms, has a solid theoretical foundation as well as good generalization performance. However vanilla SGD performs catastrophically in Binary Neural Networks (BNNs). Many studies have identified this phenomenon without explaining its causes in depth. In this paper, we try to experimentally understand the possible reasons for this and significantly improve the performance of vanilla SGD in BNNs training by adjusting the training strategy to be comparable to Adam. We subsequently propose a new optimization method for training deep neural networks (DNNs) with binary weights. In the proposed SGD with A daptive T hreshold, referred to as SGDAT, we suppress the frequency of weights flipping by thresholds and adjust the threshold of each parameter according to the number of flipping to further reduce the network noise, stabilize the network training, and improve the network generalization ability. Also, we present a complete ablation study of the hyperparameters space, as well as experimentally analyze the impact of using adaptive thresholds. Furthermore, we conduct image classification experiments over the CIFAR10, CIFAR100 and TinyImageNet datasets using BinaryNet and ResNet-18 network structure. The experiments show that SGDAT outperforms other binary optimizers. Code is available at: https://github.com/gushan/SGDAT. [ABSTRACT FROM AUTHOR]

[Display omitted] • Particle Swarm Optimization can be used to perform feature selection for domain adaptation. • It has been previously applied using classifiers to evaluate the goodness of subsets of features. • We explore a novel option: using complexity measures instead of classifiers. • We compare both methods in terms of performance, speed and size of the resulting set of features. Particle Swarm Optimization is an optimization algorithm that mimics the behaviour of a flock of birds, setting multiple particles that explore the search space guided by a fitness function in order to find the best possible solution. We apply the Sticky Binary Particle Swarm Optimization algorithm to perform feature selection for domain adaptation, a specific type of transfer learning in which the source and the target domain have a common feature space, a common task, but different distributions. When applying Particle Swarm Optimization, classification error is usually employed in the fitness function to evaluate the goodness of subsets of features. In this paper, we aim to compare this approach with using complexity metrics instead, under the assumption that reducing the complexity of the problem will lead to results that are independent from the classifier used for testing while being less computationally demanding. Therefore, we carried out experiments to compare the performance of both approaches in terms of classification accuracy, speed and number of features selected. We found out that our proposal, although in some cases incurs in a slight degradation of classification performance, it is indeed faster and selects fewer features, making it a feasible trade-off. [ABSTRACT FROM AUTHOR]

For optimization algorithms of fully complex-valued neural networks, complex-valued stepsize is helpful to make the training escape from saddle points. In this paper, an adaptive orthogonal gradient descent algorithm with complex-valued stepsize is proposed for the efficient training of fully complex-valued neural networks. The basic idea is that, at each iteration, the search direction is constructed as a combination of two orthogonal gradient directions by using the algebraic representation of complex-valued stepsize. It is then shown that the determination of suitable complex-valued stepsize is facilitated by a decoupling method such that the computational complexity involved in the training process is greatly reduced. The experiments are finally conducted on pattern classification, nonlinear channel equalization and signal prediction to confirm the advantages of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Least-squares one-class support vector machine (LS-OCSVM) is one of the most popular methods to perform one-class classification tasks, in which only the data of a specific class are available to train the classification model. However, the learning performance of LS-OCSVM heavily relies on the effectiveness of a squared loss function, which is sensitive to outliers, resulting in the poor robustness of LS-OCSVM to deal with contaminated data. In this paper, the original optimization problem of LS-OCSVM is therefore reformulated with a recently proposed robust similarity measure, called multikernel correntropy, generating a multikernel correntropy based LS-OCSVM (MKCLS-OCSVM). To find the solution to the new optimization problem effectively, a dynamic optimization algorithm developed with the popular half-quadratic optimization technique is adopted to perform the optimization process. Meanwhile, the convergence and computational complexity of the developed optimization algorithm are analyzed from theoretical perspectives. To further facilitate the implementation of MKCLS-OCSVM, an operationally simple search strategy, inspired by the hunting behavior of humpback whales, is designed for parameters selection. Experimental results on various one-class classification tasks are reported to demonstrate the performance superiority of the proposed MKCLS-OCSVM in comparison with LS-OCSVM and other robust LS-OCSVM variants. [ABSTRACT FROM AUTHOR]

An image is often corrupted by different kinds of noise during its acquisition and transmission. Conventional denoising methods can suppress the Gaussian noise effectively, but fail to maintain the quality of denoised images and may blur edges in an image. To address these short comings, this paper aims to develop an optimized adaptive thresholding function based framework for edge preserved satellite image denoising using different nature inspired algorithms which is capable of effectively removing the Gaussian noise from images without over smoothing edge details. Image denoising using adaptive thresholding functions selects the suitable threshold values to separate noise from the actual image without affecting the actual features of the image. In this approach, most widely used nature inspired optimization algorithms are exploited for learning the parameters of adaptive thresholding function required for optimum performance. It was found that the proposed adaptive differential evolution algorithm (JADE) algorithm based denoising approach has superior features and give better performance in terms of PSNR, MSE, SSIM and FSIM as compared to other methods. [ABSTRACT FROM AUTHOR]