Search

This paper considers a stochastic optimization problem over the fixed point sets of quasinonexpansive mappings on Riemannian manifolds. The problem enables us to consider Riemannian hierarchical optimization problems over complicated sets, such as the intersection of many closed convex sets, the set of all minimizers of a nonsmooth convex function, and the intersection of sublevel sets of nonsmooth convex functions. We focus on adaptive learning rate optimization algorithms, which adapt step-sizes (referred to as learning rates in the machine learning field) to find optimal solutions quickly. We then propose a Riemannian stochastic fixed point optimization algorithm, which combines fixed point approximation methods on Riemannian manifolds with the adaptive learning rate optimization algorithms. We also give convergence analyses of the proposed algorithm for nonsmooth convex and smooth nonconvex optimization. The analysis results indicate that, with small constant step-sizes, the proposed algorithm approximates a solution to the problem. Consideration of the case in which step-size sequences are diminishing demonstrates that the proposed algorithm solves the problem with a guaranteed convergence rate. This paper also provides numerical comparisons that demonstrate the effectiveness of the proposed algorithms with formulas based on the adaptive learning rate optimization algorithms, such as Adam and AMSGrad. [ABSTRACT FROM AUTHOR]

Because of the low accuracy in intrusion detection, a model of extreme learning machine based on the optimization of quantum beetle swarm algorithm is proposed. First of all, this paper proposes a quantum beetle swarm optimization algorithm, which introduces quantum mechanics and combines the advantages of beetle antennae search and particle swarm optimization. In this way, the individual can learn both their own experience and group experience, which enables the beetle to move purposefully and instructively, and improves the convergence performance of the algorithm. In extreme learning machine, it is more difficult to solve the problem in high-dimensional data. This paper proposed an improved extreme learning machine that uses the least squares QR algorithm to decompose the matrix, which can reduce the computational complexity of the traditional extreme learning machine. The improved extreme learning machine model optimized by quantum beetle swarm optimization algorithm is applied to intrusion detection, and the simulation results show that the model proposed in this paper can significantly improve detection accuracy and increase convergence rate. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have evolved as a beneficial machine learning method that has been successfully used in various applications. Currently, DNN is a superior technique of extracting information from massive sets of data in a self-organized method. DNNs have different structures and parameters, which are usually produced for particular applications. Nevertheless, the training procedures of DNNs can be protracted depending on the given application and the size of the training set. Further, determining the most precise and practical structure of a deep learning method in a reasonable time is a possible problem related to this procedure. Meta-heuristics techniques, such as swarm intelligence (SI) and evolutionary computing (EC), represent optimization frames with specific theories and objective functions. These methods are adjustable and have been demonstrated their effectiveness in various applications; hence, they can optimize the DNNs models. This paper presents a comprehensive survey of the recent optimization methods (i.e., SI and EC) employed to enhance DNNs performance on various tasks. This paper also analyzes the importance of optimization methods in generating the optimal hyper-parameters and structures of DNNs in taking into consideration massive-scale data. Finally, several potential directions that still need improvements and open problems in evolutionary DNNs are identified. [ABSTRACT FROM AUTHOR]

This review paper presents a comprehensive and full review of the so-called optimization algorithm, multi-verse optimizer algorithm (MOA), and reviews its main characteristics and procedures. This optimizer is a kind of the most recent powerful nature-inspired meta-heuristic algorithms, where it has been successfully implemented and utilized in several optimization problems in a variety of several fields, which are covered in this context, such as benchmark test functions, machine learning applications, engineering applications, network applications, parameters control, and other applications of MOA. This paper covers all the available publications that have been used MOA in its application, which are published in the literature including the variants of MOA such as binary, modifications, hybridizations, chaotic, and multi-objective. Followed by its applications, the assessment and evaluation, and finally the conclusions, which interested in the current works on the optimization algorithm, recommend potential future research directions. [ABSTRACT FROM AUTHOR]

Local feature descriptor plays a significant role in texture classification. However, in the traditional local binary pattern method, image pixels are converted into a binary pattern based on the relationship between center and neighborhood pixels. This paper introduces a novel feature extraction method named local neighbourhood edge responsive binary pattern to extract and categorize reliable texture features from images for achieving an automated classification of different types of ocean bottom sediments. Initially, the basic magnitude variance pixel values are derived depending on an odd and even pixel value of a 3 × 3 image patch. Further, the edge information is extracted using the local directional pattern method from all images. The edge response of the image is obtained using a kirsch mask in all eight directions. The encoding condition is then applied to both the local intensity and the edge information to create a unique descriptor value. Finally, a new learning algorithm called GMJAYA-ELM combines the Gaussian mutated JAYA (GMJAYA) with an extreme learning machine (ELM) for texture classification. The GMJAYA is used to optimize the input weights and hidden biases of single-hidden-layer feed-forward neural networks. Experimental findings indicate that the suggested method yields better accuracy and sensitivity efficiency among various groups. The proposed algorithm is tested by contrasting outcomes with standard learning schemes such as PSO-ELM, GA-ELM, ABC-ELM, Birdswarm-ELM, and JAYA-ELM, suggesting the dominance of GMJAYA-ELM. [ABSTRACT FROM AUTHOR]

Modeling the rate of penetration (ROP) plays a fundamental role in drilling optimization since the achievement of an optimum ROP can drastically reduce the overall cost of drilling activities. Evolved Extreme learning machine (ELM) with the evolutionary algorithms and multi-layer perceptron with Levenberg-Marquardt training algorithm (MLP-LMA) were proposed in this study to predict ROP. This paper focused mainly on two aspects. The first one was the investigation of the whale optimization algorithm (WOA) to optimize the weights and biases between input and hidden layers of ELM to enhance its prediction accuracy. The other was to adopt a prediction methodology that seeks to update the predictive model at each formation in order to reduce the dimension of input data and mitigate the effect of non real-time data such as the formation properties on the bit speed prediction. The prediction models were trained and tested using 3561 data points gathered from an Algerian field. The statistical and graphical evaluation criteria show that the ELM-WOA exhibited higher accuracy and generalization performance compared with the ELM-PSO and MLP-LMA. Furthermore, ELM-WOA was compared with two well-known ROP correlations in the literature, and the comparison results reveal that the proposed ELM-WOA model is superior to the pre-existing correlations. The findings of this study can help for the achievement of an optimum ROP and the reduction of the non-productive time. In addition, the outputs of this study can be used as an objective function during the real-time optimization of the drilling operation. [ABSTRACT FROM AUTHOR]

Support vector machine (SVM) is a widely used and reliable machine learning algorithm. It has been successfully applied to many real-world problems, with remarkable results. However, it has also been observed that SVM computational complexity increases with the increase in data size. Although many SVM speed optimization techniques have been proposed in the literature, there is still need for further improvement on the performance speed and accuracy of this algorithm. In this paper, a boundary detection algorithm for SVM speed optimization called ant colony optimization instance selection algorithm (ACOISA) is proposed. ACOISA is inspired by edge selection in ant colony optimization (ACO) algorithm, and it performs two primary functions: boundary detection and boundary instance selection. In the algorithm, ACO is used for boundary detection and k-nearest neighbor algorithm is used for boundary instance selection. Different sets of experiments are carried out to validate the efficiency of the proposed technique. All the experiments were performed on 35 datasets containing three well-known e-fraud types (credit card fraud, email spam and phishing email) and 31 other datasets available at UCI data repository. The experimental results showed that the proposed technique efficiently improved SVM training speed in 100% of the datasets used for evaluation, without significantly affecting SVM classification quality. Furthermore, the Freidman's and Holm's post hoc tests were conducted to statistically validate the credibility of the results. The statistical test results revealed that the proposed technique is significantly faster, compared to the standard SVM and some existing instance selection techniques, in all cases. [ABSTRACT FROM AUTHOR]

This paper completely introduces an exhaustive and a comprehensive review of the so-called salp swarm algorithm (SSA) and discussions its main characteristics. SSA is one of the efficient recent meta-heuristic optimization algorithms, where it has been successfully utilized in a wide range of optimization problems in different fields, such as machine learning, engineering design, wireless networking, image processing, and power energy. This review shows the available literature on SSA, including its variants, like binary, modifications and multi-objective. Followed by its applications, assessment and evaluation, and finally the conclusions, which focus on the current works on SSA, suggest possible future research directions. [ABSTRACT FROM AUTHOR]

Feature selection is one of the hottest machine learning topics in recent years. The main purposes of it are to simplify the original model, improve the readability of the model, and prevent over-fitting by searching for a suitable subset of features. There are many methods for this problem, including evolutionary algorithms and particle swarm optimization. Among them, the competitive swarm optimizer is a new optimization algorithm proposed in recent years, which is based on particle swarm optimization algorithm, and has achieved good results in high-dimensional feature selection problems, but it also has the problems of high computation time cost and easily being premature. Aiming at these problems, this paper proposes to add the crossover operator and mutation operator in the genetic algorithm to the competitive swarm optimization, so as to improve the generation speed of new individuals in the algorithm and prevent premature population. After testing on UC Irvine Machine Learning Repository, the new algorithm not only improves the computational efficiency, but also avoids the problem that the competitive swarm optimization algorithm is easy to fall into the local optimum, which greatly improves the calculation effect. [ABSTRACT FROM AUTHOR]

This paper proposes a hybridized machine-learning framework called Extreme Learning Machine using self-adaptive multi-population-based Jaya algorithm for forecasting the currency exchange value. This learning technique attempts to take the advantages of generalization ability of Extreme Learning Machines (ELMs) along with the multi-population search scheme of Jaya optimization technique. This model can very well forecast the exchange price of USD–INR and USD–EURO based on statistical measures, technical indicators and combination of both measures over a time frame varying from 1 day to 1 month ahead. Proposed model has been compared with original ELM and ELM-Jaya along with technical analysis method such as discrete wavelet neural network optimized with self-adaptive multi-population-based Jaya and the comparison of different performance measures like MAPE, Theil's U, ARV and MAE reveal that ELM using self-adaptive multi-population-based Jaya hybrid models possesses superior compared to the rest predictive models. Comparison of different features demonstrates technical indicators outperform other two features such as statistical measures and combination of both technical indicators and statistical measures. [ABSTRACT FROM AUTHOR]

Generalizing both mixed-integer linear optimization and convex optimization, mixed-integer convex optimization possesses broad modeling power but has seen relatively few advances in general-purpose solvers in recent years. In this paper, we intend to provide a broadly accessible introduction to our recent work in developing algorithms and software for this problem class. Our approach is based on constructing polyhedral outer approximations of the convex constraints, resulting in a global solution by solving a finite number of mixed-integer linear and continuous convex subproblems. The key advance we present is to strengthen the polyhedral approximations by constructing them in a higher-dimensional space. In order to automate this extended formulation we rely on the algebraic modeling technique of disciplined convex programming (DCP), and for generality and ease of implementation we use conic representations of the convex constraints. Although our framework requires a manual translation of existing models into DCP form, after performing this transformation on the MINLPLIB2 benchmark library we were able to solve a number of unsolved instances and on many other instances achieve superior performance compared with state-of-the-art solvers like Bonmin, SCIP, and Artelys Knitro. [ABSTRACT FROM AUTHOR]

We study a Newton-like method for the minimization of an objective function $$\phi $$ that is the sum of a smooth function and an $$\ell _1$$ regularization term. This method, which is sometimes referred to in the literature as a proximal Newton method, computes a step by minimizing a piecewise quadratic model $$q_k$$ of the objective function $$\phi $$ . In order to make this approach efficient in practice, it is imperative to perform this inner minimization inexactly. In this paper, we give inexactness conditions that guarantee global convergence and that can be used to control the local rate of convergence of the iteration. Our inexactness conditions are based on a semi-smooth function that represents a (continuous) measure of the optimality conditions of the problem, and that embodies the soft-thresholding iteration. We give careful consideration to the algorithm employed for the inner minimization, and report numerical results on two test sets originating in machine learning. [ABSTRACT FROM AUTHOR]

In this paper, a simple and linearly convergent Lagrangian support vector machine algorithm for the dual of the twin support vector regression (TSVR) is proposed. Though at the outset the algorithm requires inverse of matrices, it has been shown that they would be obtained by performing matrix subtraction of the identity matrix by a scalar multiple of inverse of a positive semi-definite matrix that arises in the original formulation of TSVR. The algorithm can be easily implemented and does not need any optimization packages. To demonstrate its effectiveness, experiments were performed on well-known synthetic and real-world datasets. Similar or better generalization performance of the proposed method in less training time in comparison with the standard and twin support vector regression methods clearly exhibits its suitability and applicability. [ABSTRACT FROM AUTHOR]

In this paper, extreme learning machine (ELM) for ε-insensitive error loss function-based regression problem formulated in 2-norm as an unconstrained optimization problem in primal variables is proposed. Since the objective function of this unconstrained optimization problem is not twice differentiable, the popular generalized Hessian matrix and smoothing approaches are considered which lead to optimization problems whose solutions are determined using fast Newton-Armijo algorithm. The main advantage of the algorithm is that at each iteration, a system of linear equations is solved. By performing numerical experiments on a number of interesting synthetic and real-world datasets, the results of the proposed method are compared with that of ELM using additive and radial basis function hidden nodes and of support vector regression (SVR) using Gaussian kernel. Similar or better generalization performance of the proposed method on the test data in comparable computational time over ELM and SVR clearly illustrates its efficiency and applicability. [ABSTRACT FROM AUTHOR]

As a novel learning algorithm for single-hidden-layer feedforward neural networks, extreme learning machines (ELMs) have been a promising tool for regression and classification applications. However, it is not trivial for ELMs to find the proper number of hidden neurons due to the nonoptimal input weights and hidden biases. In this paper, a new model selection method of ELM based on multi-objective optimization is proposed to obtain compact networks with good generalization ability. First, a new leave-one-out (LOO) error bound of ELM is derived, and it can be calculated with negligible computational cost once the ELM training is finished. Furthermore, the hidden nodes are added to the network one-by-one, and at each step, a multi-objective optimization algorithm is used to select optimal input weights by minimizing this LOO bound and the norm of output weight simultaneously in order to avoid over-fitting. Experiments on five UCI regression data sets are conducted, demonstrating that the proposed algorithm can generally obtain better generalization performance with more compact network than the conventional gradient-based back-propagation method, original ELM and evolutionary ELM. [ABSTRACT FROM AUTHOR]

This paper addresses the question of maximizing classifier accuracy for classifying task-related mental activity from Magnetoencelophalography (MEG) data. We propose the use of different sources of information and introduce an automatic channel selection procedure. To determine an informative set of channels, our approach combines a variety of machine learning algorithms: feature subset selection methods, classifiers based on regularized logistic regression, information fusion, and multiobjective optimization based on probabilistic modeling of the search space. The experimental results show that our proposal is able to improve classification accuracy compared to approaches whose classifiers use only one type of MEG information or for which the set of channels is fixed a priori. [ABSTRACT FROM AUTHOR]

The harmony search (HS) method is a popular meta-heuristic optimization algorithm, which has been extensively employed to handle various engineering problems. However, it sometimes fails to offer a satisfactory convergence performance under certain circumstances. In this paper, we propose and study a hybrid HS approach, HS-PBIL, by merging the HS together with the population-based incremental learning (PBIL). Numerical simulations demonstrate that our HS-PBIL is well capable of outperforming the regular HS method in dealing with nonlinear function optimization and a practical wind generator optimization problem. [ABSTRACT FROM AUTHOR]

The support vector machine (SVM) is a novel machine learning tool in data mining. In this paper, the geometric approach based on the compressed convex hull (CCH) with a mathematical framework is introduced to solve SVM classification problems. Compared with the reduced convex hull (RCH), CCH preserves the shape of geometric solids for data sets; meanwhile, it is easy to give the necessary and sufficient condition for determining its extreme points. As practical applications of CCH, spare and probabilistic speed-up geometric algorithms are developed. Results of numerical experiments show that the proposed algorithms can reduce kernel calculations and display nice performances. [ABSTRACT FROM AUTHOR]