Search

Feature selection methods aim to improve the classification performance by eliminating non-valuable features. In this paper, our aim is to apply a recent optimization technique namely the Intelligent Water Drops (IWD) algorithm to select best features for sentiment analysis. We investigate the classification performances of our proposed IWD based feature selection method by comparing one of the well-known feature selection method using Maximum Entropy classifier. Experimental results show that Intelligent Water Drops based feature selection method outperforms than ReliefF method for sentiment analysis. [ABSTRACT FROM AUTHOR]

Support vector machine (SVM), as one of the most popular classifiers, aims to find a hyperplane that can separate two classes of data with maximal margin. SVM classifiers are focused on achieving more separation between classes than exploiting the structures in the training data within classes. However, the structural information, as an implicit prior knowledge, has recently been found to be vital for designing a good classifier in different real-world problems. Accordingly, using as much prior structural information in data as possible to help improve the generalization ability of a classifier has yielded a class of effective structural large margin classifiers, such as the structured large margin machine (SLMM) and the Laplacian support vector machine (LapSVM). In this paper, we unify these classifiers into a common framework from the concept of “structural granularity” and the formulation for optimization problems. We exploit the quadratic programming (QP) and second-order cone programming (SOCP) methods, and derive a novel large margin classifier, we call the new classifier the structural regularized support vector machine (SRSVM). Unlike both SLMM at the cross of the cluster granularity and SOCP and LapSVM at the cross of the point granularity and QP, SRSVM is located at the cross of the cluster granularity and QP and thus follows the same optimization formulation as LapSVM to overcome large computational complexity and non-sparse solution in SLMM. In addition, it integrates the compactness within classes with the separability between classes simultaneously. Furthermore, it is possible to derive generalization bounds for these algorithms by using eigenvalue analysis of the kernel matrices. Experimental results demonstrate that SRSVM is often superior in classification and generalization performances to the state-of-the-art algorithms in the framework, both with the same and different structural granularities. [ABSTRACT FROM AUTHOR]

This paper proposes an improved version of the grasshopper optimization algorithm (GOA) based on the opposition-based learning (OBL) strategy called OBLGOA for solving benchmark optimization functions and engineering problems. The proposed OBLGOA algorithm consists of two stages: the first stage generates an initial population and its opposite using the OBL strategy; and the second stage uses the OBL as an additional phase to update the GOA population in each iteration. However, the OBL is applied to only half of the solutions to reduce the time complexity. To investigate the performance of the proposed OBLGOA, six sets of experiment series are performed, and they include twenty-three benchmark functions and four engineering problems. The experiments revealed that the results of the proposed algorithm were superior to those of ten well-known algorithms in this domain. Eventually, the obtained results proved that the OBLGOA algorithm can provide competitive results for optimization engineering problems compared with state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the problem of binary classification with a reject option in which one can withhold the decision of classifying an observation at a cost lower than that of misclassification. Since the natural loss function is non-convex so that empirical risk minimization easily becomes infeasible, the paper proposes minimizing convex risks based on surrogate convex loss functions. A necessary and sufficient condition for infinite sample consistency (both risks share the same minimizer) is provided. Moreover, we show that the excess risk can be bounded through the excess surrogate risk under appropriate conditions. These bounds can be tightened by a generalized margin condition. The impact of the results is illustrated on several commonly used surrogate loss functions. [ABSTRACT FROM AUTHOR]

The support vector machine (SVM) is a novel machine learning tool in data mining. In this paper, the geometric approach based on the compressed convex hull (CCH) with a mathematical framework is introduced to solve SVM classification problems. Compared with the reduced convex hull (RCH), CCH preserves the shape of geometric solids for data sets; meanwhile, it is easy to give the necessary and sufficient condition for determining its extreme points. As practical applications of CCH, spare and probabilistic speed-up geometric algorithms are developed. Results of numerical experiments show that the proposed algorithms can reduce kernel calculations and display nice performances. [ABSTRACT FROM AUTHOR]

Optimization problems aim to minimize or maximize an objective function while fulfilling related constraints. This objective function may be a single or multi-objective optimization. Many studies have been conducted on using these optimization problems in civil and construction engineering, especially for the various machine learning techniques and algorithms that have been developed for fiber reinforced polymer (FRP) applications in the rehabilitation and design of RC structures. FRP is considered the most effective and superior technique for strengthening and retrofitting due to its significant benefits over traditional methods, which have numerous drawbacks, as well as the importance of structural strengthening as a cost-effective and practical option. In this research, an insight into how to apply algorithms and machine learning approaches to optimize FRP applications in civil and construction engineering is presented, as well as a detailed analysis of the various optimization strategies used and their findings. A total of 18 case studies from previous research were discussed and critically evaluated, and they were categorized into six groups according to the algorithm or machine learning technique utilized. Based on the case studies investigated in this study, the genetic algorithm was found to be the optimal algorithm utilized for optimizing FRP applications. The result of this research provides a useful guideline for future researchers and specialists. [ABSTRACT FROM AUTHOR]

We consider the submodular function minimization (SFM) and the quadratic minimization problems regularized by the Lovász extension of the submodular function. These optimization problems are intimately related; for example, min-cut problems and total variation denoising problems, where the cut function is submodular and its Lovász extension is given by the associated total variation. When a quadratic loss is regularized by the total variation of a cut function, it thus becomes a total variation denoising problem and we use the same terminology in this paper for "general" submodular functions. We propose a new active-set algorithm for total variation denoising with the assumption of an oracle that solves the corresponding SFM problem. This can be seen as local descent algorithm over ordered partitions with explicit convergence guarantees. It is more flexible than the existing algorithms with the ability for warm-restarts using the solution of a closely related problem. Further, we also consider the case when a submodular function can be decomposed into the sum of two submodular functions F1 and F2 and assume SFM oracles for these two functions. We propose a new active-set algorithm for total variation denoising (and hence SFM by thresholding the solution at zero). This algorithm also performs local descent over ordered partitions and its ability to warm start considerably improves the performance of the algorithm. In the experiments, we compare the performance of the proposed algorithms with state-of-the-art algorithms, showing that it reduces the calls to SFM oracles. [ABSTRACT FROM AUTHOR]

Feature selection is an important preprocessing method in machine learning and data mining. This process can be used not only to reduce the amount of data to be analyzed but also to build models with stronger interpretability based on fewer features. Traditional feature selection methods evaluate the dependency and redundancy of features separately, which leads to a lack of measurement of their combined effect. Moreover, a greedy search considers only the optimization of the current round and thus cannot be a global search. To evaluate the combined effect of different subsets in the entire feature space, an adaptive feature selection method based on V-shaped binary particle swarm optimization is proposed. In this method, the fitness function is constructed using the correlation information entropy. Feature subsets are regarded as individuals in a population, and the feature space is searched using V-shaped binary particle swarm optimization. The above procedure overcomes the hard constraint on the number of features, enables the combined evaluation of each subset as a whole, and improves the search ability of conventional binary particle swarm optimization. The proposed algorithm is an adaptive method with respect to the number of feature subsets. The experimental results show the advantages of optimizing the feature subsets using the V-shaped transfer function and confirm the effectiveness and efficiency of the feature subsets obtained under different classifiers. [ABSTRACT FROM AUTHOR]

Binary classification is the problem of predicting the class a given sample belongs to. To achieve a good prediction performance, it is important to find a suitable model for a given dataset. However, it is often time consuming and impractical for practitioners to try various classification models because each model employs a different formulation and algorithm. The dificulty can be mitigated if we have a unified formulation and an eficient universal algorithmic framework for various classification models to expedite the comparison of performance of different models for a given dataset. In this paper, we present a unified formulation of various classification models (including C-SVM, l2-SVM, v-SVM, MM-FDA, MM-MPM, logistic regression, distance weighted discrimination) and develop a general optimization algorithm based on an accelerated proximal gradient (APG) method for the formulation. We design various techniques such as backtracking line search and adaptive restarting strategy in order to speed up the practical convergence of our method. We also give a theoretical convergence guarantee for the proposed fast APG algorithm. Numerical experiments show that our algorithm is stable and highly competitive to specialized algorithms designed for specific models (e.g., sequential minimal optimization (SMO) for SVM). [ABSTRACT FROM AUTHOR]

There has been a growing interest in the cooperative communication of power line and wireless dual‐medium, due to the promising transmission quality and immense flexibility of the network. However, under the condition of hybrid fading and impulsive noise, the theoretical analysis of two‐way relaying communication with power line and wireless is difficult because of the lack of closed analytical expressions. In this study, the authors propose a relaying signal processing method based on Log Likelihood Ratio information estimation for two‐way relaying systems; and a power allocation optimisation algorithm based on Q‐learning, which can adaptively select the appropriate transmission power to maximise the mutual information. More precisely, the ϵ ${\epsilon}$‐greedy strategy is adopted for the estimated forwarding protocol and the transmission power of each node is adjusted to determine the optimal power allocation factor. The performances such as mutual information and outage probability are analysed and compared and the simulation shows that the algorithm is reliable and effective. [ABSTRACT FROM AUTHOR]

The fast-growing quantity of information hinders the process of machine learning, making it computationally costly and with substandard results. Feature selection is a pre-processing method for obtaining the optimal subset of features in a data set. Optimization algorithms struggle to decrease the dimensionality while retaining accuracy in high-dimensional data set. This article proposes a novel chaotic opposition fruit fly optimization algorithm, an improved variation of the original fruit fly algorithm, advanced and adapted for binary optimization problems. The proposed algorithm is tested on ten unconstrained benchmark functions and evaluated on twenty-one standard datasets taken from the Univesity of California, Irvine repository and Arizona State University. Further, the presented algorithm is assessed on a coronavirus disease dataset, as well. The proposed method is then compared with several well-known feature selection algorithms on the same datasets. The results prove that the presented algorithm predominantly outperform other algorithms in selecting the most relevant features by decreasing the number of utilized features and improving classification accuracy. [ABSTRACT FROM AUTHOR]

Orthogonal transformations, proper decomposition, and the Moore–Penrose inverse are traditional methods of obtaining the output layer weights for an extreme learning machine autoencoder. However, an increase in the number of hidden neurons causes higher convergence times and computational complexity, whereas the generalization capability is low when the number of neurons is small. One way to address this issue is to use the fast iterative shrinkage-thresholding algorithm (FISTA) to minimize the output weights of the extreme learning machine. In this work, we aim to improve the convergence speed of FISTA by using two fast algorithms of the shrinkage-thresholding class, called greedy FISTA (G-FISTA) and linearly convergent FISTA (LC-FISTA). Our method is an exciting proposal for decision-making involving the resolution of many application problems, especially those requiring longer computational times. In our experiments, we adopt six public datasets that are frequently used in machine learning: MNIST, NORB, CIFAR10, UMist, Caltech256, and Stanford Cars. We apply several metrics to evaluate the performance of our method, and the object of comparison is the FISTA algorithm due to its popularity for neural network training. The experimental results show that G-FISTA and LC-FISTA achieve higher convergence speeds in the autoencoder training process; for example, in the Stanford Cars dataset, G-FISTA and LC-FISTA are faster than FISTA by 48.42% and 47.32%, respectively. Overall, all three algorithms maintain good values of the performance metrics on all databases. [ABSTRACT FROM AUTHOR]

Cooperative optimization algorithms have been applied with success to solve many optimization problems. However, many of them often lose their effectiveness and advantages when solving large scale and complex problems, e.g., those with interacted variables. A key issue involved in cooperative optimization is the task of problem decomposition. In this paper, a fast search operator is proposed to capture the interdependencies among variables. Problem decomposition is performed based on the obtained interdependencies. Another key issue involved is the optimization of the subproblems. A cross-cluster mutation strategy is proposed to further enhance exploitation and exploration. More specifically, each operator is identified as exploitation-biased or exploration-biased. The population is divided into several clusters. For the individuals within each cluster, the exploitation-biased operators are applied. For the individuals among different clusters, the exploration-biased operators are applied. The proposed operators are incorporated into the original differential evolution algorithm. The experiments were carried out on CEC2008, CEC2010, and CEC2013 benchmarks. For comparison, six algorithms that yield top ranked results in CEC competition are selected. The comparison results demonstrated that the proposed algorithm is robust and comprehensive for large scale optimization problems. [ABSTRACT FROM AUTHOR]

Class imbalance situations, where one class is rare compared to the other, arise frequently in machine learning applications. It is well known that the usual misclassification error is not suitable in such settings. A wide range of performance measures such as AM and QM have been proposed for this problem. However, due to computational difficulties, few learning techniques have been developed to directly optimize for AM or QM metric. To fill the gap, in this paper, we present a general structural SVM framework for directly optimizing AM and QM. We define the loss functions oriented to AM and QM, respectively, and adopt the cutting plane algorithm to solve the outer optimization. For the inner problem of finding the most violated constraint, we propose two efficient algorithms for the AM and QM problem. Empirical studies on the various imbalanced datasets justify the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

The increasing volume and velocity of the continuously generated data (data stream) challenge machine learning algorithms, which must evolve to fit real-world problems. The data stream clustering algorithms face issues such as the rapidly increasing volume of the data, the variety of the number of clusters, and their shapes. The present work aims to improve the accuracy of sequential clustering batches of data streams for scenarios in which clusters evolve dynamically and continuously, automatically estimating their number. In order to achieve this goal, three evolutionary algorithms are presented, along with three novel algorithms designed to deal with clusters of normal distribution based on goodness-of-fit tests in the context of scalable batch stream clustering with automatic estimation of the number of clusters. All of them are developed on top of MapReduce, Discretized-Stream models, and the most recent MPC frameworks to provide scalability, reliability, resilience, and flexibility. The proposed algorithms are experimentally compared with state-of-the-art methods and present the best results for accuracy for normally distributed data sets, reaching their goal. [ABSTRACT FROM AUTHOR]

A novel learning algorithm for solving global numerical optimization problems is proposed. The proposed learning algorithm is intense stochastic search method which is based on evaluation and optimization of a hypercube and is called the hypercube optimization (HO) algorithm. The HO algorithm comprises the initialization and evaluation process, displacement-shrink process, and searching space process. The initialization and evaluation process initializes initial solution and evaluates the solutions in given hypercube. The displacement-shrink process determines displacement and evaluates objective functions using new points, and the search area process determines next hypercube using certain rules and evaluates the new solutions. The algorithms for these processes have been designed and presented in the paper. The designed HO algorithm is tested on specific benchmark functions. The simulations of HO algorithm have been performed for optimization of functions of 1000-, 5000-, or even 10000 dimensions. The comparative simulation results with other approaches demonstrate that the proposed algorithm is a potential candidate for optimization of both low and high dimensional functions. [ABSTRACT FROM AUTHOR]

Artificial Neural Networks (ANNs) are applied to many complex real-world problems, ranging from image recognition to autonomous robot control. However, to design a neural network that can implement special task, it is necessary to select an appropriate biological neuron model, meanwhile, good learning algorithm should be adopted to achieve the expected goal. Neuroevolution is a form of machine learning that uses Evolutionary Algorithms (EAs) to train ANNs. EAs, for the learning algorithm used by neural networks, can provide alternative and complementary solution, which can avoid the frequently happened issues of \getting stuck in local minimum" during the iteration process made by gradient-based learning algorithms. In this paper, a method using Hybrid PSO-based Learning Algorithm (HPLA) to evolve the connection weights and network parameters of Binary-Weights Neural Network (BWNN) will be introduced. The extracted knowledge from trained BWNN can then be used to construct high-speed shift-and-add based Color Space Converter (CSC) hardware architecture. The experimental results in this research also show that the performance of implemented hardware architecture is good at RGB to YCbCr color space converting, and it also has the advantages of high-speed and lowcomplexity. [ABSTRACT FROM AUTHOR]

We study the problem of simplifying a given directed graph by keeping a small subset of its arcs. Our goal is to maintain the connectivity required to explain a set of observed traces of information propagation across the graph. Unlike previous work, we do not make any assumption about an underlying model of information propagation. Instead, we approach the task as a combinatorial problem. We prove that the resulting optimization problem is $$\mathbf{NP}$$ -hard. We show that a standard greedy algorithm performs very well in practice, even though it does not have theoretical guarantees. Additionally, if the activity traces have a tree structure, we show that the objective function is supermodular, and experimentally verify that the approach for size-constrained submodular minimization recently proposed by Nagano et al. (28th International Conference on Machine Learning, ) produces very good results. Moreover, when applied to the task of reconstructing an unobserved graph, our methods perform comparably to a state-of-the-art algorithm devised specifically for this task. [ABSTRACT FROM AUTHOR]

Sine Cosine Algorithm (SCA) that is one of the population-based metaheuristic optimization algorithms basically consists of the updating mechanism based on sine and cosine functions. In this algorithm, a few random and adaptive variables are also utilized for more effective motions of the candidate solutions. SCA has some drawbacks like other some metaheuristic algorithms. SCA tends to be stuck into the local regions in the search space and this affects negatively on the computational effort required to find the best solution point in the search space. This paper presents four different improved versions of SCA. The proposed improvements on original SCA are the innovations on the updating mechanism of SCA. To evaluate the performances of Improved Sine Cosine Algorithms (ImpSCAs), well-known numerical optimization problems including CEC 2014 test suite are used. Firstly, different analyses of the proposed ImpSCAs are dealt with such as the convergence analysis, search history analysis, trajectory analysis, average distance analysis, and computational complexity analysis. Secondly, the proposed four versions of ImpSCAs are compared with the original SCA for CEC 2014 benchmark problems with dimension sizes of 10D, 30D and 50D. Finally, original SCA and ImpSCAs are adapted to select optimal feature combination and they are tested for 10 feature selection datasets taken from the UCI machine learning repository. The benchmark results show that the performances of the ImpSCA 1 , ImpSCA 2 , and ImpSCA 4 are better than that of the original SCA. From the feature selection results, it is observed that three versions of ImpSCAs (except ImpSCA 3) outperform the original SCA in 80% of the datasets. Source codes of ImpSCAs are publicly available at https://github.com/uguryuzgec/ImpSCAs. • We focused to improve the exploration and exploitation ability of the original SCA. • The basic idea is based on the innovations on the update mechanism. • The ISCAs were evaluated using CEC2014 benchmark test suite. • The proposed ISCAs were tested for 10 popular UCI datasets. • Three versions of ISCAs (except ISCA 3) outperform the SCA in 80% of the datasets. [ABSTRACT FROM AUTHOR]

• The Rosenbrock method is applied to the Harris hawks optimization algorithm. • The proposed method is abbreviated as RLHHO. • A new random exploration technique is proposed and applied to the RLHHO. • RLHHO is tested over 53 test functions and 4 real-world problems. • RLHHO compares with multiple algorithms to verify its validity. Harris Hawks Optimization (HHO) algorithm is a newly proposed meta-heuristic optimization algorithm that simulates the hunting process of the Harris hawks. It has the characteristics of fewer adjustment parameters and a strong optimization effect, resulting in strong competitiveness in similar optimization algorithms. However, HHO is prone to premature convergence and low convergence accuracy when dealing with specific complex optimization problems. Therefore, our work integrates two novel strategies into the standard HHO to gain enhanced exploration and exploitation capabilities. Specifically, our work firstly proposed an exploration strategy based on logarithmic spiral and opposition-based learning to improve the exploration ability of HHO. Secondly, the local search technique for Rosenbrock Method (RM) is modified to dynamically fuse into the standard HHO to enhance the algorithm's local search capability and improve the convergence accuracy. The novel meta-heuristic algorithm proposed in this paper is called RLHHO. Finally, to validate the algorithm's effectiveness, the proposed RLHHO algorithm is fully performance tested with eight other traditional meta-heuristic optimization algorithms on 23 benchmark functions and 30 IEEE CEC2014 test functions. Besides, another six advanced meta-heuristics algorithms are also compared in the 30 CEC'2014 test functions. The experimental results show that RLHHO performs significantly better than HHO as well as other traditional and advanced meta-heuristic algorithms in most of the test functions. To test the scalability of RLHHO in complex real-world problems, it was used to optimize the solution of three constrained real-world engineering problems, and the experimental results show that RLHHO's powerful performance can be used as an effective tool for solving constrained engineering problems. Also, an effective hybrid model of kernel extreme learning machine is developed on the basis of RLHHO to cope with bankruptcy prediction problem. The experimental results show that this hybrid model is highly competitive with other mainstream classifiers regarding stability and prediction accuracy. The supplementary info and answers to possiblequeries will be publicly available at https://www.researchgate.net/profile/Chenyang_Li39/research. [ABSTRACT FROM AUTHOR]

Multimedia event detection is the task of detecting a specific event of interest in an user-generated video on websites. The most fundamental challenge facing this task lies in the enormously varying quality of the video as well as the high-level semantic abstraction of event inherently. In this paper, we decompose the video into several segments and intuitively model the task of complex event detection as a multiple instance learning problem by representing each video as a "bag" of segments in which each segment is referred to as an instance. Instead of treating the instances equally, we associate each instance with a reliability variable to indicate its importance and then select reliable instances for training. To measure the reliability of the varying instances precisely, we propose a visual-semantic guided loss by exploiting low-level feature from visual information together with instance-event similarity based high-level semantic feature. Motivated by curriculum learning, we introduce a negative elastic-net regularization term to start training the classifier with instances of high reliability and gradually taking the instances with relatively low reliability into consideration. An alternative optimization algorithm is developed to solve the proposed challenging non-convex non-smooth problem. Experimental results on standard datasets, i.e., TRECVID MEDTest 2013 and TRECVID MEDTest 2014, demonstrate the effectiveness and superiority of the proposed method to the baseline algorithms. • A visual-semantic guided loss is proposed to measure reliability of instance for event detection. • Training begins with high-reliability instances and gradually added instances of low reliability. • Promising experimental results show the effectiveness and superiority of the proposed method. [ABSTRACT FROM AUTHOR]

A Dandelion algorithm (DA) inspired by the seed dispersal process of dandelions has been proposed as a newly intelligent optimization algorithm. For improving its exploration ability as well as reducing the probability of its falling into a local optimum, this work proposes to add a novel competition mechanism with historical information feedback to current DA. Specifically, the fitness value of each dandelion in the next generation, which is calculated by linear prediction, is compared with the current best dandelion, and the loser is replaced by a new offspring. Current DA generates new offsprings without considering historical information. This work improves its offspring generation process by exploiting historical information with an estimation-of-distribution algorithm. Three historical information models are designed. They are best, worst, and hybrid historical information feedback models. The experimental results show that the proposed algorithms outperform DA and its variants, and the proposed algorithms are superior or competitive to nine participating algorithms benchmarked on 28 functions from CEC2013. Finally, the proposed algorithms demonstrate the effectiveness on four real-world problems, and the results indicate that the proposed algorithms have better performance than its peers. [ABSTRACT FROM AUTHOR]

Practical optimum design of structures often involves parameters with uncertainties. There have been several ways to deal with such optimisation problems, and one of the approaches is an antioptimisation process. The task is to find the optimum solution of common design variables while simultaneously searching for the worst case scenario of those parameters with uncertainties. This paper proposed a metaheuristic based on population-based incremental learning (PBIL) for solving antioptimisation of trusses. The new algorithm is called two-level PBIL which consists of outer and inner loops. Five antioptimisation problems are posed to test the optimiser performance. The numerical results show that the concept of using PBIL probability vectors for handling the antioptimisation of truss is powerful and effective. The two-level PBIL can be considered a powerful optimiser for antioptimisation of trusses. [ABSTRACT FROM AUTHOR]

Interpretability and accuracy are two important features of fuzzy systems which are conflicting in their nature. One can be improved at the cost of the other and this situation is identified as "Interpretability-Accuracy Trade-Off". To deal with this trade-off Multi-Objective Evolutionary Algorithms (MOEA) are frequently applied in the design of fuzzy systems. Several novel MOEA have been proposed and invented for this purpose, more specifically, Non-Dominated Sorting Genetic Algorithms (NSGA-II), Strength Pareto Evolutionary Algorithm 2 (SPEA2), Fuzzy Genetics-Based Machine Learning (FGBML), (2 + 2) Pareto Archived Evolutionary Strategy ((2 + 2) PAES), (2 + 2) Memetic- Pareto Archived Evolutionary Strategy ((2 + 2) M-PAES), etc. This paper introduces and reviews the approaches to the issue of developing fuzzy systems using Evolutionary Multi-Objective Optimization (EMO) algorithms considering 'Interpretability-Accuracy Trade-off' and mainly focusing on the work in the last decade. Different research issues and challenges are also discussed. [ABSTRACT FROM AUTHOR]

In this paper, we propose a nonconvex online Support Vector Machine (SVM) algorithm (LASVM-NC) based on the Ramp Loss, which has the strong ability of suppressing the influence of outliers. Then, again in the online learning setting, we propose an outlier filtering mechanism (LASVM-I) based on approximating nonconvex behavior in convex optimization. These two algorithms are built upon another novel SVM algorithm (LASVM-G) that is capable of generating accurate intermediate models in its iterative steps by leveraging the duality gap. We present experimental results that demonstrate the merit of our frameworks in achieving significant robustness to outliers in noisy data classification where mislabeled training instances are in abundance. Experimental evaluation shows that the proposed approaches yield a more scalable online SVM algorithm with sparser models and less computational running time, both in the training and recognition phases, without sacrificing generalization performance. We also point out the relation between nonconvex optimization and min-margin active learning. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a non-domination based sorting multiobjective teaching-learning-based optimization algorithm, for solving the optimal power flow (OPF) problem. The OPF problem is a nonlinear constrained multi-objective optimization problem where the fuel cost, Transmission losses and L-index are to be minimized. Since the problem is treated as a true multi-objective optimization problem, different trade-off solutions are provided. The decision maker has an option to choose a solution among the different trade-off solutions provided in the pareto-optimal front. The standard IEEE 30-bus test system is used and the results show the effectiveness of MOTLBO and confirm its potential to solve the multi-objective OPF problem. Simulation results clearly show that the proposed method is able to produce true and well distributed Pareto optimal solutions for multiobjective OPF problem and the comparison with the results reported in the literature demonstrates the superiority of the proposed approach and confirms its potential to solve the multi-objective OPF problem. [Copyright &y& Elsevier]

This paper investigates a learning formulation called structured sparsity, which is a natural extension of the standard sparsity concept in statistical learning and compressive sensing. By allowing arbitrary structures on the feature set, this concept generalizes the group sparsity idea that has become popular in recent years. A general theory is developed for learning with structured sparsity, based on the notion of coding complexity associated with the structure. It is shown that if the coding complexity of the target signal is small, then one can achieve improved performance by using coding complexity regularization methods, which generalize the standard sparse regularization. Moreover, a structured greedy algorithm is proposed to efficiently solve the structured sparsity problem. It is shown that the greedy algorithm approximately solves the coding complexity optimization problem under appropriate conditions. Experiments are included to demonstrate the advantage of structured sparsity over standard sparsity on some real applications. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we proposed a new algorithm, the Sparse AUC maximizing support vector machine, to get more sparse features and higher AUC than standard SVM. By applying p-norm where 0 < p < 1 to the weight w of the separating hyperplane (w·x) + b = 0, the new algorithm can delete less important features corresponding to smaller |w|. Besides, by applying the AUC maximizing objective function, the algorithm can get higher AUC which make the decision function have higher prediction ability. Experiments demonstrate the new algorithm''s effectiveness. Some contributions as follows: (1) the algorithm optimizes AUC instead of accuracy; (2) incorporating feature selection into the classification process; (3) conduct experiments to demonstrate the performance. [Copyright &y& Elsevier]

Abstract: Automatic communication signal recognition plays an important role for many novel computer and communication technologies. Most of the proposed techniques can only identify a few kinds of digital signal and/or low order of them. They usually require high levels of signal to noise ratio (SNR). In this paper, we investigate twofold. First, we propose an efficient system that uses a combination set of spectral characteristics and higher order moments up to eighth and higher order cumulants up to eighth as the effective features. As the classifier we used a multi-layer perceptron (MLP) neural network. In this stage we investigate different learning algorithms of MLP neural networks that some of them, such as quick prop (QP) learning algorithm, extended delta-bar-delta (EDBD), super self adaptive back propagation (SuperSAB) and conjugate gradient (CG) are proposed for the first time in the area of communication signals recognition. Experimental results show that proposed system discriminates a lot of digital communication signals with high accuracy even at very low SNRs. But a lot of features are used for this recognition. Then at the second fold, in order to reduce the complexity of the recognizer, we have proposed a novel hybrid intelligent technique. In this technique we have optimized the classifier design by Bees Algorithm (BA) for selection of the best features that are fed to the classifier. Simulation results show that the proposed technique has very high recognition accuracy with seven features selected by BA. [ABSTRACT FROM AUTHOR]

In the last few years, due to the growing ubiquity of unlabeled data, much effort has been spent by the machine learning community to develop better understanding and improve the quality of classifiers exploiting unlabeled data. Following the manifold regularization approach, Laplacian Support Vector Machines (LapSVMs) have shown the state of the art performance in semi-supervised classification. In this paper we present two strategies to solve the primal LapSVM problem, in order to overcome some issues of the original dual formulation. In particular, training a LapSVM in the primal can be efficiently performed with preconditioned conjugate gradient. We speed up training by using an early stopping strategy based on the prediction on unlabeled data or, if available, on labeled validation examples. This allows the algorithm to quickly compute approximate solutions with roughly the same classification accuracy as the optimal ones, considerably reducing the training time. The computational complexity of the training algorithm is reduced from O(n³) to O(kn²), where n is the combined number of labeled and unlabeled examples and k is empirically evaluated to be significantly smaller than n. Due to its simplicity, training LapSVM in the primal can be the starting point for additional enhancements of the original LapSVM formulation, such as those for dealing with large data sets. We present an extensive experimental evaluation on real world data showing the benefits of the proposed approach. [ABSTRACT FROM AUTHOR]

The growth of information available to learning systems and the increasing complexity of learning tasks determine the need for devising algorithms that scale well with respect to all learning parameters. In the context of supervised sequential learning, the Viterbi algorithm plays a fundamental role, by allowing the evaluation of the best (most probable) sequence of labels with a time complexity linear in the number of time events, and quadratic in the number of labels. In this paper we propose CarpeDiem, a novel algorithm allowing the evaluation of the best possible sequence of labels with a sub-quadratic time complexity. We provide theoretical grounding together with solid empirical results supporting two chief facts. CarpeDiem always finds the optimal solution requiring, in most cases, only a small fraction of the time taken by the Viterbi algorithm; meantime, CarpeDiem is never asymptotically worse than the Viterbi algorithm, thus confirming it as a sound replacement. [ABSTRACT FROM AUTHOR]

In this paper we propose a hybrid algorithm to optimize the structure of TSK type fuzzy model using backpropagation (BP) learning algorithm and non-dominated sorting genetic algorithm (NSGA-II). In a first step, BP algorithm is used to optimize the parameters of the model (parameters of membership functions and fuzzy rules). NSGA-II is used in a second phase, to optimize the number of fuzzy rules and to fine tune the parameters. A well known benchmark is used to evaluate performances of the proposed modelling approach, and compare it with other modelling approaches. [Copyright &y& Elsevier]

Abstract: This paper considers two semi-online scheduling problems, one with known optimal value and the other with known total sum, on two uniform machines with a machine speed ratio of . For the first problem, we provide an optimal algorithm for , and improved algorithms or/and lower bounds for , over which the optimal algorithm is unknown. As a result, the largest gap between the competitive ratio and the lower bound decreases to 0.02192. For the second problem, we also present algorithms and lower bounds for . The largest gap between the competitive ratio and the lower bound is 0.01762, and the length of the interval over which the optimal algorithm is unknown is 0.47382. Our algorithms and lower bounds for these two problems provide insights into their differences, which are unusual from the viewpoint of the known results on these two semi-online scheduling problems in the literature. [Copyright &y& Elsevier]

In recent years, with the fast development of the internet and online platforms such as social media feeds, news blogs, and online newspapers, deceptive reports have been universally spread online. This manipulated news is a matter of concern due to its potential role in shaping public opinion. Therefore, the fast spread of fake news creates an urgent need for automatic systems to detect deceitful articles. This motivates many researchers to introduce solutions for the automatic classification of news items. This paper proposed a novel system to detect fake news articles based on content-based features and the WOA-Xgbtree algorithm. The proposed system can be applied in different scenarios to classify news articles. The proposed approach consists of two main stages: first, the useful features are extracted and analyzed, and then an Extreme Gradient Boosting Tree (xgbTree) algorithm optimized by the Whale Optimization Algorithm (WOA) to classify news articles using extracted features. In our experiments, we considered the bases of the investigation on classification accuracy and the F1-measure. Then, we compared the optimized model with several benchmark classification algorithms based on a dataset that compiled over 40,000 various news articles recently. The results indicate that the proposed approach achieved good classification accuracy and F1 measure rate and successfully classified over 91 percent of articles. [ABSTRACT FROM AUTHOR]

The authors present a response to comments on their paper "Evidence Contrary to the Statistical View of Boosting." They state that Kristin P. Bennett provided a useful perspective on the situation by examining the convergence of boosting algorithms from the optimization point of view. Moreover, the authors agree with Andreas Buja and Werner Stuetzle that a complete view of boosting should explain both its ability to reduce bias and variance.

Due to its wide applicability, the problem of semi-supervised classification is attracting increasing attention in machine learning. Semi-Supervised Support Vector Machines (S³VMs) are based on applying the margin maximization principle to both labeled and unlabeled examples. Unlike SVMs, their formulation leads to a non-convex optimization problem. A suite of algorithms have recently been proposed for solving S³VMs. This paper reviews key ideas in this literature. The performance and behavior of various S³VM algorithms is studied together, under a common experimental setting. [ABSTRACT FROM AUTHOR]

Center-based partitioning clustering algorithms rely on minimizing an appropriately formulated objective function, and different formulations suggest different possible algorithms. In this paper, we start with the standard nonconvex and nonsmooth formulation of the partitioning clustering problem. We demonstrate that within this elementary formulation, convex analysis tools and optimization theory provide a unifying language and framework to design, analyze and extend hard and soft center-based clustering algorithms, through a generic algorithm which retains the computational simplicity of the popular k-means scheme. We show that several well known and more recent center-based clustering algorithms, which have been derived either heuristically, or/and have emerged from intuitive analogies in physics, statistical techniques and information theoretic perspectives can be recovered as special cases of the proposed analysis and we streamline their relationships. [ABSTRACT FROM AUTHOR]

The task of learning models for many real-world problems requires incorporating domain knowledge into learning algorithms, to enable accurate learning from a realistic volume of training data. This paper considers a variety of types of domain knowledge for constraining parameter estimates when learning Bayesian networks. In particular, we consider domain knowledge that constrains the values or relationships among subsets of parameters in a Bayesian network with known structure. We incorporate a wide variety of parameter constraints into learning procedures for Bayesian networks, by formulating this task as a constrained optimization problem. The assumptions made in module networks, dynamic Bayes nets and context specific independence models can be viewed as particular cases of such parameter constraints. We present closed form solutions or fast iterative algorithms for estimating parameters subject to several specific classes of parameter constraints, including equalities and inequalities among parameters, constraints on individual parameters, and constraints on sums and ratios of parameters, for discrete and continuous variables. Our methods cover learning from both frequentist and Bayesian points of view, from both complete and incomplete data. We present formal guarantees for our estimators, as well as methods for automatically learning useful parameter constraints from data. To validate our approach, we apply it to the domain of fMRI brain image analysis. Here we demonstrate the ability of our system to first learn useful relationships among parameters, and then to use them to constrain the training of the Bayesian network, resulting in improved cross-validated accuracy of the learned model. Experiments on synthetic data are also presented. [ABSTRACT FROM AUTHOR]

Learning general functional dependencies between arbitrary input and output spaces is one of the key challenges in computational intelligence. While recent progress in machine learning has mainly focused on designing flexible and powerful input representations, this paper addresses the complementary issue of designing classification algorithms that can deal with more complex outputs, such as trees, sequences, or sets. More generally, we consider problems involving multiple dependent output variables, structured output spaces, and classification problems with class attributes. In order to accomplish this, we propose to appropriately generalize the well-known notion of a separation margin and derive a corresponding maximum-margin formulation. While this leads to a quadratic program with a potentially prohibitive, i.e. exponential, number of constraints, we present a cutting plane algorithm that solves the optimization problem in polynomial time for a large class of problems. The proposed method has important applications in areas such as computational biology, natural language processing, information retrieval/extraction, and optical character recognition. Experiments from various domains involving different types of output spaces emphasize the breadth and generality of our approach. [ABSTRACT FROM AUTHOR]

Typical recommender systems adopt a static view of the recommendation process and treat it as a prediction problem. We argue that it is more appropriate to view the problem of generating recommendations as a sequential optimization problem and, consequently, that Markov decision processes (MDPs) provide a more appropriate model for recommender systems. MDPs introduce two benefits: they take into account the long-term effects of each recommendation and the expected value of each recommendation. To succeed in practice, an MDP-based recommender system must employ a strong initial model, must be solvable quickly, and should not consume too much memory. In this paper, we describe our particular MDP model, its initialization using a predictive model, the solution and update algorithm, and its actual performance on a commercial site. We also describe the particular predictive model we used which outperforms previous models. Our system is one of a small number of commercially deployed recommender systems. As far as we know, it is the first to report experimental analysis conducted on a real commercial site. These results validate the commercial value of recommender systems, and in particular, of our MDP-based approach. [ABSTRACT FROM AUTHOR]

Winner-take-all multiclass classifiers are built on the top of a set of prototypes each representing one of the available classes. A pattern is then classified with the label associated to the most 'similar' prototype. Recent proposal of SVM extensions to multiclass can be considered instances of the same strategy with one prototype per class. The multi-prototype SVM proposed in this paper extends multiclass SVM to multiple prototypes per class. It allows to combine several vectors in a principled way to obtain large margin decision functions. For this problem, we give a compact constrained quadratic formulation and we propose a greedy optimization algorithm able to find locally optimal solutions for the non convex objective function. This algorithm proceeds by reducing the overall problem into a series of simpler convex problems. For the solution of these reduced problems an efficient optimization algorithm is proposed. A number of pattern selection strategies are then discussed to speed-up the optimization process. In addition, given the combinatorial nature of the overall problem, stochastic search strategies are suggested to escape from local minima which are not globally optimal. Finally, we report experiments on a number of datasets. The performance obtained using few simple linear prototypes is comparable to that obtained by state-of-the-art kernel-based methods but with a significant reduction (of one or two orders) in response time. [ABSTRACT FROM AUTHOR]

• A new multi-objective optimization method used OBL strategy, WOA and DE algorithms • It combines DE and the OBL to improve the performance of the WOA • The MWDEO results outperformed all other algorithms in most of the test problems • 32 multi-objective test problems are used in the experiments and CEC2017 problems The optimization problems are divided into a single objective and multi-objective. Single objective optimization has only one objective function; whereas, multi-objective optimization has multiple objective functions that generate the Pareto set; therefore, solving a multi-objective problem is a challenging problem. This paper presents a new multi-objective optimization method (called MWDEO) based on improved whale optimization algorithm (WOA) by combining the differential evolution (DE) algorithm and the opposition-based learning (OBL). The MWDEO uses the WOA to perform a global exploration, whereas DE is used to exploit the search space; while the OBL is applied to improve the exploration and exploitation by generating the opposite values. The proposed algorithm is evaluated using 32 multi-objective test problems besides a set of benchmark problems of CEC'2017. The experimental results are compared with nine state-of-the-art multi-objective methods. The analysis of the results showed that the proposed MWDEO outperformed all other algorithms in most of the test problems which indicates that the proposed MWDEO is competitive and effective in solving different types of multi-objective problems. [ABSTRACT FROM AUTHOR]