Search

Multiobjective evolutionary algorithms (MOEAs) have gained much attention due to their high effectiveness and efficiency in solving multiobjective optimization problems (MOPs). However, when solving MOPs, it is important but difficult to maintain a good balance of exploration and exploitation. In addition, some reference point based MOEAs with fixed reference points perform poorly on MOPs with irregular frontiers. Therefore, this paper proposes a new multistrategy multiobjective differential evolutionary (DE) algorithm, named RLMMDE. In RLMMDE, a multistrategy and multicrossover DE optimizer is utilized to alleviate the exploration and exploitation dilemma. An adaptive reference point activation mechanism based on RL is proposed to activate the adaptive adjustment of reference points. Moreover, a reference point adaptation method is proposed to improve the performance of RLMMDE on irregular frontier problems. Experimental results of RLMMDE tested on some benchmark test suites (i.e., ZDT, DTLZ, UF, WFG, and LSMOP) and two practical mixed-variable optimization problems show that the algorithm outperforms some advanced MOEAs. [ABSTRACT FROM AUTHOR]

The Successor Feature framework for Reinforcement Learning algorithms improves task transfer by decomposing the learned state–action value function. The decomposition involves two components, one that captures future-expected state features and the other that models the task-related reward structure. However, successful transfer between tasks depends heavily on how the reward function changes, possibly leading to failure of the original Successor Feature formulation. This paper proposes the Dynamic Successor Feature framework, DynSF, by extending the mathematical formulation of the original Successor Feature framework to center around a learned state-transition model. Under this formulation, the state-transition model dynamically induces the acting policy. The flexibility of DynSF also extends to the architecture, requiring only a state-transition model and a small vector of parameters. This architecture provides immense flexibility in the choice of the model used to learn the state-transition model. The DynSF framework is evaluated and compared to other baseline algorithms through several experiments in a continuous grid world environment, a robotic Reacher, and pixels in the Doom environment. • DynSF enables a state-transition model to be used in Reinforcement Learning. • Policy and discount factors can be parameterized dynamically. • Different supervised ML algorithms can be used to learn the state transition model. • DynSF can be used for guided exploration and transfer learning. • DynSF performs better during task transfer than the Successor Feature framework. [ABSTRACT FROM AUTHOR]

Feature selection is an important preprocessing technique in the fields of data mining and machine learning. With the promotion of privacy protection awareness, recently it becomes a very practical and challenging issue to select high-quality feature subsets while ensuring the privacy of all participants. However, there is a lack of research results on this issue, i.e., feature selection under privacy protection. Aiming at the issue, this paper proposes a federated feature selection framework for the first time. In the framework, inspiring by the idea of federated learning, a credible third participant is introduced to process and integrate optimal feature subsets from multiple participants. On the basis of the framework, a federated evolutionary feature selection algorithm based on particle swarm optimization is proposed to effectively solve feature selection problems with multiple participants under privacy protection. Two new operators satisfying the requirement of privacy protection, i.e., the feature assembling strategy with multi-participant cooperation and the swarm initialization strategy guided by assembling solution, are designed to improve the ability of the proposed algorithm. Compared with several typical assembling feature selection algorithms on 15 data sets, experimental results show that the proposed algorithm can significantly improve the classification accuracy of the feature subset selected by each participant, while protecting the privacy of data. [ABSTRACT FROM AUTHOR]

The combination of Machine Learning (ML) algorithms is a solution for constructing stronger predictors than a single one. However, some approximations suggest that combining unstable algorithms provides better results than combining stable algorithms. For instance, Generative ensembles, based on re-sampling techniques, have demonstrated high performance by fusing the information of unstable base learners. Random Forest and Gradient Boosting are two well-known examples, both combining Decision Trees and providing better predictions than those obtained using a single tree. However, such successful results have not been achieved by assembling stable algorithms. This paper introduces the notion of limited learner and a new ensemble general framework called Minimally Overfitted Ensemble (MOE), a re-sampling-based ensemble approach that constructs slightly overfitted-based learners. The proposed framework works well with stable and unstable base algorithms, thanks to a Weighted RAndom Bootstrap (WRAB) sampling that provides the necessary diversity for the stable base algorithms. A hyperparameter analysis of the proposal is carried out on artificial data. Besides, its performance is evaluated on real datasets against well-known ML methods. The results confirm that the MOE framework works successfully using stable and unstable base algorithms, improving in most cases the predictive ability of single ML models and other ensemble methods. • New ensemble general framework. • A novel concept of ensemble learners is introduced. • Minimal overfitted models and limited learners are defined. • The framework works with any base machine learning training algorithm. • The proposal compares successfully to well known ensemble approaches. [ABSTRACT FROM AUTHOR]

Clustering ensemble is an unsupervised ensemble learning method that is very important in machine learning, since it integrates multiple weak base clustering results to produce a strong consistency result. This paper proposes the Markov clustering ensemble (MCE) model to solve the weak stability and robustness of soft clustering ensemble. First, the base clustering algorithms are regarded as new features of the original datasets. Then, the results of the base clustering algorithms are the values of these features, which can break through the framework of consensus cluster ensemble. Second, as the base clustering results are discrete data, the maximum information coefficient is applied to measure their similarity. Accordingly, a graph-based cluster ensemble model can be constructed with row vectors as vertices and the similarity between row vectors as edges. Then, the Markov process can be applied to infer the graph-based cluster ensemble model; therefore, the MCE algorithm is designed according to the inference. To test the performance of the MCE algorithm, clustering algorithms and clustering ensemble algorithms are used to conduct comparative experiments on ten datasets. The experimental results show that the MCE algorithm outperforms the other algorithms in terms of accuracy and purity. [ABSTRACT FROM AUTHOR]

Image segmentation has been an important technique in the field of image processing. Fine-level manual annotations are very limited and difficult for a large collection of imbalanced images, where each image contains quite different small objects. However, we find that these imbalanced images have similar decision graphs obtained by a lightweight and simple clustering algorithm Density Peaks (DPeaks). Hence, in this paper, we propose a weakly supervised image segmentation algorithm. It trains a decision curve from decision graphs of a few sample imbalanced images by SVM and Support Vector Regression (SVR), which is effective for identifying the sparse region of an imbalanced image. Besides, RangeTree is applied to accelerate RDP for large images due to the high complexity of DPeaks Clustering. Experiments prove that the proposed algorithm works well on imbalanced image data sets, not only it can recognize main things, but also has the ability to identify some relatively small objects. [ABSTRACT FROM AUTHOR]

Algorithm evaluation is a very important task for different Machine Learning (ML) problems. In this work, we assume a different perspective in ML evaluation, in which algorithms are evaluated at the instance level. In this perspective, Item Response Theory (IRT) has recently been applied to algorithm evaluation in ML in order to identify which instances are more difficult and discriminating in a dataset, while also evaluating algorithms based on their predictions for instances with different difficulty values. In IRT, a strong algorithm returns accurate predictions for the most difficult instances, while maintaining a consistent behaviour in the easiest instances. The most common IRT models adopted in the literature only deal with dichotomous responses (i.e., a response has to be either correct or incorrect). This is suitable for evaluating classification algorithms, but not adequate in application contexts where responses are recorded in a continuous scale without an upper bound, such as regression. In this paper we propose the Γ -IRT model, particularly designed for dealing with positive unbounded responses, which we model using a Gamma distribution, parameterised according to respondent ability and item difficulty and discrimination parameters. The proposed parameterisation results in item characteristic curves with more flexible shapes compared to the traditional logistic curves adopted in IRT. We apply the proposed model to evaluate student responses (number of errors) in open-ended questions extracted from Statistics exams. Then, we use Γ -IRT to assess regression model abilities, where responses are the absolute errors in test instances. This novel application represents an alternative for evaluating regression performance and for identifying regions in a regression dataset that present different levels of difficulty and discrimination. [ABSTRACT FROM AUTHOR]

One of the crucial processes in machine learning algorithms to improve the performance as well as, in some cases, to reduce the computational cost is feature selection. In other words, when there are many features, some are irrelevant; thus, irrelevant features will badly affect the learning process, so selecting the most fitting features can lead to increasing the accuracy of models in high-dimensional datasets. Recent researches benefit from an integration of various feature selection methods to find relevant features more accurately. Despite the fact that applying several feature selection methods can be efficient, the accuracy of current methods and combining the ranks of different feature selection methods are still challenging problems. In this paper, a novel ensemble of feature selection based on weighted PageRank and fuzzy logic, named EFSPF, is proposed to overcome these drawbacks. First, an M × M matrix, called Euclidean Distance Matrix (EDM), is set up, where M is the number of features. Secondly, a fuzzy Type-I is applied individually to tackle uncertainties in ranks. In other words, to specify a reliability rate for each feature, fuzzy Type-I is utilized. Then, EFSPF forms a reliable weighted graph where each feature is considered as a node. Furthermore, the weight between two nodes expresses a combination of their Euclidean Distance in EDM and the calculated reliability rates through fuzzy Type-I. Finally, the rank of each graph node is calculated via the PageRank algorithm. To demonstrate the performance of EFSPF, we have compared it with five ensemble algorithms and eight primary feature selection algorithms. The proposed method is evaluated based on Accuracy, Precision, Recall, and F1 score metrics. The analysis results show that the proposed method outperforms its rivals. The source code of the proposed method is accessible from https://github.com/mehdijoodaki/EFSPF. [ABSTRACT FROM AUTHOR]

Existing research works in scene image classification have focused on different aspects such as content features (e.g., visual information), context features (e.g., annotations, semantic information, etc.) and both. However, such works suffer from various issues such as higher feature size and lower classification performance. In this paper, we propose a new feature extraction approach for scene image representation using two kinds of rich information: content features and context features. Specifically, the new content features are generated by multi-scale foreground and background information. Similarly, the new context features are generated by the novel compact supervised codebook. Our compact supervised codebook minimizes irrelevant and redundant information, which, in result, achieves the lower-sized contextual feature vector. Finally, we combine both content and context features to represent the scene image. Our experiments on three widely used benchmark scene datasets using Support Vector Machine (SVM) classifier reveal that our proposed context and content features produce better results than existing context and content features, respectively. The fusion of the proposed two types of features significantly outperform numerous state-of-the-art features. • Propose a novel contextual feature extraction method for scene images. • Propose a novel unique tokens extraction algorithm. • Aggregate multi-scale content and contextual features to represent the scene images. • Evaluate our proposed method on three commonly-used benchmark datasets. [ABSTRACT FROM AUTHOR]

Evolutionary Computation (EC) algorithms are powerful techniques for feature selection tasks however, they reach different solutions in each run, and this is known as the stability issue. Existing solutions to finding a stable subset of features when using an EC algorithm include aggregation and frequency-based methods. These methods may return feature subsets that achieve weak or inconsistent classification performance when utilised to build classifiers, and this limitation is known as 'lack of generalisation power'. To address this limitation, this paper proposes a novel algorithm called Generalisation Power Analysis (GPA) that measures the performance of feature subsets in terms of generalisation power and hence evaluates their ability to achieve optimal or near-optimal accuracy over multiple classifiers. GPA has been designed to work with the stochastic nature of EC algorithms. Experiments with eleven benchmark datasets revealed that the proposed GPA approach consistently outperformed alternative methods in finding subsets that achieved high generalisation power. Although GPA requires relatively higher computation time compared to alternative approaches as it embeds multiple classifiers, the advantages of using GPA during feature selection outweigh this limitation since the outcome will be a robust prediction model that has been developed using a subset of features that are not biased towards a specific classifier. • EC algorithms for feature selection may return different feature subsets whenever they are run • Generalisation Power Analysis (GPA) evaluates feature subsets over multiple classifiers • Experiments show that GPA outperforms alternative approaches for finding a stable set of features • GPA can help EC algorithms find a stable set of features needed for constructing machine learning models [ABSTRACT FROM AUTHOR]

Fuzzy rough set theory has been proved to be an effective tool to deal with uncertainty data. Some different forms of fuzzy uncertainty measures have been introduced in fuzzy rough set theory, such as fuzzy information entropy, fuzzy conditional entropy, and fuzzy mutual information. However, as far as we know, most of the above fuzzy conditional entropy and fuzzy mutual information are non-monotonic, which may lead to a non-convergent learning algorithm. For this reason, this paper proposes a novel fuzzy complementary entropy based on the hybrid-kernel function. Then, based on the proposed fuzzy complementary entropy, some corresponding uncertainty measures are also proposed. Furthermore, fuzzy complementary conditional entropy and fuzzy complementary mutual information are proved to change monotonously with attributes. Finally, based on the proposed uncertainty measures, three kinds of evaluation criteria for unsupervised hybrid attribute reduction are defined and a generalized attribute reduction algorithm is designed. The experimental results show that the proposed method is an effective scheme for reducing hybrid attributes. • The novel fuzzy complementary entropy is defined by using hybrid-kernel function. • It is proved that fuzzy complementary conditional entropy and fuzzy complementary mutual information change monotonously with attributes. • The proposed uncertainty measures are applied to unsupervised hybrid attribute reduction. [ABSTRACT FROM AUTHOR]

The multi-objective vehicle routing problem with stochastic demand (MO-VRPSD) is much harder to tackle than other traditional vehicle routing problems (VRPs), due to the uncertainty in customer demands and potentially conflicted objectives. In this paper, we present an improved multi-objective learnable evolution model (IMOLEM) to solve MO-VRPSD with three objectives of travel distance, driver remuneration and number of vehicles. In our method, a machine learning algorithm, i.e., decision tree, is exploited to help find and guide the desirable direction of evolution process. To cope with the key issue of "route failure" caused due to stochastic customer demands, we propose a novel chromosome representation based on priority with bubbles. Moreover, an efficient nondominated sort using a sequential search strategy (ENS-SS) in conjunction with some heuristic operations are leveraged to handle the multi-objective property of the problem. Our algorithm is evaluated on the instances of modified Solomon VRP benchmark. Experimental results show that the proposed IMOLEM is capable to find better Pareto front of solutions and also deliver superior performance to other evolutionary algorithms. [ABSTRACT FROM AUTHOR]

Deep multi-layer neural networks represent hypotheses of very high degree polynomials to solve very complex problems. Gradient descent optimization algorithms are utilized to train such deep networks through backpropagation, which suffers from permanent problems such as the vanishing gradient problem. To overcome the vanishing problem, we introduce a new anti-vanishing back-propagated learning algorithm called oriented stochastic loss descent (OSLD). OSLD updates a random-initialized parameter iteratively in the opposite direction of its partial derivative sign by a small positive random number, which is scaled by a tuned ratio of the model loss. This paper compares OSLD to stochastic gradient descent algorithm as the basic backpropagation algorithm and Adam as one of the best backpropagation algorithms in five benchmark models. Experimental results show that OSLD is very competitive to Adam in small and moderate depth models, and OSLD outperforms Adam in very long models. Moreover, OSLD is compatible with current backpropagation architectures except for learning rates. Finally, OSLD is stable and opens more choices in front of the very deep multi-layer neural networks. • A new back-propagated gradient descent optimization algorithm without a vanishing problem is proposed. • The proposed algorithm is called oriented stochastic loss descent (OSLD). • OSLD updates w i in the opposite side of its gradient sign by a tuned ratio of random loss. • OSLD is competitive to Adam and is compatible with most backpropagation architectures. • OSLD is stable and opens more choices in front of very deep multi-layer neural networks. [ABSTRACT FROM AUTHOR]

One-Class Slab Support Vector Machines (OCSSVM) have turned out to be better in terms of accuracy in certain types of classification problems than the traditional SVMs, One Class SVMs, and other one-class classifiers. This paper proposes a fast training method for the OCSSVM using a modified Sequential Minimal Optimization (SMO) algorithm, which would enhance its scalability without a significant compromise in precision. We compared our SMO-based algorithm, the regular OCSSVM, and other state-of-the-art one-class classifiers for time and accuracy on multiple benchmark datasets. The experimental results indicate that the proposed training method provides the best tradeoff between training time and accuracy among the compared methods. It achieves accuracies similar to the regular OCSSVM and better or comparable to existing state-of-the-art one-class classifiers. It provides better scalability than the regular OCSSVM and most other classifiers. [ABSTRACT FROM AUTHOR]

Searching for information about a specific person is a frequent online activity. In most cases, users are aided in the search process by queries containing a name in Web search engines. Typically, Web search engines provide just a few accurate results associated with a name-containing query. Most existing solutions for suggesting synonyms in online search are based on pattern matching and phonetic encoding, however very often, the performance of such solutions is less than optimal. In this paper, we propose SpokenName2Vec , a novel and generic algorithm which addresses the synonym suggestion problem by utilizing automated speech generation, and deep learning to produce novel spoken name embeddings. These embeddings capture the way people pronounce names in a particular language and accent. Utilizing a name's pronunciation can help detect names that sound alike, but are written differently. We demonstrated the proposed approach on a large-scale dataset with more than 250,000 forenames and surnames and evaluated it on two ground truth datasets containing 7400 forenames and 25,000 surnames (including their verified synonyms). The performance of SpokenName2Vec was found superior to the 10 other algorithms evaluated, including phonetic encoding, string similarity, and machine learning algorithms. The results obtained emphasize the potential of spoken name embeddings for improved synonym suggestion. • Proposing SpokenName2Vec, a novel and generic algorithm which addresses the synonym suggestion problem by utilizing automated speech generation and deep learning to produce spoken name embeddings. • Behind the Name dataset: In total, 37,916 synonyms were retrieved for the 7,399 distinct names. • Spoken Name dataset: 250K WAV files associated with the names in the dataset for 11 languages. • A demonstration of the quality of SpokenName2Vec on forenames and surnames, including a comparison to other 10 algorithms. [ABSTRACT FROM AUTHOR]