Search

• FLA has attracted attention of researchers for its research significance and application. • This paper attempts to give an overview of fitness landscape analysis for optimization. • Future research works are discussed and given in four aspects. Over past few decades, as a powerful analytical tool to characterize the fitness landscape of a problem, fitness landscape analysis (FLA) has been widely concerned and utilized for all kinds of optimization areas. Since its introduction by Sewell Wright in 1932, FLA has attracted more and more attention of researchers for its research significance and application value such as an intuitive understanding features of complex optimization problems, explaining evolutionary algorithm behavior, assessing performances of algorithms, and guiding selections and/or configures of algorithms. This paper attempts to give an overview of fitness landscape analysis and its typical application for optimization so far. We hope that this survey can help to understand features of complex optimization problems in depth and thus to improve the certain algorithm performance of for a given optimization problem. [ABSTRACT FROM AUTHOR]

In this paper, we propose a graph classification approach for automatically determining whether to use a monolithic or a decomposition-based solution method. In this approach, an optimization problem is represented as a graph that captures the structural and functional coupling among the variables and constraints of the problem via an appropriate set of features. Given this representation, a graph classifier can be built to assist a solver in selecting the best solution strategy for a given problem with respect to some metric of choice. The proposed approach is used to develop a classifier that determines whether a convex Mixed Integer Nonlinear Programming problem should be solved using branch and bound or the outer approximation algorithm. Finally, it is shown how the learned classifier can be incorporated into existing mixed integer optimization solvers. • A deep learning framework is proposed to decide when to use a decomposition. • A feature-based graph representation of optimization problems is proposed. • The proposed approach is applied to Mixed Integer Nonlinear optimization problems. • Integration of the proposed approach with optimization solvers is discussed. [ABSTRACT FROM AUTHOR]

The process of extracting valuable and novel insights from raw data involves a series of complex steps. In the realm of Automated Machine Learning (AutoML), a significant research focus is on automating aspects of this process, specifically tasks like selecting algorithms and optimising their hyper-parameters. A particularly challenging task in AutoML is automatic workflow composition (AWC). AWC aims to identify the most effective sequence of data preprocessing and machine learning algorithms, coupled with their best hyper-parameters, for a specific dataset. However, existing AWC methods are limited in how many and in what ways they can combine algorithms within a workflow. Addressing this gap, this paper introduces EvoFlow , a grammar-based evolutionary approach for AWC. EvoFlow enhances the flexibility in designing workflow structures, empowering practitioners to select algorithms that best fit their specific requirements. EvoFlow stands out by integrating two innovative features. First, it employs a suite of genetic operators, designed specifically for AWC, to optimise both the structure of workflows and their hyper-parameters. Second, it implements a novel updating mechanism that enriches the variety of predictions made by different workflows. Promoting this diversity helps prevent the algorithm from overfitting. With this aim, EvoFlow builds an ensemble whose workflows differ in their misclassified instances. To evaluate EvoFlow 's effectiveness, we carried out empirical validation using a set of classification benchmarks. We begin with an ablation study to demonstrate the enhanced performance attributable to EvoFlow 's unique components. Then, we compare EvoFlow with other AWC approaches, encompassing both evolutionary and non-evolutionary techniques. Our findings show that EvoFlow 's specialised genetic operators and updating mechanism substantially outperform current leading methods in predictive performance. Additionally, EvoFlow is capable of discovering workflow structures that other approaches in the literature have not considered. • A grammar-guided genetic programming approach for automated workflow composition. • The grammar support workflows with more flexible structure than previous methods. • An ensemble of workflows is built to guarantee diversity in terms of predictions. • Our method outperforms the existing methods in the literature. • Discovery of a wider spectrum of solutions than the baseline evolutionary method. [ABSTRACT FROM AUTHOR]

Quantitative structure–activity relationship (QSAR) models are tools for linking chemical activities with molecular structures and compositions. Due to the concern about the proliferating number of disinfection byproducts (DBPs) in water and the associated financial and technical burden, researchers have recently begun to develop QSAR models to investigate the toxicity, formation, property, and removal of DBPs. However, there are no standard procedures or best practices regarding how to develop QSAR models, which potentially limit their wide acceptance. In order to facilitate more frequent use of QSAR models in future DBP research, this article reviews the processes required for QSAR model development, summarizes recent trends in QSAR-DBP studies, and shares some important resources for QSAR development (e.g., free databases and QSAR programs). The paper follows the four steps of QSAR model development, i.e., data collection, descriptor filtration, algorithm selection, and model validation; and finishes by highlighting several research needs. Because QSAR models may have an important role in progressing our understanding of DBP issues, it is hoped that this paper will encourage their future use for this application. [ABSTRACT FROM AUTHOR]

Abstract A large number of optimization algorithms have been proposed. However, the no free lunch (NFL) theorems inform us that no algorithm can solve all types of optimization problems. An approach, which can suggest the most suitable algorithm for different types of problems, is valuable. In this paper, we propose an approach called sequential algorithm portfolio (SAP) which belongs to the inter-disciplinary fields of algorithm portfolio and algorithm selection. It uses a pre-trained predictor to predict the most suitable algorithm and a termination mechanism to automatically stop the optimization algorithms. The SAP is easy to implement and can incorporate any optimization algorithm. We experimentally compare SAP with two state-of-the-art algorithm portfolio approaches and single optimization algorithms. The result shows that SAP is a well-performing algorithm portfolio approach. [ABSTRACT FROM AUTHOR]

Abstract A wide range of clustering algorithms exists, most of them expose many hyperparameters, on which clustering partition quality depends. Simultaneous algorithm (model) selection and its hyperparameters optimization is considered to be a sophisticated task, which is known according to some sources as combined algorithm selection and hyperparameter optimization. In this paper, we focus on problem of selecting a clustering algorithm and its hyperparameter vector simultaneously given a dataset in order to achieve the best partition quality. We propose a method for selecting a proper clustering algorithm and its hyperparameter vector using reinforcement learning. Instead of tuning hyperparameters for all available clustering algorithms and selecting one showing the best performance, we make them to compete for time that they can use for optimizing their own hyperparameters. In our algorithm, we use a framework for multi-armed bandit problem, which is a special case of reinforcement learning. Each clustering algorithm is considered as an arm in the multi-armed bandit setting, while assigning a time budget to optimize hyperparameters of a clustering algorithm is considered as playing the corresponding arm. We conducted series of experiments for comparing out reinforcement learning approach to the classical exhaustive search approach. We conducted experiments on 20 datasets from UCI Repository such as Iris, haberman, krvskp, glass and other. We use 19 cluster validity indices to validate the clusters, built by selected and configured algorithm. As a hyperparameter optimization algorithm, we used SMAC. Our approach managed to improve model selection and hyperparameter optimization process, by sustaining the exploration-exploitation trade-off and spending available time budget more wisely. [ABSTRACT FROM AUTHOR]

Constraint Programming (CP) is a powerful technique for solving large-scale combinatorial problems. Solving a problem proceeds in two distinct phases: modelling and solving. Effective modelling has a huge impact on the performance of the solving process. Even with the advance of modern automated modelling tools, search spaces involved can be so vast that problems can still be difficult to solve. To further constrain the model, a more aggressive step that can be taken is the addition of streamliner constraints, which are not guaranteed to be sound but are designed to focus effort on a highly restricted but promising portion of the search space. Previously, producing effective streamlined models was a manual, difficult and time-consuming task. This paper presents a completely automated process to the generation, search and selection of streamliner portfolios to produce a substantial reduction in search effort across a diverse range of problems. The results demonstrate a marked improvement in performance for both Chuffed, a CP solver with clause learning, and lingeling, a modern SAT solver. [ABSTRACT FROM AUTHOR]

Algorithm selection (AS), selecting the algorithm best suited for a particular problem instance, is acknowledged to be a key issue to make the best out of algorithm portfolios. This paper presents a collaborative filtering approach to AS. Collaborative filtering, popularized by the Netflix challenge, aims to recommend the items that a user will most probably like, based on the previous items she liked, and the items that have been liked by other users. As first noted by Stern et al. [47] , algorithm selection can be formalized as a collaborative filtering problem, by considering that a problem instance “likes better“ the algorithms that achieve better performance on this particular instance. Two merits of collaborative filtering (CF) compared to the mainstream algorithm selection (AS) approaches are the following. Firstly, mainstream AS requires extensive and computationally expensive experiments to learn a performance model, with all algorithms launched on all problem instances, whereas CF can exploit a sparse matrix, with a few algorithms launched on each problem instance. Secondly, AS learns a performance model as a function of the initial instance representation, whereas CF builds latent factors to describe algorithms and instances, and uses the associated latent metrics to recommend algorithms for a specific problem instance. A main contribution of the proposed algorithm recommender Alors system is to handle the cold start problem – emitting recommendations for a new problem instance – through the non-linear modeling of the latent factors based on the initial instance representation, extending the linear approach proposed by Stern et al. [47] . The merits and generality of Alors are empirically demonstrated on the ASLib [6] and OpenML [53] benchmarks. [ABSTRACT FROM AUTHOR]

• The smart industry underpinning Industrial Internet of Things (IIoT). • A hot issue is merging traditional and IIoT concepts. • Machine learning (ML) methods in the IIoT vertical concept. • A detailed explanation of the ML methods. • Data science - hierarchical data processing at the edge, fog, and cloud levels. Smart power plants are no longer just futuristic ideas but are rapidly becoming a reality, and for this to be achievable, an application of artificial intelligence in Industrial Internet of Things (IIoT) concepts is necessary. This paper presents the place and role of Data Science (DS) and Machine Learning (ML) on edge, fog, and cloud levels of vertical IIoT concepts of power plants. A comprehensive functional analysis of edge, fog, and cloud levels has been done. Data analyzing, preprocessing, and processing are described on all levels separately. Limitations in signal conversion and data preprocessing at the edge level (edge computing), data processing and analysis at the fog level (fog computing), and information processing at the application and cloud levels (high-end and cloud computing) are described in detail. ML algorithms have been selected based on the particular management and control level. The proposed concepts represent management and maintenance improvements with minimal investment and avoidance of production downtime. [ABSTRACT FROM AUTHOR]

Algorithm selection is an important task in different domains of knowledge. Meta-learning treats this task by adopting a supervised learning strategy. Training examples in meta-learning (called meta-examples) are generated from experiments performed with a pool of candidate algorithms in a number of problems, usually collected from data repositories or synthetically generated. A meta-learner is then applied to acquire knowledge relating features of the problems and the best algorithms in terms of performance. In this paper, we address an important aspect in meta-learning which is to produce a significant number of relevant meta-examples. Generating a high quality set of meta-examples can be difficult due to the low availability of real datasets in some domains and the high computational cost of labelling the meta-examples. In the current work, we focus on the generation of meta-examples for meta-learning by combining: (1) a promising approach to generate new datasets (called datasetoids ) by manipulating existing ones; and (2) active learning methods to select the most relevant datasets previously generated. The datasetoids approach is adopted to augment the number of useful problem instances for meta-example construction. However not all generated problems are equally relevant. Active meta-learning then arises to select only the most informative instances to be labelled. Experiments were performed in different scenarios, algorithms for meta-learning and strategies to select datasets. Our experiments revealed that it is possible to reduce the computational cost of generating meta-examples, while maintaining a good meta-learning performance. [ABSTRACT FROM AUTHOR]

This paper considers the challenge of recognizing how the properties of a graph determine the performance of graph coloring algorithms. In particular, we examine how spectral properties of the graph make the graph coloring task easy or hard. We address the question of when an exact algorithm is likely to provide a solution in a reasonable computation time, when we are better off using a quick heuristic, and how the properties of the graph influence algorithm performance. A new methodology is introduced to visualize a collection of graphs in an instance space defined by their properties, with an optimal feature selection approach included to ensure that the instance space preserves the topology of an algorithm performance metric. In this instance space we can reveal how different algorithms perform across a variety of instance classes with various properties. We demonstrate the methodology using a sophisticated branch and bound exact algorithm, and the faster heuristic approaches of integer rounding of a linear programming relaxation of the graph coloring formulation, as well as a greedy degree-saturation heuristic. Our results demonstrate that spectral properties of the graph instances can provide useful descriptions of when each of these algorithms will provide the best solution for a given computational effort. [ABSTRACT FROM AUTHOR]

Abstract: Dynamic real-world applications that generate data continuously have introduced new challenges for the machine learning community, since the concepts to be learned are likely to change over time. In such scenarios, an appropriate model at a time point may rapidly become obsolete, requiring updating or replacement. As there are several learning algorithms available, choosing one whose bias suits the current data best is not a trivial task. In this paper, we present a meta-learning based method for periodic algorithm selection in time-changing environments, named MetaStream. It works by mapping the characteristics extracted from the past and incoming data to the performance of regression models in order to choose between single learning algorithms or their combination. Experimental results for two real regression problems showed that MetaStream is able to improve the general performance of the learning system compared to a baseline method and an ensemble-based approach. [Copyright &y& Elsevier]

Feature selection in real-world data mining problems is essential to make the learning task efficient and more accurate. Identifying the best feature selection algorithm, among the many available, is a complex activity that still relies heavily on human experts or some random trial-and-error procedure. Thus, the automated machine learning community has taken some steps towards the automation of this process. In this paper, we address the metalearning challenge of recommending feature selection algorithms by proposing a novel meta-feature engineering model. Our model considers a broad collection of meta-features that enable the study of the relationship between the dataset properties and the feature selection algorithm performance in terms of several criteria. We arrange the input meta-features into eight categories: (i) simple, (ii) statistical, (iii) information-theoretical, (iv) complexity, (v) landmarking, (vi) based on symbolic models, (vii) based on images, and (viii) based on complex networks (graphs). The target meta-features emerge from a multi-criteria performance measure, based on five individual performance indexes, that assesses feature selection methods grounded in information, distance, dependence, consistency, and precision measures. We evaluate our proposal using a recently developed framework that extracts the input meta-features from 213 benchmark datasets, and ranks the assessed feature selection algorithms, to fill in the target meta-features in meta-bases. This evaluation uses five state-of-the-art classification methods to induce recommendation models from meta-bases: C4.5, Random Forest, XGBoost, ANN, and SVM. The results showed that it is possible to reach an average accuracy of up to 90% applying our meta-feature engineering model. This work is the first to use an extensive empirical evaluation to provide a careful discussion of the strengths and limitations of more than 160 meta-features. These meta-features, while designed to aid the task of feature selection algorithm recommendation, can readily be employed in other metalearning scenarios. Therefore, we believe our findings are a valuable contribution to the fields of automated machine learning and data mining, as well as to the feature extraction and pattern recognition communities. • A novel meta-feature engineering model recommends feature selection algorithms. • The proposal obtains promising results from 213 datasets with hit rates of up to 90%. • Some simple, landmarking, image, and graph-based input meta-features highlighted. • A multi-criteria performance measure rigorously assesses candidate algorithms. • Chains of binary or multiclass classifiers can efficiently rank candidate algorithms. [ABSTRACT FROM AUTHOR]

• Method extracts a vectorial embedding representation of image classification tasks. • Method evaluated in a metalearning case study with six CNN algorithms and 45 datasets. • Feature vectors of datasets with similar difficulty are close in the embedding space. • The results overcame the baseline methods in all assessed performance measures. • Meta-learning with Dataset2Vec reached high precision, overcoming image descriptors. Convolutional Neural Networks (CNNs) have become the main solution for image classification tasks in different applications. Although several CNN architectures are available, there is no best architecture regardless the problem at hand. The selection of the most suitable CNN architecture is usually performed by trial and error, which may take much time and computational resources. Meta-learning (MtL) is a framework developed in machine learning to perform algorithm selection based on the meta-features of each task being solved. Such meta-features are usually descriptive characteristics extracted from the training dataset available in the task at hand. Despite the increasing attention of MtL for algorithm selection, its success strongly depends on defining relevant meta-features to represent the classification tasks of interest. This paper proposes the ImageDataset2Vec method for extracting meta-features to describe image classification datasets. ImageDataset2Vec adopts a pre-trained deep neural network to extract features from images datasets, embedding them in a single feature vector. The derived meta-features are employed by MtL to select CNN architectures for image classification. The proposed approach was evaluated for selecting among six CNN algorithms in 45 two-classes image datasets. The results showed that MtL using ImageDataset2Vec overcame different baseline methods, selecting the best possible CNN algorithm in 84.45 % of the datasets. Furthermore, the proposal was statistically equivalent to the ground truth when the best CNN is recommended, i.e., when MtL does not select the best CNN, it selects a competitive algorithm. These results show that the proposal was able to extract representative features from image datasets. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a sensor fusion framework for selecting online the most reliable logical sensors and the most suitable algorithm for fusing sensor data in a robot platform. The framework is rule-based, employing the concept of using the simplest sensor fusion algorithm with the most reliable sensors. The framework is realized by implementing measures that were developed to quantify online sensor performance. Statistical, histogram, time series and graphical analyses demonstrate the advantages of this new framework. [Copyright &y& Elsevier]

Abstract: This paper introduces a new method for learning algorithm evaluation and selection, with empirical results based on classification. The empirical study has been conducted among 8 algorithms/classifiers with 100 different classification problems. We evaluate the algorithms’ performance in terms of a variety of accuracy and complexity measures. Consistent with the No Free Lunch theorem, we do not expect to identify the single algorithm that performs best on all datasets. Rather, we aim to determine the characteristics of datasets that lend themselves to superior modelling by certain learning algorithms. Our empirical results are used to generate rules, using the rule-based learning algorithm C5.0, to describe which types of algorithms are suited to solving which types of classification problems. Most of the rules are generated with a high confidence rating. [Copyright &y& Elsevier]

• Features of initial solutions predict the performance of perturbation algorithms. • Low performance variation within portfolios makes algorithm selection difficult. • Hybrid selection models are useful for portfolios with low performance variation. • Accuracy is not a fair measure to evaluate the performance of algorithm selectors. In this paper, we present the construction process of a per-instance algorithm selection model to improve the initial solutions of Curriculum-Based Course Timetabling (CB-CTT) instances. Following the meta-learning framework, we apply a hybrid approach that integrates the predictions of a classifier and linear regression models to estimate and compare the performance of four meta-heuristics across different problem sub-spaces described by seven types of features. Rather than reporting the average accuracy, we evaluate the model using the closed SBS-VBS gap, a performance measure used at international algorithm selection competitions. The experimental results show that our model obtains a performance of 0.386, within the range obtained by per-instance algorithm selection models in other combinatorial problems. As a result of the process, we conclude that the performance variation between the meta-heuristics has a significant role in the effectiveness of the model. Therefore, we introduce statistical analyses to evaluate this factor within per-instance algorithm portfolios. [ABSTRACT FROM AUTHOR]