Search

The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is a commonly used iterative optimisation heuristic for optimising black-box functions. CMA-ES comes in many flavours with different configuration settings. In this work, we investigate whether CMAES suffers from structural bias and which modules and parameters affect the strength and type of structural bias. Structural bias occurs when an algorithm or a component of the algorithm biases the search towards a specific direction in the search space irrespective of the objective function. In addition to this investigation, we propose a method to assess the relationship between structural bias and the performance of configurations with different types of bias on the BBOB suite of benchmark functions. Surprisingly for such a popular algorithm, 90.3% of the 1 620 CMA-ES configurations were found to have Structural Bias. Some interesting patterns between module settings and bias types are presented and further insights are discussed.

Bayesian optimization is often used to optimize expensive black box optimization problems with long simulation times. Typically Bayesian optimization algorithms propose one solution per iteration. The downside of this strategy is the sub-optimal use of available computing power. To efficiently use the available computing power (or a number of licenses etc.) we introduce a multi-point acquisition function for parallel efficient multi-objective optimization algorithms. The multi-point acquisition function is based on the hypervolume contribution of multiple solutions simultaneously, leading to well spread solutions along the Pareto frontier. By combining this acquisition function with a constraint handling technique, multiple feasible solutions can be proposed and evaluated in parallel every iteration. The hypervolume and feasibility of the solutions can easily be estimated by using multiple cheap radial basis functions as surrogates with different configurations. The acquisition function can be used with different population sizes and even for one shot optimization. The strength and generalizability of the new acquisition function is demonstrated by optimizing a set of black box constraint multi-objective problem instances. The experiments show a huge time saving factor by using our novel multi-point acquisition function, while only marginally worsening the hypervolume after the same number of function evaluations.

Differential Evolution is a popular optimisation method with a small number of parameters. However, different hyper-parameters and Differential Evolution variants such as different mutation operators and the F and Cr parameter may introduce structural bias. Structural bias is a form of bias where artefacts in the algorithm lead to a preference to particular regions in the search space regardless of the objective function. Many algorithm configurations suffer from structural bias, but it is very hard to automatically detect it and even harder to detect what kind of structural bias is involved and what component might be the cause of it. A comprehensive study of the occurrence and type of structural bias in Differential Evolution configurations has not yet been carried out till now. In this chapter, we systematically evaluate 10980 Differential Evolution configurations on structural bias with the open-source BIAS toolbox. Using this toolbox we identify which configurations cause bias and what type of bias it is. In addition, we analyse the results to make clear recommendations on which components and parameters can be used in Differential Evolution to ensure unbiased behaviour within reasonable computational budget.

Kriging or Gaussian Process Regression is applied in many fields as a non-linear regression model as well as a surrogate model in the field of evolutionary computation. However, the computational and space complexity of Kriging, that is cubic and quadratic in the number of data points respectively, becomes a major bottleneck with more and more data available nowadays. In this paper, we propose a general methodology for the complexity reduction, called cluster Kriging, where the whole data set is partitioned into smaller clusters and multiple Kriging models are built on top of them. In addition, four Kriging approximation algorithms are proposed as candidate algorithms within the new framework. Each of these algorithms can be applied to much larger data sets while maintaining the advantages and power of Kriging. The proposed algorithms are explained in detail and compared empirically against a broad set of existing state-of-the-art Kriging approximation methods on a well-defined testing framework. According to the empirical study, the proposed algorithms consistently outperform the existing algorithms. Moreover, some practical suggestions are provided for using the proposed algorithms., Submitted to IEEE Computational Intelligence Magazine for review

This paper proposes a novel Self-Adaptive algorithm for Multi-Objective Constrained Optimization by using Radial Basis Function Approximations, SAMO-COBRA. The algorithm automatically determines the best Radial Basis Function-fit as surrogates for the objectives as well as the constraints, to find new feasible Pareto-optimal solutions. The algorithm also uses hyper-parameter tuning on the fly to improve its local search strategy. In every iteration one solution is added and evaluated, resulting in a strategy requiring only a small number of function evaluations for finding a set of feasible solutions on the Pareto frontier. The proposed algorithm is compared to a wide set of other state-of-the-art algorithms (NSGA-II, NSGA-III, CEGO, SMES-RBF) on 18 constrained multi-objective problems. In the experiments we show that our algorithm outperforms the other algorithms in terms of achieved Hypervolume after given a fixed small evaluation budget. These results suggest that SAMO-COBRA is a good choice for optimizing constrained multi-objective optimization problems with expensive function evaluations.

Automated symmetry detection is still a difficult task in 2021. However, it has applications in computer vision, and it also plays an important part in understanding art. This paper focuses on aiding the latter by comparing different state-of-the-art automated symmetry detection algorithms. For one of such algorithms aimed at reflectional symmetries, we propose postprocessing improvements to find localised symmetries in images, improve the selection of detected symmetries and identify another symmetry type (rotational). In order to detect rotational symmetries, we contribute a machine learning model which detects rotational symmetries based on provided reflection symmetry axis pairs. We demonstrate and analyze the performance of the extended algorithm to detect localised symmetries and the machine learning model to classify rotational symmetries.

This contribution shows how, in the preliminary design stage, naval architects can make more informed decisions by using machine learning. In this ship design phase, little information is available, and decisions need to be made in a limited amount of time. However, it is in the preliminary design phase where the most influential decisions are made regarding the global dimensions, the machinery, and therefore the performance and costs. In this paper it is shown that a machine learning algorithm trained with data from reference vessels are more accurate when estimating key performance indicators compared to existing empirical design formulas. Finally, the combination of the trained models with optimization algorithms shows to be a powerful tool for finding Pareto-optimal designs from which the naval architect can learn.

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link. Heuristic optimisation algorithms are in high demand due to the overwhelming amount of complex optimisation problems that need to be solved. The complexity of these problems is well beyond the boundaries of applicability of exact optimisation algorithms and therefore require modern heuristics to find feasible solutions quickly. These heuristics and their effects are almost always evaluated and explained by particular problem instances. In previous works, it has been shown that many such algorithms show structural bias, by either being attracted to a certain region of the search space or by consistently avoiding regions of the search space, on. special test function designed to ensure uniform 'exploration' of the domain. In this paper, we analyse the emergence of such structural bias for Differential Evolution (DE) configurations and, specifically, the effect of different mutation, crossover and correction strategies. We also analyse the emergence of the structural bias during the run-time of each algorithm. We conclude with recommendations of which configurations should be avoided in order to run DE unbiased.

Industrial manufacturing processes, such as the production of steel or the stamping of car body parts, are complex semi-batch processes with many process steps, machine parameters and quality indicators. To optimize these complex processes, for example by reducing the number of defects or increasing the throughput, a great number of requirements need to be taken into consideration. In this dissertation a framework for monitoring and optimizing these complex industrial processes is presented. The framework is specifically tailored to the production processes of Tata Steel and BMW Group. Both are industrial partners of the PROMIMOOC project. The framework consists of several components of which; preprocessing, outlier detection, predictive modeling and optimization are the main technical components that are the focus of this work. For each of these components a possible implementation is proposed and the challenges in implementing these components in an industrial manufacturing setting are discussed

Predictive maintenance (PdM) is a maintenance policy that uses the past, current, and prognosticated health condition of an asset to predict when timely maintenance should occur. PdM overcomes challenges of more conservative policies, such as corrective or scheduled maintenance. The remaining useful life (RUL) is a critical notion in PdM that determines the time remaining until a system is no longer useful and requires maintenance. Among the approaches employed to estimate the RUL, data-driven PdM methods have shown to be a good candidate due to their (mostly) domain-agnostic nature and broad applicability mos on the asset’s generated data. Nevertheless, there are various challenges to consider in data-driven PdM, such as algorithm selection, hyperparameter optimization, and uncertainty of the RUL estimation. This thesis proposes solutions and frameworks for these challenges using simulated datasets. We furthermore dive into scheduling optimization which is the next step in PdM and point towards the importance of understanding the data generating process in PdM using real-world data. Finally, we show how a method originally developed for PdM in the automotive industry can lend itself to the medical domain, exhibiting the significance of knowledge-transfer and the versatility of data-driven methods.

In design optimization problems, engineers typically handcraft design representations based on personal expertise, which leaves a fingerprint of the user experience in the optimization data. Thus, learning this notion of experience as transferrable design features has potential to improve the performance of similar, yet more challenging, design optimization problems. However, engineering design data are unstructured, high-dimensional and often have no canonical representation, which poses several challenges for machine learning algorithms. In this thesis, geometric deep learning techniques, in particular 3D point cloud autoencoders, are utilized to learn novel shape-generative models from engineering optimization data. Through different sets of experiments, it is shown that these autoencoders are scalable to high-dimensional engineering models and have comparable optimization performance to state-of-the-art representations. Furthermore, a novel network feature visualization technique is proposed, which provides a geometric interpretation of the knowledge stored in the network and allows one to select sub-sets of degrees of freedom to modify and optimize shapes. Due to the domain agnosticism of the autoencoders’ latent space, the learned representations are exploited in multi-task optimization problems to enable knowledge transfer between tasks and foster commonality between the optimized shapes. Finally, to improve the state of readiness of the 3D models generated by the point cloud autoencoder for engineering simulations, a novel architecture is proposed: Point2FFD. The novel architecture learns to generate 3D polygonal meshes based on input 3D point clouds and a set of existing handcrafted mesh templates parameterized with free-form deformation. Based on shape-generative and optimization experiments, it is shown that Point2FFD generates 3D models with better overall quality than state-of-the-art point cloud (variational) autoencoders and improves the quality of designs in vehicle aerodynamic optimization problems.

It is a common technique in global optimization with expensive black-box functions to learn a surrogate-model of the response function from past evaluations and use it to decide on the location of future evaluations.In surrogate-model-assisted optimization, selecting the right modeling technique without preliminary knowledge about the objective function can be challenging. It might be beneficial if the algorithm trains many different surrogate models and selects the model with the smallest training error. This approach is known as model selection.In this thesis, a generalization of this approach is developed. Instead of choosing a single model, the optimal convex combinations of model predictions is used to combine surrogate models into one more accurate ensemble surrogate model.This approach is studied in a fundamental way, by first evaluating minimalistic ensembles of only two surrogate models in detail and then proceeding to ensembles with more surrogate models.Finally, the approach is adopted and evaluated in the context of sequential parameter optimization. Besides discussing the general strategy, the optimal frequency of learning the convex combination of weights is investigated.The results provide insights into the performance, scalability, and robustness of the approach.