Search

For any machine learning model, finding the optimal hyperparameter setting has a direct and significant impact on the model's performance. In this paper, we discuss different types of hyperparameter optimization techniques. We compare the performance of some of the hyperparameter optimization techniques on image classification datasets with the help of AutoML models. In particular, the paper studies Bayesian optimization in depth and proposes the use of genetic algorithm, differential evolution and covariance matrix adaptation—evolutionary strategy for acquisition function optimization. Moreover, we compare these variants of Bayesian optimization with conventional Bayesian optimization and observe that the use of covariance matrix adaptation—evolutionary strategy and differential evolution improves the performance of standard Bayesian optimization. We also notice that Bayesian optimization tends to perform poorly when genetic algorithm is used for acquisition function optimization. [ABSTRACT FROM AUTHOR]

Knowledge Graphs represent real-world facts and are used in several applications; however, they are often incomplete and have many missing facts. Link prediction is the task of completing these missing facts from existing ones. Embedding models based on Tensor Factorization attain state-of-the-art results in link prediction. Nevertheless, the embeddings they produce can not be easily interpreted. Inspired by previous work on word embeddings, we propose inducing sparsity in the bilinear tensor factorization model, RESCAL, to build interpretable Knowledge Graph embeddings. To overcome the difficulties that stochastic gradient descent has when producing sparse solutions, we add l 1 regularization to the learning objective by using the generalized Regularized Dual Averaging online optimization algorithm. The proposed method substantially improves the interpretability of the learned embeddings while maintaining competitive performance in the standard metrics. [ABSTRACT FROM AUTHOR]

A displacement aggregation strategy is proposed for the curvature pairs stored in a limited-memory BFGS (a.k.a. L-BFGS) method such that the resulting (inverse) Hessian approximations are equal to those that would be derived from a full-memory BFGS method. This means that, if a sufficiently large number of pairs are stored, then an optimization algorithm employing the limited-memory method can achieve the same theoretical convergence properties as when full-memory (inverse) Hessian approximations are stored and employed, such as a local superlinear rate of convergence under assumptions that are common for attaining such guarantees. To the best of our knowledge, this is the first work in which a local superlinear convergence rate guarantee is offered by a quasi-Newton scheme that does not either store all curvature pairs throughout the entire run of the optimization algorithm or store an explicit (inverse) Hessian approximation. Numerical results are presented to show that displacement aggregation within an adaptive L-BFGS scheme can lead to better performance than standard L-BFGS. [ABSTRACT FROM AUTHOR]

In data mining, mining high utility itemset (HUI) is one among the recent thrust area that receives several approaches for solving it in an effective manner. In the past decade, addressing optimization problems using evolutionary algorithms are an unavoidable strategy due to its convergence towards optimal solution within the stipulated time. The results of evolutionary algorithms on various optimization problems are far effective when compared to the exhaustive approaches with respect to computational time. The problem with HUI is discovering a set of items from a transactional database that possess high level of utility when compared with other distinctive sets. This problem becomes harder while addressing the count of items in the database while its higher and computational time to solve this problem using exhaustive search becomes exponential as proposition of items in transaction database increases. In this paper, an optimization model based on the biological behaviour of grey wolf is proposed; the model namely grey wolf optimization algorithm is used to solve HUI using five different Boolean operations. The proposed model is evaluated using standard performance metrics over synthetic datasets and real-world datasets. The proposed model results are then compared with recent HUIM models to show the significance. [ABSTRACT FROM AUTHOR]

The complexity of plug-in hybrid-electric vehicles (PHEVs) motivates the simultaneous integration of component design and supervisory control strategy decisions. Methods from combined optimal design and control (co-design) are generally used to manage such integrated system design decisions. Although several studies have investigated the PHEV powertrain co-design problem, the impact of key vehicle performance criteria such as 0–60 mph acceleration time and all-electric-range (AER) has rarely been explicitly included in such system co-design problems. This is problematic as these vehicle performance criteria strongly affect component sizing and control strategy in a way that a non-performance-based co-design solution could become sub-optimal. Therefore, this study addresses this issue by formulating and solving a PHEV powertrain co-design problem that explicitly includes vehicle-level performance constraints. In particular, a three-phase, PHEV powertrain co-design problem is solved to simultaneously identify the optimal supervisory control strategies during the acceleration performance and standard duty cycle phases, along with the optimal component designs spanning all phases (acceleration performance, standard duty cycle, and AER performance) such that the vehicle powertrain cost is minimized. A relatively new, balanced co-design approach known as multidisciplinary dynamic system design optimization (MDSDO) is used to solve the problem. The optimal design and standard duty cycle supervisory control trajectories are compared to the solution of a non-performance-based co-design problem. The results indicate that the formal inclusion of vehicle performance criteria in a co-design problem significantly affects component design and supervisory control strategies. [ABSTRACT FROM AUTHOR]

Multi-robot task allocation determines the task sequence and distribution for a group of robots in multi-robot systems, which is one of constrained combinatorial optimization problems and more complex in case of cooperative tasks because they introduce additional spatial and temporal constraints. To solve multi-robot task allocation problems with cooperative tasks efficiently, a subpopulation-based genetic algorithm, a crossover-free genetic algorithm employing mutation operators and elitism selection in each subpopulation, is developed in this paper. Moreover, the impact of mutation operators (swap, insertion, inversion, displacement, and their various combinations) is analyzed when solving several industrial plant inspection problems. The experimental results show that: (1) the proposed genetic algorithm can obtain better solutions than the tested binary tournament genetic algorithm with partially mapped crossover; (2) inversion mutation performs better than other tested mutation operators when solving problems without cooperative tasks, and the swap-inversion combination performs better than other tested mutation operators/combinations when solving problems with cooperative tasks. As it is difficult to produce all desired effects with a single mutation operator, using multiple mutation operators (including both inversion and swap) is suggested when solving similar combinatorial optimization problems. [ABSTRACT FROM AUTHOR]