Search

In this paper, we present a novel clustering approach that eliminates the need for predefined cluster centres and cluster counts, addressing common limitations in traditional clustering algorithms. We achieve this by allowing each data point within the problem space to serve as a potential cluster centre, determined by the presence of nearby data points. Three distinct algorithms are introduced: i) Identifying valid data points for clustering by assessing their presence within the specified vicinity of each potential cluster centre, allowing data points to belong to multiple clusters, ii) Calculating the fuzzy membership of shared data points among the clusters and iii) Retaining shared data points in the cluster with the highest membership and removing them from others. A comprehensive evaluation of this method using synthetic data and benchmark datasets from well-known problems demonstrates promising performance. Our approach offers increased flexibility, ease of use, and enhances the fidelity of cluster centres to the underlying data distribution. [ABSTRACT FROM AUTHOR]

With the deployment of workflow and other applications, cloud computing is accessible and offers assistance for optimizing workflow execution and enhancing performance. Existing research, however, tends to disregard the influence of dataset migration on workflow execution and focuses more on task execution time. This study suggests a new model for the problem of data-intensive workflow execution. Firstly, according to the structure of the workflow scheduling problem, it is divided into two sub-problems: data placement and task scheduling. The two sub-problems interact with each other and a bi-level optimum model is established. By seeking a better allocation strategy for the dataset placement and then seeking the best task-scheduling solution. Secondly, an improved multitasking bi-level evolutionary algorithm (IM-BLEA) is proposed. When dealing with the lower-level optimization problem (LLOP), offspring are selected by sorting individuals by their performance and overall performance in the population, and this environmental selection enhances the diversity and searchability of the population. Finally, compared with the other multitasking algorithm, IM-BLEA has good performance. Simulation results based on real scientific workflows show that the algorithm improves the values of transfer time and number of selected data centers by 56% and 10% compared to the comparison algorithm. • A bi-level optimization model in data-intensive workflows is proposed. • The upper-level and lower-level problems are data placement and task scheduling. • Multitasking evolutionary algorithms on bi-level optimization model. • Based on the modeling of the problem, an algorithm (IM-BLEA) is proposed. • Simulation experiments based on scientific workflows to validate algorithms. [ABSTRACT FROM AUTHOR]

Resource sharing is an inherent characteristic of cloud data centers. Virtual Machines (VMs) and/or Containers that are co-located in the same physical server often compete for resources leading to interference. The noisy neighbor’s effect refers to an anomaly caused by a VM/container limiting resources accessed by another one. Our main contribution is an online, lightweight and application-agnostic solution for anomaly detection, that follows an unsupervised approach. It is based on comparing models for different lags: Dirichlet Process Gaussian Mixture Models to characterize the resource usage profile of the application, and distance measures to score the similarity among models. An alarm is raised when there is an abrupt change in short-term lag (i.e. high distance score for short-term models), while the long-term state remains constant. We test the algorithm for different cloud workloads: websites, periodic batch applications, Spark-based applications, and Memcached server. We are able to detect anomalies in the CPU and memory resource usage with up to 82–96% accuracy (recall) depending on the scenario. Compared to other baseline methods, our approach is able to detect anomalies successfully, while raising low number of false positives, even in the case of applications with unusual normal behavior (e.g. periodic). Experiments show that our proposed algorithm is a lightweight and effective solution to detect noisy neighbor effect without any historical info about the application, that could also be potentially applied to other kind of anomalies. [ABSTRACT FROM AUTHOR]

• An improved genetic algorithm is proposed for permutation-based problems. • A hybrid virtual machine placement algorithm is proposed for cloud data centers. • The proposed hybrid algorithm significantly reduces the power consumption. • The proposed hybrid algorithm achieves high resources utilization rates. The high energy consumption of cloud data centers presents a significant challenge from both economic and environmental perspectives. Server consolidation using virtualization technology is widely used to reduce the energy consumption rates of data centers. Efficient Virtual Machine Placement (VMP) plays an important role in server consolidation technology. VMP is an NP-hard problem for which optimal solutions are not possible, even for small-scale data centers. In this paper, a hybrid VMP algorithm is proposed based on another proposed improved permutation-based genetic algorithm and multidimensional resource-aware best fit allocation strategy. The proposed VMP algorithm aims to improve the energy consumption rate of cloud data centers through minimizing the number of active servers that host Virtual Machines (VMs). Additionally, the proposed VMP algorithm attempts to achieve balanced usage of the multidimensional resources (CPU, RAM, and Bandwidth) of active servers, which in turn, reduces resource wastage. The performance of both proposed algorithms are validated through intensive experiments. The obtained results show that the proposed improved permutation-based genetic algorithm outperforms several other permutation-based algorithms on two classical problems (the Traveling Salesman Problem and the Flow Shop Scheduling Problem) using various standard datasets. Additionally, this study shows that the proposed hybrid VMP algorithm has promising energy saving and resource wastage performance compared to other heuristics and metaheuristics. Moreover, this study reveals that the proposed VMP algorithm achieves a balanced usage of the multidimensional resources of active servers while others cannot. [ABSTRACT FROM AUTHOR]