Search

Cloud computing is a promising paradigm for deploying applications due to its large resource offerings on a pay-as-you-go basis. This paper examines the problem of determining the most cost-effective provisioning of a multi-tenant database system as a service over public clouds. The authors formulate the problem of resource provisioning, and then define a framework to solve it. Their framework uses heuristic based algorithms to select cost-effective configurations. The algorithms can optionally balance resource costs against penalties incurred from the violation of Service Level Agreements (SLAs) or opt for non SLA violating configurations. The specific resource demands on the virtual machines for a workload and SLAs are accounted for by the performance and cost models, which are used to predict performance and expected cost respectively. The work validates our approach experimentally using workloads based on standard TPC database benchmarks in the Amazon EC2 cloud.

Data analytics applications are well-suited for a cloud environment. In this paper we examine the problem of provisioning resources in a public cloud to execute data analytic workloads. The goal of our provisioning method is to determine the most cost-effective configuration for a given data analytic workload. Provisioning a workload in a public cloud environment faces several challenges: it is difficult to develop accurate performance prediction models using standard methods; the space of possible configurations is very large so exact solutions cannot be efficiently determined, and the mix and intensity of query classes in a workload vary dynamically over time. We provide a formulation of the provisioning problem and then define a framework to solve the problem. Our framework contains a cost model to predict the cost of executing a workload on a configuration and a method of selecting configurations. The cost model balances resource costs and penalties from SLAs. The specific resource demands and frequencies are accounted for by queueing network models of the Virtual Machines (VMs), which are used to predict performance. We evaluate our approach experimentally using sample data analytic workloads on Amazon EC2.

In the new digital age, the pace and volume of growing transportation related data is exceeding our ability to manage and analyze it. In this position paper, we present a data engine, Godzilla, to ingest real-time traffic data and support analytic and data mining over traffic data. Godzilla is a multi-cluster approach to handle large volumes of growing data, changing workloads and varying number of users. The data originates at multiple sources, and consists of multiple types. Meanwhile, the workloads belong to three camps, namely batch processing, interactive queries and graph analysis. Godzilla support multiple language abstractions from scripting to SQL-like language.

The rate of data growth in many domains is straining our ability to manage and analyze it. Cloud computing appears as a promising platform for data-intensive computing because it offers “infinite” resources on demand, and on a pay-as-you-go basis. Surprisingly, we observe that public clouds are not being used for “serious” data-processing on a continuous basis except by the cloud vendors. We primarily attribute this observation to the long data transfer times over wide-area networks between the clients and the centralized public clouds. We introduce the idea of a near-cloud that brings a public cloud in the close proximity of a user to overcome the data transfer bottlenecks. The near-clouds also provide a unique opportunity to localize the privacy and security policies to boost confidence of a user in using shared resources for data processing. To the cloud providers, near-cloud ensures a minimum constant stream of revenue from dedicated clients.

The cloud computing paradigm provides the "illusion" of infinite resources and, therefore, becomes a promising candidate for large-scale data-intensive computing. In this paper, we explore experiment-driven performance models for data-intensive workloads executing in an infrastructure-as-a-service (IaaS) public cloud. The performance models help in predicting the workload behaviour, and serve as a key component of a larger framework for resource provisioning in the cloud. We determine a suitable prediction technique after comparing popular regression methods. We also enumerate the variables that impact variance in the workload performance in a public cloud. Finally, we build a performance model for a multi-tenant data service in the Amazon cloud. We find that a linear classifier is sufficient in most cases. On a few occasions, a linear classifier is unsuitable and non-linear modeling is required, which is time consuming. Consequently, we recommend that a linear classifier be used in training the performance model in the first instance. If the resulting model is unsatisfactory, then non-linear modeling can be carried out in the next step.

The amount of data available for many areas is increasing faster than our ability to process it. The promise of “infinite” resources given by the cloud computing paradigm has led to recent interest in exploiting clouds for large-scale data intensive computing. Data-intensive computing presents new challenges for systems management in the cloud including new processing frameworks, such as MapReduce, and costs inherent with large data sets in distributed environments. Workload management, an important component of systems management, is the discipline of effectively managing, controlling and monitoring “workflow” across computing systems. This chapter examines the state-of-the-art of workload management for data-intensive computing in clouds. A taxonomy is presented for workload management of data-intensive computing in the cloud and use the taxonomy to classify and evaluate current workload management mechanisms.