Search

Network function virtualization (NFV) brings great conveniences and benefits for the enterprises to outsource their network functions to the cloud datacenter. In this paper, we address the virtual network function (VNF) placement problem in cloud datacenter considering users’ service function chain requests (SFCRs). To optimize the resource utilization, we take two less-considered factors into consideration, which are the time-varying workloads, and the basic resource consumptions (BRCs) when instantiating VNFs in physical machines (PMs). Then the VNF placement problem is formulated as an integer linear programming (ILP) model with the aim of minimizing the number of used PMs. Afterwards, a Two-StAge heurisTic solution (T-SAT) is designed to solve the ILP. T-SAT consists of a correlation-based greedy algorithm for SFCR mapping (first stage) and a further adjustment algorithm for virtual network function requests (VNFRs) in each SFCR (second stage). Finally, we evaluate T-SAT with the artificial data we compose with Gaussian function and trace data derived from Google's datacenters. The simulation results demonstrate that the number of used PMs derived by T-SAT is near to the optimal results and much smaller than the benchmarks. Besides, it improves the network resource utilization significantly. [ABSTRACT FROM AUTHOR]

Cloud service providers offer virtual machines (VMs) as services to users over Internet. As VMs are running on physical machines (PMs), PM power consumption needs to be considered. Meanwhile, VMs running on the same PM share physical resources, and there exists great resource contention, which results in VM performance degradation. Therefore, how to place VMs to reduce PM power consumption and guarantee VM performance is still one major challenge. However, existing VMPs did not study VM performance degradation, so they could not guarantee VM performance. To solve the high power consumption and VMs performance degradation problems, this paper explores the balance between saving PM power and guaranteeing VM performance, and proposes a power-aware and performance-guaranteed VMP (PPVMP). First, we investigate the relationship between power consumption and CPU utilization to build a non-linear power model, which is helpful for the following VMP. Second, we construct VM performance models to present the VM performance degradation trend. Third, based on these models, we formulate VMP as a bi-objective optimization problem, which tries to minimize PM power consumption and guarantee VM performance. We then propose an algorithm based on ant colony optimization to solve it. Finally, the results show the efficiency of our algorithm. [ABSTRACT FROM AUTHOR]

Double auction-based pricing model is an efficient pricing model to balance users’ and providers’ benefits. Existing double auction mechanisms usually require both users and providers to bid with the unit price and the number of VMs. However, in practice users seldom know the exact number of VMs that meets their task requirements, which leads to users’ task requirements inconsistent with providers’ resource. In this paper, we propose a truthful double auction mechanism, including a matching process as well as a pricing and VM allocation scheme, to bridge users’ task requirements and providers’ resources in two-sided cloud markets. In the matching process, we design a cost-aware resource algorithm based on Lyapunov optimization techniques to precisely obtain the number of VMs that meets users’ task requirements. In the pricing and VM allocation scheme, we apply the idea of second-price auction to determine the final price and the number of provisioned VMs in the double auction. We theoretically prove our proposed mechanism is individual-rational, truthful and budget-balanced, and analyze the optimality of proposed algorithm. Through simulation experiments, the results show that the individual profits achieved by our algorithm are 12.35 and 11.02 percent larger than that of scale-out and greedy scale-up algorithms respectively for 90 percent of users, and the social welfare of our mechanism is only 7.01 percent smaller than that of the optimum mechanism in the worst case. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a unified framework for studying various cloud traffic management problems, ranging from geographical load balancing to backbone traffic engineering. We first abstract these real-world problems as a multi-facility resource allocation problem, and then present two distributed optimization algorithms by exploiting the special structure of the problem. Our algorithms are inspired by Alternating Direction Method of Multipliers (ADMM), enjoying a number of unique features. Compared to dual decomposition, they converge with non-strictly convex objective functions; compared to other ADMM-type algorithms, they not only achieve faster convergence under weaker assumptions, but also have lower computational complexity and lower message-passing overhead. The simulation results not only confirm these desirable features of our algorithms, but also highlight several additional advantages, such as scalability and fault-tolerance. [ABSTRACT FROM AUTHOR]

Contemporary cloud networks are being challenged by the rapid increase of user demands and growing concerns about global warming, due to their substantial energy consumption. This requires future data centers to be both energy efficient and sustainable, which calls for leveraging cutting-edge features and the flexibility provided by the modern smart grids. To fulfill those goals, this paper puts forward a systematic approach to designing energy-aware traffic-efficient geographical load balancing schemes for data-center networks that are not only optimal, but also computationally efficient and amenable to distributed implementation. Under this comprehensive approach, workload and power balancing schemes are designed jointly across the network, both delay-tolerant and interactive workloads are accommodated, novel smart-grid features such as energy storage units are incorporated to cope with renewables, and incentive pricing mechanisms are adopted in the design. To further account for the spatio-temporal variation of demands, energy prices and renewables, the task is formulated as a two-timescale stochastic optimization. Leveraging dual stochastic approximation and the fast iterative shrinkage-thresholding algorithm (FISTA), the proposed optimization is decomposed across time slots (first-stage) and data centers (second-stage). While the resultant online algorithm is strictly feasible and provably optimal under a Markovian assumption for the underlying random processes, extensive numerical tests further demonstrate that it also works well in real-data scenarios, where the underlying randomness is highly correlated across time. [ABSTRACT FROM PUBLISHER]

Increasingly, many bandwidth-intensive applications have been ported to the cloud platform. In practice, however, some disadvantages including equipment failures, bandwidth overload and long-distance transmission often damage the QoS about data availability, bandwidth provision and access locality respectively. While some recent solutions have been proposed to cope with one or two of disadvantages, but not all. Moreover, as the number of data objects scales, most of the current offline algorithms solving a constraint optimization problem suffer from low computational efficiency. To overcome these problems, in this paper we propose an approach that aims to make fully efficient use of the cloud resources to enable bandwidth-intensive applications to achieve the desirable level of SLA-specified QoS mentioned above cost-effectively and timely. First we devise a constraint-based model that describes the relationship among data object placement, user cells bandwidth allocation, operating costs and QoS constraints. Second, we use the distributed heuristic algorithm, called DREAM-L, that solves the model and produces a budget solution to meet SLA-specified QoS. Third, we propose an object-grouping technique that is integrated into DREAM-L , called DREAM-LG, to significantly improve the computational efficiency of our algorithm. The results of hundreds of thousands of simulation-based experiments demonstrate that DREAM-LG provides much better data availability, bandwidth provision and access locality than the state-of-the-art solutions at modest cloud operating costs and within a small and acceptable range of time. [ABSTRACT FROM PUBLISHER]

Scheduling and dispatching tools for high-performance computing (HPC) machines have the key role of mapping jobs to the available resources, trying to maximize performance and quality-of-service (QoS). Allocation and Scheduling in the general case are well-known NP-hard problems, forcing commercial schedulers to adopt greedy approaches to improve performance and QoS. Search-based approaches featuring the exploration of the solution space have seldom been employed in this setting, but mostly applied in off-line scenarios. In this paper, we present the first search-based approach to job allocation and scheduling for HPC machines, working in a production environment. The scheduler is based on Constraint Programming, an effective programming technique for optimization problems. The resulting scheduler is flexible, as it can be easily customized for dealing with heterogeneous resources, user-defined constraints and different metrics. We evaluate our solution both on virtual machines using synthetic workloads, and on the Eurora HPC with production workloads. Tests on a wide range of operating conditions show significant improvements in waitings and QoS in mid-tier HPC machines w.r.t state-of-the-art commercial rule-based dispatchers. Furthermore, we analyze the conditions under which our approach outperforms commercial approaches, to create a portfolio of scheduling algorithms that ensures robustness, flexibility and scalability. [ABSTRACT FROM AUTHOR]

Cloud computing provides a new opportunity for Video Service Providers (VSP) to running compute-intensive video applications in a cost effective manner. Under this paradigm, a VSP may rent virtual machines (VMs) from multiple geo-distributed datacenters that are close to video requestors to run their services. As user demands are difficult to predict and the prices of the VMs vary in different time and region, optimizing the number of VMs of each type rented from datacenters located in different regions in a given time frame becomes essential to achieve cost effectiveness for VSPs. Meanwhile, it is equally important to guarantee users' Quality of Experience (QoE) with rented VMs. In this paper, we give a systematic method called Dynamical Request Redirection and Resource Provisioning (DYRECEIVE) to address this problem. We formulate the problem as a stochastic optimization problem and design a Lyapunov optimization framework based online algorithm to solve it. Our method is able to minimize the long-term time average cost of renting cloud resources while maintaining the user QoE. Theoretical analysis shows that our online algorithm can produce a solution within an upper bound to the optimal solution achieved through offline computing. Extensive experiments shows that our method is adaptive to request pattern changes along time and outperforms existing algorithms. [ABSTRACT FROM AUTHOR]

This paper proposes a multi-objective evolutionary algorithm (MOEA)-based task scheduling approach for determining Pareto optimal solutions with simultaneous optimization of performance (P), energy (E ), and temperature (T). Our algorithm includes problem-specific solution encoding, determining the initial population of the solution space, and the genetic operators that collectively work on generating efficient solutions in fast turnaround time. Multiple schedules offer a diverse range of values for makespan, energy consumed, and peak temperature and thus present an efficient way of identifying trade-offs among the desired objectives, for a given application and machine pair. We also present a methodology for selecting one solution from the Pareto front given the user's preference. The proposed algorithm for scheduling tasks to cores achieves three-way optimization with fast turnaround time. The proposed algorithm is advantageous because it reduces both energy and temperature together rather than in isolation. We evaluate the proposed algorithm using implementation and simulation, and compare it with integer linear programming as well as with other scheduling algorithms that are energy- or thermal-aware. The time complexity of the proposed scheme is considerably better than the compared algorithms. [ABSTRACT FROM PUBLISHER]

This paper presents a new MapReduce cloud service model, Cura, for provisioning cost-effective MapReduce services in a cloud. In contrast to existing MapReduce cloud services such as a generic compute cloud or a dedicated MapReduce cloud, Cura has a number of unique benefits. First, Cura is designed to provide a cost-effective solution to efficiently handle MapReduce production workloads that have a significant amount of interactive jobs. Second, unlike existing services that require customers to decide the resources to be used for the jobs, Cura leverages MapReduce profiling to automatically create the best cluster configuration for the jobs. While the existing models allow only a per-job resource optimization for the jobs, Cura implements a globally efficient resource allocation scheme that significantly reduces the resource usage cost in the cloud. Third, Cura leverages unique optimization opportunities when dealing with workloads that can withstand some slack. By effectively multiplexing the available cloud resources among the jobs based on the job requirements, Cura achieves significantly lower resource usage costs for the jobs. Cura’s core resource management schemes include cost-aware resource provisioning, VM-aware scheduling and online virtual machine reconfiguration. Our experimental results using Facebook-like workload traces show that our techniques lead to more than 80 percent reduction in the cloud compute infrastructure cost with upto 65 percent reduction in job response times. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of distributed resource management for secondary users in Cognitive Radio Networks (CRNs), through a topology aware and frequency agile cross-layer approach. We exploit the theory of spatial processes and propose a Markov Random Field (MRF) based framework, which enables secondary CRN users to achieve efficient and viable mechanisms in the lower protocol stack layers by exchanging local only information. Specifically, through Gibbs sampling secondary users can optimize in a distributed and parallel manner their channel allocation, medium access and routing without resolving to otherwise computationally demanding optimization approaches. Through analysis and simulation we exhibit the efficacy of the proposed framework and show that a semi-parallel implementation can significantly reduce the required overhead cost compared to the sequential Gibbs sampling approach, while retaining a very close performance to the latter. We also study the emerging trade-offs by demonstrating the performance benefits in terms of channel assignment, medium access and data flow. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the cooperative resource sharing and pricing for the licensed Primary User (PU) and Cognitive Radio Networks (CRNs), where the PU jointly determines how to share its under-utilized radio resource with Secondary Users (SUs) and how to charge the SUs accordingly. Meanwhile, the SUs jointly determine how to utilize the shared radio resource from the PU and their preferred payments. Since both the PU and SUs expect to benefit from cooperation, we model their interactions as a Nash bargaining problem. Viewing the nonconvexity of bargaining problem, we first propose a two-step procedure to solve it efficiently. The two-step procedure explores the connection between the bargaining problem and its associated social optimization problem, and thus turns the original nonconvex bargaining problem into two consecutive convex optimization problems. We then propose two efficient algorithms, each with guaranteed convergence, to solve these two problems, respectively. Numerical results show that our proposed two-step procedure achieves the optimality of the bargaining problem with significantly reduced computational complexity. Also, our joint resource sharing and pricing scheme guarantees that each SU and PU can positively benefit from the cooperative bargaining, and the benefit is fairly allocated among them. [ABSTRACT FROM AUTHOR]

Cooperative communication (CC) has been proposed for achieving spatial diversity without requiring multiple antennas on the same node. Many efforts in exploiting the benefits of CC focus on improving the performance in terms of outage probability or channel capacity. However, the energy efficiency of CC, which is critical for the applications with energy constraints, has been little studied. In this paper, we study the lifetime maximization problem for multiple source-destination pairs using CC in multi-channel wireless networks by an optimal dynamic allocation of resources in terms of power, channel, cooperative relay, and transmission time fraction. We prove it NP-hard and formulate it as a mixed-integer nonlinear programming (MINLP) problem, which is then transformed into a mixed-integer linear programming (MILP) problem using linearization and reformulation techniques. By exploiting several problem-specific characteristics, a time-efficient branch-and-bound algorithm is proposed to solve the MILP problem. Extensive simulations are conducted to show that the proposed algorithm can significantly improve the performance of energy efficiency over existing solutions. [ABSTRACT FROM AUTHOR]

The analysis of real-world complex networks has been the focus of recent research. Detecting communities helps in uncovering their structural and functional organization. Valuable insight can be obtained by analyzing the dense, overlapping, and highly interwoven $(k)$-clique communities. However, their detection is challenging due to extensive memory requirements and execution time. In this paper, we present a novel, parallel $(k)$-clique community detection method, based on an innovative technique which enables connected components of a network to be obtained from those of its subnetworks. The novel method has an unbounded, user-configurable, and input-independent maximum degree of parallelism, and hence is able to make full use of computational resources. Theoretical tight upper bounds on its worst case time and space complexities are given as well. Experiments on real-world networks such as the Internet and the World Wide Web confirmed the almost optimal use of parallelism (i.e., a linear speedup). Comparisons with other state-of-the-art $(k)$-clique community detection methods show dramatic reductions in execution time and memory footprint. An open-source implementation of the method is also made publicly available. [ABSTRACT FROM PUBLISHER]

In this paper, we introduce the Bodyguard Allocation Problem (BAP) game, that illustrates the behavior of processes with contradictory individual goals in distributed systems. In particular, the game deals with the conflict of interest between two classes of processes that maximize/minimize their distance to a special process called the root. A solution of the BAP game represents a rooted spanning tree in which there exists a condition of equilibrium with maximum social welfare. We analyze the inefficiency of equilibria of the game based on both a completely cooperative and noncooperative approach. Additionally, we design two algorithms, CBAP and DBAP, that provide approximated solutions for the BAP game. We prove that both algorithms always terminate in a configuration with equilibrium and we analyze their running time based on the approach of cooperation used. We perform experimental simulations to compare the overall quality of equilibria obtained by the proposed algorithms. [ABSTRACT FROM AUTHOR]

The increasing popularity of large-scale distributed applications in datacenters has led to the growing demand of distributed application state monitoring. These application state monitoring tasks often involve collecting values of various status attributes from a large number of nodes. One challenge in such large-scale application state monitoring is to organize nodes into a monitoring overlay that achieves monitoring scalability and cost effectiveness at the same time. In this paper, we present REMO, a REsource-aware application state MOnitoring system, to address the challenge of monitoring overlay construction. REMO distinguishes itself from existing works in several key aspects. First, it jointly considers intertask cost-sharing opportunities and node-level resource constraints. Furthermore, it explicitly models the per-message processing overhead which can be substantial but is often ignored by previous works. Second, REMO produces a forest of optimized monitoring trees through iterations of two phases. One phase explores cost-sharing opportunities between tasks, and the other refines the tree with resource-sensitive construction schemes. Finally, REMO also employs an adaptive algorithm that balances the benefits and costs of overlay adaptation. This is particularly useful for large systems with constantly changing monitoring tasks. Moreover, we enhance REMO in terms of both performance and applicability with a series of optimization and extension techniques. We perform extensive experiments including deploying REMO on a BlueGene/P rack running IBM's large-scale distributed streaming system—System S. Using REMO in the context of collecting over 200 monitoring tasks for an application deployed across 200 nodes results in a 35-45 percent decrease in the percentage error of collected attributes compared to existing schemes. [ABSTRACT FROM PUBLISHER]

To reduce the cost of infrastructure and electrical energy, enterprise datacenters consolidate workloads on the same physical hardware. Often, these workloads comprise both transactional and long-running analytic computations. Such consolidation brings new performance management challenges due to the intrinsically different nature of a heterogeneous set of mixed workloads, ranging from scientific simulations to multitier transactional applications. The fact that such different workloads have different natures imposes the need for new scheduling mechanisms to manage collocated heterogeneous sets of applications, such as running a web application and a batch job on the same physical server, with differentiated performance goals. In this paper, we present a technique that enables existing middleware to fairly manage mixed workloads: long running jobs and transactional applications. Our technique permits collocation of the workload types on the same physical hardware, and leverages virtualization control mechanisms to perform online system reconfiguration. In our experiments, including simulations as well as a prototype system built on top of state-of-the-art commercial middleware, we demonstrate that our technique maximizes mixed workload performance while providing service differentiation based on high-level performance goals. [ABSTRACT FROM PUBLISHER]

Dynamic virtual server provisioning is critical to quality-of-service assurance for multitier Internet applications. In this paper, we address three important challenging problems. First, we propose an efficient server provisioning approach on multitier clusters based on an end-to-end resource allocation optimization model. It is to minimize the number of virtual servers allocated to the system while the average end-to-end response time guarantee is satisfied. Second, we design a model-independent fuzzy controller for bounding an important performance metric, the 90th-percentile response time of requests flowing through the multitier architecture. Third, to compensate for the latency due to the dynamic addition of virtual servers, we design a self-tuning component that adaptively adjusts the output scaling factor of the fuzzy controller according to the transient behavior of the end-to-end response time. Extensive simulation results, using two representative customer behavior models in a typical three-tier web cluster, demonstrate that the provisioning approach is able to significantly reduce the number of virtual servers allocated for the performance guarantee compared to an existing representative approach. The approach integrated with the model-independent self-tuning fuzzy controller can efficiently assure the average and the 90th-percentile end-to-end response time guarantees on multitier clusters. [ABSTRACT FROM AUTHOR]

Lossy compression techniques are commonly used by long-term data-gathering applications that attempt to identify trends or other interesting patterns in an entire system since a data packet need not always be completely and immediately transmitted to the sink. In these applications, a nonterminal sensor node jointly encodes its own sensed data and the data received from its nearby nodes. The tendency for these nodes to have a high spatial correlation means that these data packets can be efficiently compressed together using a rate-distortion strategy. This paper addresses the optimal rate-distortion allocation problem, which determines an optimal bit rate of each sensor based on the target overall distortion to minimize the network transmission cost. We propose an analytically optimal rate-distortion allocation scheme, and we also extend it to a distributed version. Based on the presented allocation schemes, a greedy heuristic algorithm is proposed to build the most efficient data transmission structure to further reduce the transmission cost. The proposed methods were evaluated using simulations with real-world data sets. The simulation results indicate that the optimal allocation strategy can reduce the transmission cost to 6\sim 15\% of that for the uniform allocation scheme. [ABSTRACT FROM AUTHOR]