Search

Scalability of distributed deep learning (DL) training with parameter server (PS) architecture is often communication constrained in large clusters. There are recent efforts that use a layer by layer strategy to overlap gradient communication with backward computation so as to reduce the impact of communication constraint on the scalability. However, the approaches could bring significant overhead in gradient communication. Meanwhile, they cannot be effectively applied to the overlap between parameter communication and forward computation. In this article, we propose and develop iPart, a novel approach that partitions communication and computation in various partition sizes to overlap gradient communication with backward computation and parameter communication with forward computation. iPart formulates the partitioning decision as an optimization problem and solves it based on a greedy algorithm to derive communication and computation partitions. We implement iPart in the open-source DL framework BigDL and perform evaluations with various DL workloads. Experimental results show that iPart improves the scalability of a cluster of 72 nodes by up to 94 percent over the default PS and 52 percent over the layer by layer strategy. [ABSTRACT FROM AUTHOR]

In virtualization environments, static memory allocation for virtual machines (VMs) can lead to severe service level agreement (SLA) violations or inefficient use of memory. Dynamic memory allocation mechanisms such as ballooning and memory hotplug were proposed to handle the dynamics of memory demands. However, these mechanisms so far have not been quantitatively or comparatively studied. In this paper, we first develop a runtime system called U-tube, which provides a framework to adopt memory hotplug or ballooning for dynamic memory allocation. We then implement fine-grained memory hotplug in Xen. We demonstrate the effectiveness of U-tube for dynamic memory management through two case studies: dynamic memory balancing and memory overcommitment. With these two case studies, we make a quantitative comparison between memory hotplug and ballooning. The experiments show that there is no absolute winner for different scenarios. Our findings can be very useful for practitioners to choose the suitable dynamic memory management techniques in different scenarios. [ABSTRACT FROM AUTHOR]

Cloud computing has a key requirement for resource configuration in a real-time manner. In such virtualized environments, both virtual machines (VMs) and hosted applications need to be configured on-the-fly to adapt to system dynamics. The interplay between the layers of VMs and applications further complicates the problem of cloud configuration. Independent tuning of each aspect may not lead to optimal system wide performance. In this paper, we propose a framework, namely CoTuner, for coordinated configuration of VMs and resident applications. At the heart of the framework is a model-free hybrid reinforcement learning (RL) approach, which combines the advantages of Simplex method and RL method and is further enhanced by the use of system knowledge guided exploration policies. Experimental results on Xen-based virtualized environments with TPC-W and TPC-C benchmarks demonstrate that CoTuner is able to drive a virtual server cluster into an optimal or near-optimal configuration state on the fly, in response to the change of workload. It improves the systems throughput by more than 30 percent over independent tuning strategies. In comparison with the coordinated tuning strategies based on basic RL or Simplex algorithm, the hybrid RL algorithm gains 25 to 40 percent throughput improvement. [ABSTRACT FROM PUBLISHER]

Consistent hashing-based DHT networks have an inherent load balancing problem. The problem becomes more severe in heterogeneous networks with nonuniform and time-varying popular files. Existing DHT load balancing algorithms are mainly focused on the issues caused by node heterogeneity. To deal with skewed lookups, this paper presents an elastic routing table (EAT) mechanism for query load balancing, based on the observation that high-degree nodes tend to receive more traffic load. The mechanism allows each node to have a routing table of variable size corresponding to node capacities. The indegree and outdegree of the routing table can also be adjusted dynamically in response to the change of file popularity and network churn. Theoretical analysis proves that the routing table degree is bounded. The EAT mechanism facilitates locality-aware randomized query forwarding to further improve lookup efficiency. By relating query forwarding to a supermarket customer service model, we prove that a two-way randomized query forwarding policy should lead to an exponential improvement in query processing time over random walking. Simulation results demonstrate the effectiveness of the EAT mechanism and its related query forwarding policy for congestion and query load balancing. In comparison with existing "virtual-server"-based load balancing algorithms and other routing table control approaches, the EAT-based congestion control protocol yields significant improvement in query lookup efficiency. [ABSTRACT FROM AUTHOR]

Structured peer-to-peer overlay networks, like distributed hash tables (DHTs), map data items to the network based on a consistent hashing function. Such mapping for data distribution has an inherent load balance problem. Data redistribution algorithms based on randomized matching of heavily loaded nodes with light ones can deal with the dynamics of DHTs. However, they are unable to consider the proximity of the nodes simultaneously. There are other methods that rely on auxiliary networks to facilitate locality-aware load redistribution. Due to the cost of network construction and maintenance, the locality-aware algorithms can hardly work for DHTs with churn. This paper presents a locality-aware randomized load-balancing algorithm to deal with both the proximity and network churn at the same time. We introduce a factor of randomness in the probing of lightly loaded nodes in a range of proximity. We further improve the efficiency by allowing the probing of multiple candidates (d-way) at a time. Simulation results show the superiority of the locality-aware two-way randomized algorithm in comparison with other random or locality-aware algorithms. In DHTs with churn, it performs no worse than the best churn-resilient algorithm. It takes advantage of node capacity heterogeneity and achieves good load balance effectively even in a skewed distribution of items. [ABSTRACT FROM AUTHOR]

A scalable e-commerce server should be able to provide different levels of quality of service (QoS) to different types of requests based on clients' navigation patterns and the server capacity. E-Commerce workloads are composed of sessions. In this paper, we propose a session-based two-dimensional (2D) service differentiation model for online transactions: intersession and intrasession. The intersession model aims to provide different levels of QoS to sessions from different customer classes, and the intrasession model aims to provide different levels of QoS to requests in different states of a session. A primary performance metric of online transactions is slowdown. It measures the waiting time of a request relative to its service time. We present a processing rate allocation scheme for 2D proportional slowdown differentiation. We then introduce service slowdown as a systemwide QoS metric of an e-commerce server. It is defined as the weighted sum of request slowdown in different sessions and in different session states. We formulate the problem of 2D service differentiation as an optimization of processing rate allocation with the objective of minimizing the service slowdown of the server. We prove that the derived rate allocation scheme based on the optimization guarantees client requests' slowdown to be square-root proportional to their prespecified differentiation weights in both intersession and intrasession dimensions. We evaluate this square-root proportional rate allocation scheme and a proportional rate allocation scheme via extensive simulations. Results validate that both schemes can achieve predictable, controllable, and fair 2D service differentiation on e-commerce servers. The square-root proportional rate allocation scheme provides 2D service differentiation at a minimum cost of service slowdown. [ABSTRACT FROM AUTHOR]

To provide ubiquitous access to the proliferating rich media on the Internet, scalable streaming servers must be able to provide differentiated services to various client requests. Recent advances of transcoding technology make network-I/O bandwidth usages at the server communication ports controllable by request schedulers on the fly. In this article, we propose a transcoding-enabled bandwidth allocation scheme for service differentiation on streaming servers. It aims to deliver high bit rate streams to high priority request classes without overcompromising low priority request classes. We investigate the problem of providing differentiated streaming services at application level in two aspects: stream bandwidth allocation and request scheduling. We formulate the bandwidth allocation problem as an optimization of a harmonic utility function of the stream quality factors and derive the optimal streaming bit rates for requests of different classes under various server load conditions. We prove that the optimal allocation, referred to as harmonic proportional allocation, not only maximizes the system utility function, but also guarantees proportional fair sharing between classes with different prespecified differentiation weights. We evaluate the allocation scheme, in combination with two popular request scheduling approaches, via extensive simulations and compare it with an absolute differentiation strategy and a proportional- share strategy tailored from relative differentiation in networking. Simulation results show that the harmonic proportional allocation scheme can meet the objective of relative differentiation in both short and long timescales and greatly enhance the service availability and maintain low queueing delay when the streaming system is highly loaded. [ABSTRACT FROM AUTHOR]

A bulk synchronous computation proceeds in phases that are separated by barrier synchronization. For dynamic bulk synchronous computations that exhibit varying phase-wise computational requirements, remapping at runtime is an effective approach to ensure parallel efficiency. This paper introduces a novel remapping strategy for computations whose workload changes can be modeled as a Markov chain. The use of Markovian model allows us to treat statistical dependence and more complex structure than the usual independent identically distributed random variable assumptions. Our models are quite general and we do not need to impose conditions on the dynamics of the underlying process other than the transition probability matrix. It is shown that optimal remapping can be formulated as a binary decision process: remap or not at a given synchronizing instant. The optimal strategy is then developed for long lasted computations by employing optimal stopping rules in a stochastic control framework. The existence of optimal controls is established. Necessary and sufficient conditions for the optimality are obtained. Furthermore, a policy iteration algorithm is devised to reduce computational complexity and enhance fast convergence to the desired optimal control. [ABSTRACT FROM AUTHOR]

Presents a time stamp Inspector/Executor algorithm for runtime parallelization of general DOACROSS loops that have indirect access patterns. Principal objectives of loop parallelization; Time stamp algorithm allowing partially concurrent reads; Time stamp algorithm allowing fully concurrent reads; Space and time complexity; Performance evaluation.

The nineteen papers in this special issue focus on the development and applications of cyber-physical systems. These systems refer to the tight link between computational, virtual, and physical resources that drive the potential of having a pervasive effect in the citizens everyday life. Cyber-physical systems find direct applicability in a wide range of areas and disciplines, and promise to transform our world with systems that will far exceed those of today in terms of: effectiveness, adaptability, energy efficiency, precision, reliability, and safety. [ABSTRACT FROM AUTHOR]

In-network data aggregation is a useful technique to reduce redundant data and to improve communication efficiency. Traditional data aggregation schemes for wireless sensor networks usually rely on a fixed routing structure to ensure data can be aggregated at certain sensor nodes. However, they cannot be applied in highly mobile vehicular environments. In this paper, we propose an adaptive forwarding delay control scheme, namely Catch-Up, which dynamically changes the forwarding speed of nearby reports so that they have a better chance to meet each other and be aggregated together. The Catch-Up scheme is designed based on a distributed learning algorithm. Each vehicle learns from local observations and chooses a delay based on learning results. The simulation results demonstrate that our scheme can efficiently reduce the number of redundant reports and achieve a good trade-off between delay and communication overhead. [ABSTRACT FROM AUTHOR]

Live migration of virtual machines (VM) across physical hosts provides a significant new benefit for administrators of data centers and clusters. Previous memory-to-memory approaches demonstrate the effectiveness of live VM migration in local area networks (LAN), but they would cause a long period of downtime in a wide area network (WAN) environment. This paper describes the design and implementation of a novel approach, namely, CR/TR-Motion, which adopts checkpointing/recovery and trace/replay technologies to provide fast, transparent VM migration for both LAN and WAN environments. With execution trace logged on the source host, a synchronization algorithm is performed to orchestrate the running source and target VMs until they reach a consistent state. CR/TR-Motion can greatly reduce the migration downtime and network bandwidth consumption. Experimental results show that the approach can drastically reduce migration overheads compared with memory-to-memory approach in a LAN: up to 72.4 percent on application observed downtime, up to 31.5 percent on total migration time, and up to 95.9 percent on the data to synchronize the VM state. The application performance overhead due to migration is kept within 8.54 percent on average. The results also show that for a variety of workloads migrated across WANs, the migration downtime is less than 300 milliseconds. [ABSTRACT FROM PUBLISHER]

As e-commerce services are exponentially growing, businesses need quantitative estimates of client-perceived response times to continuously improve the quality of their services. Current server-side nonintrusive measurement techniques are limited to nonsecured HTTP traffic. In this paper, we present the design and evaluation a monitor, namely sMonitor, which is able to measure client-perceived response times for both HTTP and HTTPS traffic. At the heart of sMonitor is a novel size-based analysis method that parses live packets to delimit different webpages and to infer their response times. The method is based on the observation that most HTTP(S)-compatible browsers send significantly larger requests for container objects than those for embedded objects. sMonitor is designed to operate accurately in the presence of complicated browser behaviors, such as parallel downloading of multiple webpages and HTTP pipelining, as well as packet losses and delays. It requires only to passively collect network traffic in and out of the monitored secured services. We conduct comprehensive experiments across a wide range of operating conditions using live secured Internet services, on the PlanetLab, and on controlled networks. The experimental results demonstrate that sMonitor is able to control the estimation error within 6.7 percent, in comparison with the actual measured time at the client side. [ABSTRACT FROM AUTHOR]

Understanding server capacity is crucial to system capacity planning, configuration, and QoS-aware resource management. Conventional stress testing approaches measure server capacity offline in terms of application-level performance metrics like response time and throughput. They are limited in measurement accuracy and timeliness. In a multitier website, resource bottleneck often shifts between tiers as client access pattern changes. This makes the problem of online capacity measurement even more challenge. This paper presents an online measurement approach based on low-level hardware performance metrics such as instructions execution rate and cache access behavior. Such metrics together define a system internal running state. The measurement approach uses machine learning techniques to infer application-level performance at each tier from a set of selected hardware performance counters. A coordinated predictor is induced over individual tier-wide models to make global system performance prediction and identify the bottleneck when the system becomes overloaded. Experiments were conducted on a two-tier Tomcat/MySQL-configured website using TPC-W benchmarks. Experimental results demonstrated that this approach was able to achieve an overload prediction accuracy of higher than 90 percent for a priori known input traffic mix and over 85 percent accuracy even for traffic causing frequent bottleneck shifting. It costs less than 0.5 percent runtime overhead for data collection and no more than 50 ms for each online decision making. [ABSTRACT FROM AUTHOR]

Internet traffic often exhibits a structure with rich high-order statistical properties like self-similarity and long-range dependency (LRD). This greatly complicates the problem of server performance modeling and optimization. Existing tools like queuing models in most cases only hold in mean value analysis under the assumption of simplified traffic structures. In this paper, we present a filter model to characterize the relationship among the factors of server capacity, request scheduling, and service quality for general input traffic. By the model, a server scheduler operates as an finite-duration impulse response (FIR) filter that transforms request processes into workload processes with the objective of minimizing load variation or overload probability, and meanwhile, without violating request response deadlines as defined in service-level agreements. We present a design and analysis of the filter for traffic with strong LRD in the frequency domain. Most Internet traffic has monotonically decreasing strength of variation functions over frequency. For this type of input traffic, we prove that optimal schedulers must have a convex structure. Uniform resource allocation is an extreme case of the convexity and is proved to be optimal for Poisson traffic. We integrate the convex structural principle with the Generalized Processor Sharing (GPS) discipline and show that the enhanced GPS policy improves the service quality significantly. Furthermore, we show that the presence of LRD in the input traffic results in shift of variation strength from high frequency to lower frequency bands and consequently leads to a degradation of the service quality. [ABSTRACT FROM PUBLISHER]