Search

Mobile cloud gaming requires a very low end-to-end latency. Edge computing significantly reduces network latency. However, in mobility scenarios, the user will frequently move out of the edge server's coverage area, requiring frequent migration of the game instance. This paper presents Talaria, an in-engine content synchronisation solution for unnoticeable game instance migration between edge servers. Talaria creates a minimal instance with content immediately relevant to the game experience, allowing the client to switch servers in a minimal amount of time. The remaining content is then synchronised according to priority until the game's state is coherent between both instances. Our implementation of Talaria as a Unity engine plugin reduces the game's downtime by 61% compared to one-off server migration, with an average latency below 25 ms for the server migration, and 87 ms for the entire game synchronisation.

With the widespread use of Docker and Kubernetes, OS-level virtualization has become a key technology to deploy and run software. At the same time, data centers and cloud providers offer shared computing resources on demand. The use of these resources usually leads to a larger trusted computing base and less control over the data. We present a confidential computing concept for the migration of operating system containers in secure encrypted virtual machines so that these are protected from the operator and administrator. In our approach, processes inside of the containers remain intact, i.e., they keep their state and do not have to be restarted. Network services inside of the containers remain unchanged and reachable. This is typically called live migration. Integrity and confidentiality of the data inside of the containers is enforced during migration as well as on the destination platform, namely in transit, in use and at rest. The authenticity and integrity of the destination platform is verified using remote attestation before any data is transferred. While our core concept is not specific to a particular hardware, we present two different approaches corresponding to the first generation of AMD SEV as well as SEV-SNP. Our proof of concept implementation is based on the first generation of SEV.

Deterministic databases are able to run transactions efficiently in a distributed setting with minimal coordination. Research has shown many benefits from having determinism, from more lightweight database replication, to no/cheaper distributed commit, to high throughput during live migration. Besides those known applications, permissioned blockchain can also be viewed as an application of the deterministic database with security added. A permissioned blockchain maintains a fully-replicated database, where each replica is synchronized by getting blocks of updates. Early permissioned blockchains run a consensus protocol (e.g., PBFT) to agree on the input and rely on the replicas to execute the transactions serially to uphold determinism. Fabric allows concurrent transaction executions but rely on running consensus to agree on the read-write-sets and broadcasting them, which incurs an excessive volume of network traffic. Deterministic concurrency control combines the best of both worlds. It enables solving consensus on the small input, and each replica is able to execute transactions independently with high concurrency. Although powerful, contemporary deterministic databases are still primitive because they support only one isolation level -- Serializable. Furthermore, many deterministic concurrency control schemes are overly pessimistic and cause many unnecessary transaction aborts. These factors motivate us to design DCC, a suite of better deterministic concurrency control schemes, not only for Serializable but also for practical isolation levels beyond Serializable.

Host software updates are critical to ensure the security, reliability and compliance of public clouds. Many updates require a virtualization component restart or operating system reboot. Virtual machines (VMs) running on the updated servers must either be restarted or live migrated off. Reboots can result in downtime for the VMs on the order of ten minutes, and has further impact on the workloads running in the VMs because cached state is lost. Live migration (LM) is a technology that can avoid the need to shutdown VMs. However, LM requires turn space in the form of already-patched hosts, consumes network, CPU and other resources that scale with the amount of and level of activity of VM, and has variable impact on VM performance and availability, making it too expensive and disruptive for zero-day security updates that must be applied across an entire fleet on the order of hours. We present a novel update technology, virtual machine preserving host updates (VM-PHU), that does not require turn space, consumes no network and little CPU, preserves VM state, and causes minimal VM blackout time that does not scale with VM resource usage. VM-PHU persists the memory and device state of all running guest VMs, reboots the host and virtualization components into updated code, restores the state of the VMs, and then resumes them. VM-PHU makes use of several techniques to minimize VM blackout time. One is to use kernel soft reboot (KSR) to directly transition to an updated host operating system, bypassing firmware reset of the server and attached devices. To minimize resource consumption and VM disruption, VM-PHU leaves VM memory in physical memory pages and other state in persisted pages across the soft reboot, and VM-PHU implements a mechanism called fast close to enable a reboot to proceed without waiting for the completion of in-flight VM I/Os to remote storage devices. We have implemented VM-PHU in Microsoft Azure hosting millions of servers and show results of several zero-day updates that demonstrate VM blackout times on the order of seconds. VM-PHU provides significant benefits to both customers and public cloud vendors by minimizing application downtime while enabling fast and resource efficient updates, including zero-day patches.

Live migration is a crucial feature in existing virtualization platforms. Since memory is dirtied rapidly during the execution of a virtual machine (VM), boosting memory migration speed becomes a significant factor in guaranteeing a high-level success ratio and efficiency. However, the statically-configured migration strategy cannot cope with various workloads running in VMs, resulting in frequently aborted migration processes and low success ratio. This paper proposed a one-for-all migration architecture called Adaptive Live Migration (AdaMig) to address these issues. This QEMU-based solution dynamically switches migration methods and tunes related parameters by monitoring the run-time statistics from the migration process and the physical host. Once AdaMig detects the tendency that migration cannot converge, it will switch to another migration method to synchronize remaining dirty pages. During the whole process, AdaMig also dynamically tunes migration parameters according to current resources available in the physical host and migration efficiency. Experimental results reflect that AdaMig improves the success ratio from 26.7% to 93.3% over various workloads, and migration time is reduced by up to 45.5% in comparison with the original solution in QEMU.

Live migration is the process of moving an application of a running virtual machine between physical machines without disconnecting the client or application. In recent years, as cloud computing has developed at a rapid pace, numerous virtual systems have been developed, and container-type operating system level virtualization technology, which is attracting attention at a higher speed and lower resource consumption rate than a virtual machine which use Hypervisor. The tool that deploys and manages these containers is called a container orchestration tool, and the most widely used orchestration framework is Kubernetes. Currently, there is no function for live migration in Kubernetes. In this paper, we developed an environment where containers can be live migration in Kubernetes, and research was conducted to reduce downtime of live migration.

Checkpoint/Restart (C/R) is widely used to provide fault tolerance on High-Performance Computing (HPC) systems. However, Parallel File System (PFS) overhead and failure uncertainty cause significant application overhead. This paper develops an adaptive multi-level C/R model that incorporates a failure prediction and analysis model, which orchestrates failure prediction, checkpointing, checkpoint frequency, and proactive live migration along with the additional benefit of Burst Buffers (BB). It effectively reduces the overheads due to failures, checkpointing, and recovery. Simulation results for the Summit supercomputer yield a reduction of ~20%-86% in application overhead due to BBs, orchestrated failure prediction, and migration. We also observe a ~29% decrease in checkpoint writes to BBs, which can increase the longevity of the BB storage devices.

Bare-metal cloud platforms allow customers to rent remote physical servers and install their preferred operating systems and software to make the best of servers' raw hardware capabilities. However, this quest for bare-metal performance compromises cloud manageability. To avoid overheads, cloud operators cannot install traditional hypervisors that provide common manageability functions such as live migration and introspection. We aim to bridge this gap between performance, isolation, and manageability for bare-metal clouds. Traditional hypervisors are designed to limit and emulate hardware access by virtual machines (VM). In contrast, we propose Directvisor - a hypervisor that maximizes a VM's ability to directly access hardware for near-native performance, yet retains hardware control and manageability. Directvisor goes beyond traditional direct-assigned (pass-through) I/O devices by allowing VMs to directly control and receive hardware timer interrupts and inter-processor interrupts (IPIs) besides eliminating most VM exits. At the same time, Directvisor supports seamless (low-downtime) live migration and introspection for such VMs having direct hardware access.

In clouds and data centers, GPU servers which consist of multiple GPUs are widely deployed. Current state-of-the-art GPU scheduling algorithm are "static" in assigning applications to different GPUs. These algorithms usually ignore the dynamics of the GPU utilization and are often inaccurate in estimating resource demand before assigning/running applications, so there is a large opportunity to further load balance and to improve GPU utilization. Based on CUDA (Compute Unified Device Architecture), we develop a runtime system called DCUDA which supports "dynamic" scheduling of running applications between multiple GPUs. In particular, DCUDA provides a realtime and lightweight method to accurately monitor the resource demand of applications and GPU utilization. Furthermore, it provides a universal migration facility to migrate "running applications" between GPUs with negligible overhead. More importantly, DCUDA transparently supports all CUDA applications without changing their source codes. Experiments with our prototype system show that DCUDA can reduce 78.3% of overloaded time of GPUs on average. As a result, for different workloads consisting of a wide range applications we studied, DCUDA can reduce the average execution time of applications by up to 42.1%. Furthermore, DCUDA also reduces 13.3% energy in the light load scenario.

Service live migration in edge computing systems, which means migrating running services from one edge node to another with negligible cost of time, is becoming one of the cornerstones for various edge management tasks, such as task offloading, fault handling, load balancing and maintaining service continuity against users' mobility. However, due to the limited network bandwidth and computing resources, it is much more challenging to design and implement service live migration within an edge computing system. To overcome this challenge, we present an optimized flow to accelerate migration. Improvements are made from software, system and architecture aspects, which are expected to drastically reduce migration overheads and improve system efficiency.

Data migration has become one of the key technologies for load balance and high performance services in the distributed systems. In past years, the data migration researches focused on several factors including resource utilization, data quality or security improvement and migration time reduction, whereas the investigation on how to provide continuous system service during the data migration process has not attracted enough attention. In this paper, the migration operation cost and the user service quality are both weighted in the proposed migration scheme so as to guarantee a continuous high quality of user service during the data migration procedure. First, a cost-based migration model is built. Based on this model, the migration operation among the nodes of the data cluster is formulated as an optimization problem which targets to search for the optimal execution sequence. Then, an improved grey wolf optimization algorithm (IGWO) is proposed to search for the optimized migration solutions. In addition, to guarantee the high quality service for some critical user requests, the bandwidth reserved for the migration operations is dynamically managed through the on-demand programming of SDN (Software Defined Network) controller. We conducted the experiments on the Greenplum database, and the results showed the effectiveness of the proposed migration scheme.

Data compression is usually perceived as a mean to help improve performance of live migration of Virtual Machines (VM). However, such perception may be wrong since there are many factors involved in the combination of live migration and data compression. This paper presents a novel performance model to describe the speedup of VM live migration when used in combination with data compression mechanisms, namely the Amdahl's Law of data compression for live migration of VMs. The speedup is calculated by dividing the total live migration time when the migration is performed in combination with data compression by that without data compression. This model introduces live migration bandwidth as a new factor in the speedup calculation. The live migration bandwidth is the data transmission rate of a live migration mechanism. According to our model, the speedup of live migration using data compression would approach zero as the live migration bandwidth increases toward infinity. A set of theorems derived from the model are presented. A number of experiments have been conducted to confirm these theorems. In the experiments, QEMU-KVM's pre-copy live migration is used to migrate VMs running several memory-intensive OpenMP NAS Parallel benchmarks with and without data compression. Live migrations are performed over 100 Mbps and 1 Gbps networks. Experimental results confirm the validity of the proposed Amdahl's Law of data compression model.

To solve the problem that OpenStack cloud platforms can not well support the migration of virtual machines across clusters, several cross-cluster migration technologies are proposed. Based on a novel multi-cluster collabrative architecture, a cross-cluster static migration approach and a cross-cluster live migration approach are provided. The experiments of comparing our approachs with the original intra-cluster migration are carried out. The results show that the proposed approaches have nearly the same migration time cost but gain better performance in disaster tolerance.

Hypervisors are increasingly complex and must be often updated for applying security patches, bug fixes, and feature upgrades. However, in a virtualized cloud infrastructure, updates to an operational hypervisor can be highly disruptive. Before being updated, virtual machines (VMs) running on a hypervisor must be either migrated away or shut down, resulting in downtime, performance loss, and network overhead. We present a new technique, called HyperFresh, to transparently replace a hypervisor with a new updated instance without disrupting any running VMs. A thin shim layer, called the hyperplexor, performs live hypervisor replacement by remapping guest memory to a new updated hypervisor on the same machine. The hyperplexor leverages nested virtualization for hypervisor replacement while minimizing nesting overheads during normal execution. We present a prototype implementation of the hyperplexor on the KVM/QEMU platform that can perform live hypervisor replacement within 10ms. We also demonstrate how a hyperplexor-based approach can used for sub-second relocation of containers for live OS replacement.

High availability is the most important and challenging problem for cloud providers. However, virtual machine monitor (VMM), a crucial component of the cloud infrastructure, has to be frequently updated and restarted to add security patches and new features, undermining high availability. There are two existing live update methods to improve the cloud availability: kernel live patching and Virtual Machine (VM) live migration. However, they both have serious drawbacks that impair their usefulness in the large cloud infrastructure: kernel live patching cannot handle complex changes (e.g., changes to persistent data structures); and VM live migration may incur unacceptably long delays when migrating millions of VMs in the whole cloud, for example, to deploy urgent security patches. In this paper, we propose a new method, VMM live upgrade, that can promptly upgrade the whole VMM (KVM & QEMU) without interrupting customer VMs. Timely upgrade of the VMM is essential to the cloud because it is both the main attack surface of malicious VMs and the component to integrate new features. We have built a VMM live upgrade system called Orthus. Orthus features three key techniques: dual KVM, VM grafting, and device handover. Together, they enable the cloud provider to load an upgraded KVM instance while the original one is running and "cut-and-paste'' the VM to this new instance. In addition, Orthus can seamlessly hand over passthrough devices to the new KVM instance without losing any ongoing (DMA) operations. Our evaluation shows that Orthus can reduce the total migration time and downtime by more than $99%$ and $90%$, respectively. We have deployed Orthus in one of the largest cloud infrastructures for a long time. It has become the most effective and indispensable tool in our daily maintenance of hundreds of thousands of servers and millions of VMs.

With the popularity of virtualization technologies, virtualization security issues have been appeared, which greatly hampers the development of its further application. Bringing the encryption card to the virtual environment is a good idea to enforce the security of the virtual platform. However, due to the particularity of the encryption card equipment, there is some difficult to virtualize the encryption card equipment. One of the most important difficulties is supporting Live Migration. As for the first question that we must ensure the encryption task continuity when migrating a virtual encryption card, we propose the encryption field migration for the physical encryption card, which guarantees the continuity of virtualization encryption card. For the second problem that Live Migration of virtualization encryption card must be protected from any attack, this paper designs migration protocol to ensure the security. For the last problem that the encryption card live migration should not increase the migration time of virtual machine drastically, including total migration time and downtime, we separate the migration protocol to a different stage, and complete in the different stage of virtual machine migration. By this way, we decrease the influence of virtual machine migration time. The implement and results of experiments on Kernel-based Virtual Machine indicate that the proposed scheme is feasible in function and performance.

With the rapid development of cloud computing, more and more users choose to deploy applications on virtual machines on the cloud platform. And OpenStack is one of the mainstream cloud platforms in cloud computing. The load imbalance of cloud platform is a concern. Live migration of virtual machines is an effective way to achieve load balancing and optimize resource utilization. With the rapid expansion of the scheduling domain of the cloud platform, the traditional centralized migration strategy begins to lack reliability and scalability. In this paper, we propose a new VM dynamic scheduling algorithm based on the grey Markov prediction model. In our algorithm, the gray Markov prediction model is used to predict the state of the node load information, so as to cooperate with the VM dynamic scheduling mechanism to properly migrate the VM, and set the upper and lower thresholds of the node to achieve load balancing and reduce energy consumption. The experimental results show that the GMDS algorithm achieves the reduction of energy consumption by implementing load balancing of the entire system and reducing the number of running nodes, which are superior to the existing migration strategy.

Seamless mobility of mobile objects (MOs) such as containers or virtual machines cannot be guaranteed when MOs move across different networks of which the IP address domains are different. In this paper, we propose what is termed the live migration (LM) system and protocol that together provide seamless mobility for MOs connected to the Internet moving across different networks. The components of the LM system are the LM distributed hash table (DHT) system that stores the locator information of MOs and the LM Access Router, cooperating with the DHT system, which supports the mobility of MOs. The LM Access Router inserts the identifier of an MO into a packet and routes the packet to the LM Access Router to which the MO is associated. The LM system does not require modification of the network protocol stack of the host on which an MO is running, and it can support mobility with the IPv4 and IPv6 Internet protocols by introducing LM Access Routers into the network. We implemented the LM system on a testbed and conducted an experiment on the live migration of containers and virtual machines across networks. The findings showed that downtime of the live migration of a container to a host on a different network can enable the seamless live migration of MOs across networks.

Uninterrupted uptime is a critical aspect of Virtual Machines (VMs) offered by cloud hosting providers. Google's VMs run on top of rapidly changing infrastructure: we regularly update hardware and host software, and we must quickly respond to failing hardware. Frequent change is critical to both development velocity---deploying new versions of services and infrastructure---and the ability to respond rapidly to defects, including critical security fixes. Typically these updates would be disruptive, resulting in VM termination or restart. In this paper we present how we use VM live migration at scale to eliminate this disruption with minimal impact to the guest, performing over 1,000,0001migrations monthly in our production fleet, with 50ms median blackout, 300ms 99th percentile blackout.

With the rapid growth of data centers, thousands of large data centers with lots of computing nodes are established. Development of huge Cloud Data Centers (CDCs) has led to enormous energy consumption. Thus it is necessary for data centers to periodically resource-scaling for VM. For this reason, numerous schemes of resource scaling based on migration method are designed in the CDCs. However, cloud service providers (CSPs) have not yet provided sufficient efficiency to meet user requirements such as energy efficiency, performance and cost. The performance that migration takes depends on multi-metric like the resource utilization, and latency. And most of the previous researches did not consider the multi-metric such as server state transition, its effect to performance and power. In this paper, we proposed resource-scaling method based on migration. We performed resource scaling based on fuzzy system and analyzed effect of metric to maximize energy efficiency. Finally, we demonstrated the proposed method by using different resource scaling strategies.

Checkpoint/restore (a.k.a checkpoint/restart) is a technique which is naturally described by its two parts. The first one is a checkpoint. It allows creating snapshot of an application. The second one is restart. It uses the snapshot to run a copy of the application in the state it was during the time of the checkpoint. Moreover, a new instance can be created on another machine or after a period of time. Checkpoint/restore has found many applications in real-world problems such as live migration, load balancing, crash recovering, debugging and many others.The paper describes checkpoint/restore facility for file locks in Linux in userspace. Two types of file locks will be considered in particular: file lease and OFD lock. The solution described can be used in userspace and it does not require extra kernel modules or modification of the operating system. Implementation is based on CRIU project and intensively uses its infrastructure.

Scalable in-memory key-value stores provide low-latency access times of a few microseconds and perform millions of operations per second per server. With all data in memory, these systems should provide a high level of reconfigurability. Ideally, they should scale up, scale down, and rebalance load more rapidly and flexibly than disk-based systems. Rapid reconfiguration is especially important in these systems since a) DRAM is expensive and b) they are the last defense against highly dynamic workloads that suffer from hot spots, skew, and unpredictable load. However, so far, work on in-memory key-value stores has generally focused on performance and availability, leaving reconfiguration as a secondary concern. We present Rocksteady, a live migration technique for the RAMCloud scale-out in-memory key-value store. It balances three competing goals: it migrates data quickly, it minimizes response time impact, and it allows arbitrary, fine-grained splits. Rocksteady migrates 758 MB/s between servers under high load while maintaining a median and 99.9th percentile latency of less than 40 and 250 μs, respectively, for concurrent operations without pauses, downtime, or risk to durability (compared to 6 and 45 μs during normal operation). To do this, it relies on pipelined and parallel replay and a lineagelike approach to fault-tolerance to defer re-replication costs during migration. Rocksteady allows RAMCloud to defer all repartitioning work until the moment of migration, giving it precise and timely control for load balancing.

VM handoff enables rapid and transparent placement changes to executing code in edge computing use cases where the safety and management attributes of VM encapsulation are important. This versatile primitive offers the functionality of classic live migration but is highly optimized for the edge. Over WAN bandwidths ranging from 5 to 25 Mbps, VM handoff migrates a running 8 GB VM in about a minute, with a downtime of a few tens of seconds. By dynamically adapting to varying network bandwidth and processing load, VM handoff is more than an order of magnitude faster than live migration at those bandwidths.

Edge clouds are emerging as a popular paradigm of computation. In edge clouds, computation and storage can be distributed across a large number of locations, allowing applications to be hosted at the edge of the network close to the end-users. Virtual machine live migration is a key mechanism which enables applications to be nimble and nomadic as they respond to changing user locations and workload. However, VM live migration in edge clouds poses a number of challenges. Migrating VMs between geographically separate locations over slow wide-area network links results in large migration times and high unavailability of the application. This is due to network reconfiguration delays as user traffic is redirected to the newly migrated location. In this paper, we propose the use of multi-path TCP to both improve VM migration time and network transparency of applications. We evaluate our approach in a commercial public cloud environment and an emulated lab based edge cloud testbed using a variety of network conditions and show that our approach can reduce migration times by up to 2X while virtually eliminating downtimes for most applications.

The level of demand for bare-metal cloud services has increased rapidly because such services are cost-effective for several types of workloads, and some cloud clients prefer a single-tenant environment due to the lower security vulnerability of such enviornments. However, as the bare-metal cloud does not utilize a virtualization layer, it cannot use live migration. Thus, there is a lack of manageability with the bare-metal cloud. Live migration support can improve the manageability of bare-metal cloud services significantly. This paper suggests an on-demand virtualization technique to improve the manageability of bare-metal cloud services. A thin virtualization layer is inserted into the bare-metal cloud when live migration is requested. After the completion of the live migration process, the thin virtualization layer is removed from the host. We modified BitVisor [19] to implement on-demand virtualization and live migration on the x86 architecture. The elapsed time of on-demand virtualization was negligible. It takes about 20 ms to insert the virtualization layer and 30 ms to remove the one. After removing the virtualization layer, the host machine works with bare-metal performance.

Live migration is one of the key technologies to improve data center utilization, power efficiency, and maintenance. Various live migration algorithms have been proposed; each exhibiting distinct characteristics in terms of completion time, amount of data transferred, virtual machine (VM) downtime, and VM performance degradation. To make matters worse, not only the migration algorithm but also the applications running inside the migrated VM affect the different performance metrics. With service-level agreements and operational constraints in place, choosing the optimal live migration technique has so far been an open question. In this work, we propose an adaptive machine learning-based model that is able to predict with high accuracy the key characteristics of live migration in dependence of the migration algorithm and the workload running inside the VM. We discuss the important input parameters for accurately modeling the target metrics, and describe how to profile them with little overhead. Compared to existing work, we are not only able to model all commonly used migration algorithms but also predict important metrics that have not been considered so far such as the performance degradation of the VM. In a comparison with the state-of-the-art, we show that the proposed model outperforms existing work by a factor 2 to 5.

When integrated with push notifications, a live migration function can be used to ensure that systems have a high reliability in cases of hardware failures. However, the dependencies for a large range of hardware devices need to be addressed before realizing emergency live migration. Our platform introduces container-based light virtualization and an automated build function to isolate an application so that it can be deployed on different devices such as Edison, Raspberry Pi model B, BeagleBone Black, and Odroid XU3.

General purpose GPU (GPGPU) computing in virtualized environments leverages PCI passthrough to achieve GPU performance comparable to bare-metal execution. However, GPU passthrough prevents service administrators from performing virtual machine migration between physical hosts. Crane is a new technique for virtualizing OpenCL-based GPGPU computing that achieves within 5.25% of passthrough GPU performance while supporting VM migration. Crane interposes a virtualization-aware OpenCL library that makes it possible to reclaim and subsequently reassign physical GPUs to a VM without terminating the guest or its applications. Crane also enables continued GPU operation while the VM is undergoing live migration by transparently switching between GPU passthrough operation and API remoting.

Live migration of a virtual machine (VM) is a powerful technique with benefits of server maintenance, resource management, dynamic workload re-balance, etc. Modern research has effectively reduced the VM live migration (VMLM) time to dozens of milliseconds, but live migration still exhibits failures if it cannot terminate within the given time constraint. The ability to predict this type of failure can avoid wasting networking and computing resources on the VM migration, and the associated system performance degradation caused by wasting these resources. The cost of VM live migration highly depends on the application workload of the VM, which may undergo frequent changes. At the same time, the available system resources for VM migration can also change substantially and frequently. To account for these issues, we present a solution called MigVisor, which can accurately predict the behaviour of VM migration using working-set model. This can enable system managers to predict the migration cost and enhance the system management efficacy. The experimental results prove the design suitability and show that the MigVisor has a high prediction accuracy since the average relative error between the predicted value and the measured value is only 6.2%~9%.

Cloud computing platforms routinely use virtualization to improve service availability, resiliency, and flexibility. Live migration of Virtual Machines (VM) is a key technique to quickly migrate workloads in response to events such as impending failure or load changes. Despite extensive research, state-of-the-art live migration approaches take a long time to migrate a VM (in the order of tens of seconds for moderately sized VMs) which in turn negatively impacts the application performance during migration. We present Quick Eviction, a new approach to significantly speed up the eviction of a VM from the source host with low impact on VM's performance during migration. Our approach can also improve the effectiveness of VM resilience tools that back up a VM's state in anticipation of system failures. The insight behind Quick Eviction is that the majority of time to evict a VM during migration is due to the transfer of memory contents over the network, sometimes repeatedly. Before migration, Quick Eviction regularly snapshots the VM's memory to a destination or a failover node. During the actual migration, Quick Eviction has to transfer only a small amount of dirtied memory resulting in a very short time to completely evict the VM out of the source. The key challenge is to dynamically adapt the snapshot intervals so as to have minimal impact on application performance during migration. We show that our implementation of Quick Eviction in the KVM/QEMU platform reduces the eviction time of a VM by significant factors for both idle and write-intensive VMs compared to traditional pre-copy algorithm, while maintaining low overheads on application performance and network.

Back-and-forth live migration, which means a running VM migrates between two physical machines back and forth, has several important applications. Traditional methods treat each migration as a single event, so the VM releases its system resources on the source site after migration. However, many resources can be kept to mitigate the cost of the next migration back to the machine. This paper presents performance optimization methods for back-and-forth live migration. Different from previous work, our approach can keep the data center resiliency. We leverage the technique of snapshot and the bitmap model, which are available in most existing VM management systems. Using the snapshot, a VM can be immediately restarted from the saved state. The bitmaps model is used to avoid redundant data transmission to decrease the costs of migration. We implemented the bank-and-forth live migration optimization methods in QEMU-KVM 2.0. The experiments show that the proposed methods can significantly reduce the overhead of migrations. The total migration time can be saved up to 99% for some applications.

Live migration of Java Virtual Machines (JVMs) consumes significant amounts of time and resources, imposing relevant application performance overhead. This problem is specially hard when memory modified by applications changes faster than it can be transferred through the network (to a remote host). Current solutions to this problem resort to several techniques which depend on high-speed networks and application throttling, require lots of CPU time to compress memory, or need explicit assistance from the application. We propose a novel approach, Garbage Collector (GC) assisted JVM Live Migration for Java Server Applications (ALMA). ALMA makes a snapshot to be migrated containing a minimal amount of application state, by taking into account the amount of reachable memory (i.e. live data) detected by the GC. The main novelty of ALMA is the following: ALMA analyzes the JVM heap looking for regions in which a collection phase is advantageous w.r.t. the network bandwidth available (i.e. it pays to collect because a significant amount of memory will not be part of the snapshot). ALMA is implemented on OpenJDK 8 and extends CRIU (a Linux disk-based process checkpoint/restore tool) to support process live migration over the network. We evaluate ALMA using well-known JVM performance benchmarks (SPECjvm2008 and DaCapo), and by comparing it to other previous approaches. ALMA shows very good performance results.

Graphic Process Unit (GPU) is the core for graphics computation. Because of its parallel nature it has potential to enhance the computation in a wide range. An example is the adoption of GPU virtualization in the cloud environment for high performance computing. However, GPU has its limitations. For various reasons GPU may hang in applications, whilst full GPU virtualization does not support live migration like CPU in Xen. The state-of-the-art method to get round is to reset the GPU hardware via a specific mechanism provided by GPU vendors. The disadvantage of such reset operation is that it causes application unavailability in order to maintain stability of the operating system. This paper presents an efficient method to address the above limitations. In particular, we start from the gVirt [28] method, and develop a novel checkpointing mechanism within our open source solution1 which we define as High Availability gVirt (gHA for short). The key of our scheme is to checkpoint and back up the whole Virtual Machine (VM) to a new host. In circumstances when GPU hangs, the backup VM takes over to guarantee the high availability of the virtualized environment. Not surprisingly, overhead occurs when our checkpointing mechanism is applied. Our tests show that downtime of the VM backup is between 220 ms and 400 ms, only 100-200 ms more than the idle VM without GPU virtualization support, which is a quite acceptable result. For numerous GPU workload tests, our evaluation shows that different GPU workloads achieve 65-92% of gVirt performance, which is a good trade-off between performance and stability. Our solution occupies 80-180Mbps bandwidth during execution, which is fairly manageable as the total bandwidth of 1Gbps between the two hosts.

Dynamic virtual machine consolidation strategies refer to a number of resource management algorithms aiming at finding the right balance between energy consumption and SLA violations by using live migration techniques in virtualized cloud data centers. Most strategies found in the literature typically focus on centralized approaches, with a single management node responsible for VM placement. These approaches suffer from poor scalability, as the management node may become a performance bottleneck if the number of physical and virtual machines grows. In this paper we propose a fully decentralized dynamic virtual machine consolidation strategy on top of an unstructured P2P network of physical host nodes and investigate the performance of the strategy in terms of energy consumption, average CPU utilization, performance degradation due to overloading, performance degradation due to migration and total number of sleep nodes inside data center. The experimental results show that the proposed P2P strategy can achieve a global efficiency in terms of energy and performance very close to a centralized approach while assuring scalability due to increasing the number of hosts in data center.

In recent years, high power consumption in the cloud data center environment become major issue. Ever increasing resource demands increase the power consumption in cloud environment. High power consumption increase the operating budget, it responsible for co2 emissions which harmful to environment. So, we need to minimize the power consumption. Although several algorithms have been suggested to solve this problem, in this paper we present exhibit energy utilization assessment on the impacts of live migration of VMs. We proposed framework which objective is to combine the most extreme number of Virtual machines (VMs) on least number of Hosts. We used linear programming for optimal Virtual machine (VM) reallocation. As appeared by experimental results, minimize power consumption in cloud data center environment. Experiment performed using CloudSim Toolkit.

Virtualization technology adds great opportunities and challenges to the cloud computing paradigm. Resource management can be efficiently enhanced by employing Live Virtual Machine Migration (LVMM) techniques. Based on the literature of LVMM implementation in the virtualization environment, middle-boxes such as firewalls do not work effectively after LVMM as it introduces dynamic changes in network status and traffic, which may lead to critical security vulnerabilities. One key security hole is that the security context of the firewall do not move with the Virtual Machine after LVMM is triggered. This leads to inconsistency in the firewall level of protection of the migrated Virtual Machine. There is a lack in the literature of practical studies that address this problem in cloud computing platform. This paper demonstrates a practical analysis using OpenStack testbed to study the firewalls limitations in protecting virtual machines after LVMM. Two network scenarios are used to evaluate this problem. The results show that the security context problem does not exist in the stateless firewall but can exist in the stateful firewall.

Single-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (ysical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly assigned to a virtual machine (VM) in passthrough mode to significantly improve the network performance. However, VF passthrough mode is not compatible with live migration, which is an essential capability that enables many advanced virtualization features such as high availability and resource provisioning. To solve this problem, we design SRVM which provides hypervisor support to ensure the VF device can be correctly used by the migrated VM and the applications. SRVM is implemented in the hypervisor without modification in guest operating systems or guest VM drivers. Our experimental results show that SRVM can effectively migrate all memory state, and there is no data loss or corruption in applications after live migration. SRVM does not increase VM downtime. It only costs limited resources (an extra CPU core), and there is no significant runtime overhead in VM network performance. In fact, since the VF can continue to be used during the pre-copy ase, it offers network throughput which is 9.6 times and network latency which is 98% lower compared to other solutions that switch to para-virtualization mode during live migration.

Virtual machine (VM) migration demands distinct properties under resource oversubscription and workload surges. We present enlightened post-copy, a new mechanism for VMs under contention that evicts the target VM with fast execution transfer and short total duration. This design contrasts with common live migration, which uses the down time of the migrated VM as its primary metric; it instead focuses on recovering the aggregate performance of the VMs being affected. In enlightened post-copy, the guest OS identifies memory state that is expected to encompass the VM's working set. The hypervisor accordingly transfers its state, mitigating the performance impact on the migrated VM resulting from post-copy transfer. We show that our implementation, with modest instrumentation in guest Linux, resolves VM contention up to several times faster than live migration.

The live migration technology of virtual machine is very helpful for dynamic workload balance, server consolidation and fault tolerance in cloud computing environment. It is important to build live migration strategies which lead to low migration time, thus helping reduce migration cost while achieving migration goal. So we look into the topic of building a model to quantitatively predict live migration time. We thoroughly analyze the key parameters that affect the migration time and construct a live migration time cost model based on KVM. The evaluation of time cost model shows that the average prediction accuracy is above 90% in comparison with measured time. Based on time cost model, we propose 2 optimized live migration strategies for different application scenarios, one for load balance and fault tolerance, the other for server consolidation. The evaluation shows our optimized migration strategies can save 35%~50% of cost compared to the random migration strategy. We believe this should be the first comprehensive study of optimizing live migration strategy with migration time cost as a key factor.

Energy consumption management in cloud data centres is a very challenging research issue at present and the growth in cloud data centres have only compounded this challenge. In this paper, we focus on minimizing energy consumption of datacentres assuming the placement of VMs in physical servers as a bin packing problem having variable class constraints. We propose two algorithms, namely an energy-aware heuristic VM placement algorithm and an energy aware live Migration algorithm with variable resource class constraints intact for current resources' status. This paper presents some simulations carried out to investigate the efficacy of the proposed algorithms and the results obtained demonstrate the superiority of the proposed scheme.

Migration is the method of sending virtual machine(s) (VMs) from one physical machine (PM) to other. Migration of VMs may be required for maintenance or up gradation of PMs without disturbing the executions being carried out by them. Unlike offline migration, in live migration, VM(s) are kept continuously powered-up. In this research, we intend to improve the total migration time and total downtime, with help of using pre-copy approach, against post copy approach where downtime is considerable. In our proposed work, low as well as high dirty page rates are considered.

The Cloud provider offers computing resources such as processing capabilities, storage, network bandwidth etc. to the consumers on pay-as-go model. Due to its wide adaptability and ease of use, Cloud has emerged drastically over the last few years. However, increase in Cloud usage has lead to increase in number of computing resources and hence, increase in energy consumption. Recently, the issue of effective energy consumption has been addressed by a good number of researchers. In our work, we intend to study various energy efficient techniques for Cloud which uses resource allocation and consolidation to achieve the same. Further, we try to modify the existing technique by adding an extra parameter viz. time which assists in maximizing resource utilization and minimizing number of running servers.

Dynamic Server Consolidation (DSC) uses the live migration facility of virtualization technology to consolidate virtual machines (VMs) on fewer physical machines (PMs) when demand is low, to use minimal PMs without violating the Service Level Agreements (SLAs). The implicit goal of DSC is to maximize the data center profit, or minimize the cost, which is generally approximated in terms of PM operational cost and some direct cost given to migrations. In this paper we propose a profit model in which costs of actions, such as VM migrations, manifest as SLA penalties and increased operating costs, rather than some direct artificial costs given to them. The model can be used as an important tool to evaluate different consolidation algorithms, in terms of single cost metric. We also propose an algorithm for DSC, based on the stability of VM placements, which is an improved form of an existing algorithm, and use our metric to compare the two algorithms for different values of cost parameters. In most cases our algorithm reduces costs by upto 28%.

Software emulation has been proven useful in many scenarios, such as software testing, malware analysis, and intrusion response. However, fine-grained software emulation (e.g., at the instruction level) incurs considerable execution overhead (about 8x performance degradation), which hampers its use in production settings. In this paper, we propose MOSE (Live Migration based On-the-fly Software Emulation) that combines the performance advantages of hardware virtualization and the fine-grained analysis capability (comprehensiveness) of whole-system software emulation. Namely, a system can run as normal on a hardware-virtualized platform at near native speed, but when needed, it can be live-migrated to an emulator, not necessarily running on the same physical system, for in-depth analysis and triage; when the analysis is complete, the virtual machine can be migrated back to benefit from full hardware-virtualization again. In this way, the performance degradation is only experienced during analysis and triage. To demonstrate this new capability, we built a proof of concept on-the-fly software emulation system, based on QEMU/KVM and DECAF, the Dynamic Executable Code Analysis Framework. We also perform three case studies: automated kernel panic triage, live-patching a security vulnerability, and on-demand symbolic execution, to illustrate on-demand instruction level analysis.

Virtualization is the key underlying technology enabling cloud providers to host services for a large number of customers. Live migration is an essential feature of virtualization that allows transfer of virtual machines from one physical server to another. Live migration facilitates workload balancing, fault tolerance, online system maintenance, consolidation of virtual machines etc. The pre-copy algorithm is the most used method for the live migration of virtual machines. One of the objectives of live migration is that it should have minimum migration time as well as downtime so that application running on VM will be suspended for negligible time. In this paper we proposed an algorithm to improve the total migration time which is modifies the existing counting Sort algorithm for the live Migration and time series based algorithm. This algorithm takes dirty counts of group of pages instead of individual dirty counts of each page and send those pages only which are not highly dirtied.

Although many models exist to predict the time taken to migrate a virtual machine from one physical machine to another, our empirical validation of these models has shown the 90th percentile error to be 46% (43 secs) and 159% (112 secs) for KVM and Xen live migration, respectively. Our analysis reveals that these models are fundamentally flawed as they all fail to take into account the following three critical parameters: (i) the writable working set size, (ii) the number of pages eligible for the skip technique, (iii) the relation of the number of skipped pages with the page dirty rate and the page transfer rate, and incorrectly model the key parameter---the number of new pages dirtied per unit time. In this paper, we propose a novel model that takes all these parameters into account. We present a thorough validation with 53 workloads and show that the 90th percentile error in the estimated migration times is only 12% (8 secs) and 19% (14 secs) for KVM and Xen live migration, respectively.

Penetration testing---the process of probing a deployed system for security vulnerabilities---involves a fundamental tension. If one tests a production system, there is a real danger of collateral damage; this is particularly true for systems hosted in the cloud due to the presence of other tenants. If one tests against a separate system brought up to model the live one, the dynamic state of the production system is not captured, and the value of the test is reduced. This paper presents Potassium, which provides penetration testing as a service (PTaaS) and resolves this tension for system owners, penetration testers, and cloud providers. Potassium uses techniques originally developed for live migration of virtual machines to clone them instead, capturing their full disk, memory, and network state. Potassium isolates the cloned system from the rest of the cloud, providing confidence that side effects of the penetration test will not harm other tenants. The penetration tester effectively owns the cloned system, allowing testing to be more thorough, efficient, and automatable. Experiments with our Potassium prototype show that PTaaS can detect real-world vulnerabilities while having minimal impact on cloud-based production systems.

Virtualization owes its popularity mainly to its ability to consolidate software systems from many servers into a single server without sacrificing the desirable isolation between applications. This not only reduces the total cost of ownership, but also enables rapid deployment of complex software and application-agnostic live migration between servers for load balancing, high-availability, and fault-tolerance.However, virtualization is no free lunch. To achieve isolation, virtualization environments need to add an additional layer of abstraction between the bare metal hardware and the application. This inevitably introduces a performance overhead. High-performance main-memory database systems are specifically susceptible to additional software abstractions as they are closely optimized and tuned for the underlying hardware. In this work, we analyze in detail how much overhead modern virtualization options introduce for high-performance main-memory database systems. We evaluate and compare the performance of HyPer and MonetDB under three modern virtualization environments for analytical as well as transactional workloads. Our experiments show that the overhead depends on the system and virtualization environment being used. We further show that main-memory database systems can be efficiently deployed in virtualized cloud environments such as the Google Compute Engine and that "friendship" between modern virtualization and main-memory database systems is indeed possible.

Database-as-a-service has been gaining popularity in cloud computing because multitenant databases can reduce costs by sharing off-the-shelf resources. However, due to heavy workloads, resource sharing often causes a hot spot; one node is overloaded even while others are not. Unfortunately, a hot spot can lead to violation of service level agreements and destroy customer satisfaction. To efficiently address the hot spot problem, we propose a middleware approach called Madeus that conducts database live migration. To make efficient database live migration possible, we also introduce the lazy snapshot isolation rule (LSIR) that enables concurrently propagating syncsets, which are the datasets needed to synchronize slave with master databases. Madeus provides efficient database live migration by implementing the LSIR under snapshot isolation. Unlike current approaches, Madeus is pure middleware that is transparent to the database management system and based on commodity hardware and software. To demonstrate the superiority of our approach over current approaches, we experimentally evaluated Madeus by using PostgreSQL with the TPC-W benchmark. The results indicate that Madeus achieves more efficient live migration than three other types of middleware approaches, especially under heavy workloads; therefore, it can effectively resolve hot spots.

Live migration of virtual machines (VMs) can consume excessive time and resources, and may affect application performance significantly if VM memory pages get dirtied faster than their content can be transferred to the destination. Existing approaches to this problem transfer memory content faster with high-speed networks, slow down the dirtying of memory pages by throttling the execution of applications, or reduce the amount of memory content to be transferred, for example, using compression. However, these approaches incur high resource costs or application performance penalties. In this paper, we propose to skip the transfer of VM memory pages that need not be migrated for the execution of running applications at the destination, by exploiting applications' assistance. We have designed a generic framework for application-assisted live migration and then used it to build and evaluate JAVMM, which migrates VMs running various types of Java applications skipping the transfer of garbage in Java memory. Our experimental results show that JAVMM can reduce the completion time, the network traffic of transferring memory pages, and the application downtime of Java VM migration, all by up to over 90%, compared to the vanilla Xen VM migration, without incurring noticeable performance penalty to applications.