Search

The web browser is one of the most significant applications on mobile devices such as smartphones. However, the user experience of mobile web browsing is undesirable because of the slow resource loading. To improve the performance of web resource loading, client-side cache has been adopted as a key mechanism. However, the existing passive measurement studies cannot comprehensively characterize the “ client-side ” cache performance of mobile web browsing. For example, most of these studies mainly focus on client-side implementations but not server-side configurations, suffer from biased user behaviors, and fail to study “ miscached ” resources. To address these issues, in this article, we present a proactive approach to making a comprehensive measurement study on client-side cache performance. The key idea of our approach is to proactively crawl resources from hundreds of websites periodically with a fine-grained time interval. Thus, we are able to uncover the resource update history and cache configurations at the server side, and analyze the cache performance in various time granularities. Based on our collected data, we build a new cache analysis model and study the upper bound of how high percentage of resources could potentially be cached and how effectively the caching works in practice. We report detailed analysis results of different websites and various types of web resources, and identify the problems caused by unsatisfactory cache performance. In particular, we identify two major problems— Redundant Transfer and Miscached Resource , which lead to unsatisfactory cache performance. We investigate three main root causes: Same Content , Heuristic Expiration , and Conservative Expiration Time , and discuss what mobile web developers can do to mitigate those problems.

On account of their large footprint, on-chip last-level caches in multi-core systems are one of the most vulnerable components to soft errors. However, vulnerability to soft errors highly depends on the configuration and parameters of the last-level cache, especially when executing different applications concurrently. In this article we propose a novel reliability-aware reconfigurable last-level cache architecture (R 2 Cache) and cache vulnerability model for multi-cores. R 2 Cache supports various reliability-wise efficient cache configurations (i.e., cache parameter selection and cache partitioning) for different concurrently executing applications. The proposed vulnerability model takes into account the vulnerability of both the data and tag arrays as well as the active cache area for applications in different execution phases. To enable runtime adaptations, we introduce a lightweight online vulnerability predictor that exploits the knowledge of performance metrics like number of L2 misses to accurately estimate the cache vulnerability to soft errors. Based on the predicted vulnerabilities of different concurrently executing applications in the current execution epoch, our runtime reliability manager reconfigures the cache such that, for the next execution epoch, the total vulnerability for all concurrently executing applications is minimized under user-provided tolerable performance/energy overheads. In scenarios where single-bit error correction for cache lines may be afforded, vulnerability-aware reconfigurations can be leveraged to increase the reliability of the last-level cache against multi-bit errors. Compared to state-of-the-art vulnerability-minimizing and reconfigurable caches, the proposed architecture provides 35.27% and 23.42% vulnerability savings, respectively, when averaged across numerous experiments, while reducing the vulnerability by more than 65% and 60%, respectively, for selected applications and application phases.

Newcache is a secure cache that can thwart cache side-channel attacks to prevent the leakage of secret information. All caches today are susceptible to cache side-channel attacks, despite software isolation of memory pages in virtual address spaces or virtual machines. These cache attacks can leak secret encryption keys or private identity keys, nullifying any protection provided by strong cryptography. Newcache uses a novel dynamic, randomized memory-to-cache mapping to thwart contention-based side-channel attacks, rather than the static mapping used by conventional set-associative caches. In this article, the authors present an improved design of Newcache, in terms of security, circuit design and simplicity. They show Newcache's security against a suite of cache side-channel attacks. They evaluate Newcache's system performance for cloud computing, smartphone, and SPEC benchmarks and find that Newcache performs as well as conventional set-associative caches, and sometimes better. They also designed a VLSI test chip with a 32-Kbyte Newcache and a 32-Kbyte, eight-way, set-associative cache and verified that the access latency, power, and area of the two caches are comparable. These results show that Newcache can be used as L1 data and instruction caches to improve security without impacting performance.

Solid State Drives (SSDs) have been extensively deployed as the cache of hard disk-based storage systems. The SSD-based cache generally supplies ultra-large capacity, whereas managing so large a cache introduces excessive memory overhead, which in turn makes the SSD-based cache neither cost-effective nor energy-efficient. This work targets to reduce the memory overhead introduced by the replacement policy of SSD-based cache. Traditionally, data structures involved in cache replacement policy reside in main memory. While these in-memory data structures are not suitable for SSD-based cache any more since the cache is much larger than ever. We propose a memory-efficient framework which keeps most data structures in SSD while just leaving the memory-efficient data structure (i.e., a new bloom proposed in this work) in main memory. Our framework can be used to implement any LRU-based replacement policies under negligible memory overhead. We evaluate our proposals via theoretical analysis and prototype implementation. Experimental results demonstrate that, our framework is practical to implement most replacement policies for large caches, and is able to reduce the memory overhead by about $10 \times$ .

Cache compression improves the performance of a multi-core system by being able to store more cache blocks in a compressed format. Compression is achieved by exploiting data patterns present within a block. For a given cache space, compression increases the effective cache capacity. However, this increase is limited by the number of tags that can be accommodated at the cache. Prefetching is another technique that improves system performance by fetching the cache blocks ahead of time into the cache and hiding the off-chip latency. Commonly used hardware prefetchers, such as stream and stride, fetch multiple contiguous blocks into the cache. In this paper we propose prefetched blocks compaction (PBC) wherein we exploit the data patterns present across these prefetched blocks. PBC compacts the prefetched blocks into a single block with a single tag, effectively increasing the cache capacity. We also modify the cache organization to access these multiple cache blocks residing in a single block without any need for extra tag look-ups. PBC improves the system performance by 11.1 percent with a maximum of 43.4 percent on a four-core system.

Caching is an essential technique to improve throughput and latency in a vast variety of applications. The core idea is to duplicate content in memories distributed across the network, which can then be exploited to deliver requested content with less congestion and delay. The traditional role of cache memories is to deliver the maximal amount of requested content locally rather than from a remote server. While this approach is optimal for single-cache systems, it has recently been shown to be significantly suboptimal for systems with multiple caches (i.e., cache networks). Instead, cache memories should be used to enable a coded multicasting gain. In this article, we survey these recent developments. We discuss both the fundamental performance limits of cache networks and the practical challenges that need to be overcome in real-life scenarios.

In the past few years, multi-core processors incorporating four, six, eight, or more cores on a single die have become ubiquitous. Those cores, having their own private caches, often share a higher level cache memory, which leads to compete among different tasks. This can seriously affect the average performance of multi-core systems as the probability of cache hit could be lowered. In realizing this, we study the problem of scheduling bag-of-tasks (BoT) applications with shared cache constraint on multi-core systems. We first use cache space isolation techniques to divide shared caches into partitions. Then, we give a motivational example and outline the shared cache aware scheduling problem of multi-core systems. Finally, to provide an optimum solution for this problem, we propose a heuristic shared cache contention aware scheduling (SCAS) algorithm on multi-core systems. Our extensive simulation performance evaluation study clearly demonstrate that our proposed SCAS algorithm outperforms the existing traditional scheduling algorithm Min-min and the modified algorithm MSCAS in terms of schedule length and average response time.

In multicore systems with shared cache, multiple tasks run on multiple cores simultaneously and compete for the shared cache. Cache interference occurs when a task running on one core replaces cache data belonging to other tasks running on other cores. With today's multicore systems running tasks with different priorities, the need for providing QoS guarantees on cache usage is gaining importance. Solutions to reduce cache interference and provide cache QoS mainly used a cache partitioning approach to split the cache among different cores. The solutions were implemented and validated only on simulators and not on real systems. This paper discusses new techniques that are used on real systems to (1) experimentally measure the amount of interference caused by multiple coscheduled programs, (2) reduce interference miss rate of some programs at the expense of others and (3) provide cache QoS guarantees to programs and ensure their miss rates remain below a ceiling.

Utilizing small (e.g., 4 KB) pages incurs frequent TLB misses on modern big memory applications, substantially degrading the performance of the system. Large (e.g., 1 GB) pages or direct segments can alleviate this penalty due to page table walks, but at the same time such a strategy exposes the organizational and operational details of modern DRAM-based memory systems to applications. Row-buffer conflicts caused by accesses heading to the same DRAM bank but different rows from multiple threads are regarded as the main culprits behind the very large gaps between peak and achieved main memory throughput, but hardware-based approaches in memory controllers have achieved only limited success whereas existing proposals that change memory allocators cannot be applied to large pages or direct segments. In this paper, we propose a set of application-level techniques to improve the effective main memory bandwidth. The techniques stem from the two key observations that 1) each thread of an application exclusively accesses certain datasets for a short or long period of time, and 2) superfluous memory reads originating from a cache’s write allocation policy can be avoided if scatters during the data shuffling pass through intermediate cache-friendly buffers. Experiments with a contemporary x86 server show that combining large pages with the proposed address linearization, bank coloring, and write streaming techniques improves the performance of the three big memory applications of high-throughput key-value store, fast-Fourier transform, and radix sort by 37.6, 22.9, and 68.1 percent, respectively.

Caches are used to improve memory access time and energy consumption. The cache configuration which enables the best performance often differs between applications due to diverse memory access patterns. The authors present a new concept, called switchable cache, where multiple cache configurations exist on chip, leveraging the abundant transistors available due to what is known as the dark silicon phenomenon. Only one cache configuration is active at any given time based on the application under execution, while all other configurations remain inactive (dark). They describe an architecture to enable seamless integration of multiple cache configurations, and a novel design space exploration methodology to rapidly pre-determine the optimal set of configurations at design-time, for a given group of applications. For design spaces containing trillions of design points, the authors’ exploration methodology always found the optimal solution in less than 2 s. The switchable cache improved memory access time by up to 26.2% when compared to a fixed cache.

Cache based attacks can overcome software-level isolation techniques to recover cryptographic keys across VM-boundaries. Therefore, cache attacks are believed to pose a serious threat to public clouds. In this work, we investigate the effectiveness of cache attacks in such scenarios. Specifically, we apply the Flush+Reload and Prime+Probe methods to mount cache side-channel attacks on a popular OpenSSL implementation of AES. The attacks work across cores in the cross-VM setting and succeeds to recover the full encryption keys in a short time—suggesting a practical threat to real-life systems. Our results show that there is strong information leakage through cache in virtualized systems and the software implementations of AES must be approached with caution. Indeed, for the first time, we demonstrate the effectiveness of the attack across co-located instances on the Amazon EC2 cloud. We argue that for secure usage of world's most commonly used block cipher such as AES, one should rely on secure, constant-time hardware implementations offered by CPU vendors.

Due to explosive increase in the amount of data produced recently, cloud storage system is required to offer high and stable performance. However, VM (Virtual Machine) migration may result in lowered storage service performance. Especially, in an environment where the host-side flash cache is used in a cloud system, the existing warmed up cache is lost and the problematic cold start begins at a new cache due to a VM migration. In this paper, we first demonstrate and analyze the cold start problem and then propose Cachemior (Cache migrator) which enables efficient hot start of the flash cache.

The compiled native code generated by a just-in-time (JIT) compiler in managed language virtual machines (VM) is placed in a region of memory called the code cache. Code cache management (CCM) in a VM is responsible to find and evict methods from the code cache to maintain execution correctness and manage program performance for a given code cache size or memory budget. Effective CCM can also boost program speed by enabling more aggressive JIT compilation, powerful optimizations, and improved hardware instruction cache and I-TLB performance. Though important, CCM is an overlooked component in VMs. We find that the default CCM policies in Oracle’s production-grade HotSpot VM perform poorly even at modest memory pressure. We develop a detailed simulation-based framework to model and evaluate the potential efficiency of many different CCM policies in a controlled and realistic, but VM-independent environment. We make the encouraging discovery that effective CCM policies can sustain high program performance even for very small cache sizes. Our simulation study provides the rationale and motivation to improve CCM strategies in existing VMs. We implement and study the properties of several CCM policies in HotSpot. We find that in spite of working within the bounds of the HotSpot VM’s current CCM sub-system, our best CCM policy implementation in HotSpot improves program performance over the default CCM algorithm by 39%, 41%, 55%, and 50% with code cache sizes that are 90%, 75%, 50%, and 25% of the desired cache size, on average.

Today most of the systems run general purpose applications which have non-uniform memory accesses. This work proposes a new technique for last level cache (LLC) organization, named as hashed cache using overuse distance for ways sharing (HCOWS), to reorganize the cache memory of a system to contribute better performance in multicore systems. The proposed HCOWS technique can be applied on multicore platforms, to improve performance of cache memory and hence of a system for myriad of general purpose applications. A new mapping technique is employed using universal hashing which is highly random. This scheme employs sharing of ways instead of set sharing. This work evaluates the performance of cache hierarchy using average memory access time. HCOWS scheme-based system has given better miss rate reduction when compared with the simple baseline system by 33 % in dual core, 37 % in quad core and 41 % in octa core, which substantiates the fact that increased number of cores dispenses better performance. The proposed scheme has high impact on miss rate reduction with the increase in size of LLC rather than the associativity of LLC.

Several recent studies have shown that router FIB caching offers excellent hit rates with cache sizes that are an order of magnitude smaller than the original forwarding table. However, hit rate alone is not sufficient - other performance metrics such as memory accesses, robustness to cache attacks, queuing delays from cache misses, etc., should be considered before declaring FIB caching viable.In this paper we tackle several pragmatic questions about FIB caching. We characterize cache performance in terms of memory accesses and delay due to cache misses. We study cache robustness to pollution attacks and show that in order to evict the most popular prefixes an attacker must sustain packet rates higher than the link capacity. We show that caching was robust even during a recent flare of NTP attacks. We carry out a longitudinal study of cache hit rates over four years and show the hit rate is unchanged over that duration. We characterize cache misses to determine which services are impacted by FIB caching. We conclude that FIB caching is viable by several metrics, not just impressive hit rates.

The flash-based SSD is used as a tiered cache between RAM and HDD. Conventional schemes do not utilize the nonvolatile feature of SSD and cannot cache write requests. Writes are a significant, or often dominant, fraction of storage workloads. To cache write requests, the SSD cache should persistently and consistently manage its data and metadata, and guarantee no data loss even after a crash. Persistent cache management may require frequent metadata changes and causes high overhead. Some researchers insist that a nonvolatile persistent cache requires new additional primitives that are not supported by general SSDs in the market. We proposed a fully persistent read/write cache, which improves both read and write performance, does not require any special primitive, has a low overhead, guarantees the integrity of the cache metadata and the consistency of the cached data, even during a crash or power failure, and is able to recover the flash cache quickly without any data loss. We implemented the persistent read/write cache as a block device driver in Linux. Our scheme aims at virtual desktop infra servers. So the evaluation was performed with massive, real desktop traces of five users for ten days. The evaluation shows that our scheme outperforms an LRU version of SSD cache by 50% and the read-only version of our scheme by 37%, on average, for all experiments. This paper describes most of the parts of our scheme in detail. Detailed pseudo-codes are included in the Appendix.

The emergence of new memory technologies provides us with opportunity to enhance the properties of existing memory architectures. One such technology is Phase Change Memory (PCM) which boasts superior scalability, power savings, non-volatility, and a performance competitive to Dynamic Random Access Memory (DRAM). In this paper, we propose a write buffer architecture for Solid-State Drives (SSDs) which attempts to exploit PCM as a DRAM alternative while alleviating its issues such as long write latency, high write energy, and finite endurance. To this end and based on thorough I/O characterization of desktop and enterprise applications, we propose a hybrid DRAM-PCM SSD cache design with an intelligent data movement scheme. This architecture manages to improve energy efficiency while enhancing performance and endurance. To study the design trade-offs between energy, performance, and endurance, we augmented Microsoft's DiskSim SSD model with a detailed hybrid cache using PCM and DRAM parameters from a rigorous survey of device prototypes. We study the design choices of implementing different PCM and DRAM arrays to achieve the best trade-off between energy and performance. The results display up to 77 percent power savings compared to a DRAM cache and up to percent reduction in request response time for a variety of workloads, while greatly improving disk endurance.

Cache memory, although important for boosting application performance, is also a source of execution time variability, and this makes its use difficult in systems requiring worst-case execution time (WCET) guarantees. Cache locking is a promising approach for simplifying WCET estimation and providing predictability, and hence, several commercial processors provide ability for locking cache. However, cache locking also has several disadvantages (e.g., extra misses for unlocked blocks, complex algorithms required for selection of locking contents) and hence, a careful management is required to realize the full potential of cache locking. In this article, we present a survey of techniques proposed for cache locking. We categorize the techniques into several groups to underscore their similarities and differences. We also discuss the opportunities and obstacles in using cache locking. We hope that this article will help researchers gain insight into cache locking schemes and will also stimulate further work in this area.

The internal state of the complex modern processors often needs to be dumped out frequently during postsilicon validation. Since the caches hold most of the state, the volume of data dumped and the transfer time are dominated by the large caches present in the architecture. The limited bandwidth to transfer data present in these large caches off-chip results in stalling the processor for long durations when dumping the cache contents off-chip. To alleviate this, we propose to transfer only those cache lines that were updated since the previous dump. Since maintaining a bit-vector with a separate bit to track the status of individual cache lines is expensive, we propose two methods: 1) where a bit tracks multiple cache lines and 2) an Interval Table which stores only the starting and ending addresses of continuous runs of updated cache lines. Both methods require significantly lesser space compared with a bit-vector, and allow the designer to choose the amount of space to allocate for this design-for-debug feature. The impact of reducing storage space is that some nonupdated cache lines are dumped too. We attempt to minimize such overheads. We propose a scheme to share such cache update tracking hardware (or Update Trackers) across multiple caches in case of physically distributed caches so that they are replicated fewer times, thereby limiting the area overhead. We show that the proposed Update Trackers occupy less than 1% of cache area for both the shared and distributed caches.

Spin-Transfer Torque RAM (STT-RAM) has the advantages of circuit density and ignorable leakage power. However, it suffers from the bad write latency and poor write power consumption. Therefore, it is difficult to replace entire SRAM with STT-RAM in the L1 cache, but we can relax the retention time of STT-RAM cell to improve its write performance and replace some of the SRAM capacity to reduce leakage power. In this paper, we propose a locality-aware approach for L1 cache design with hybrid SRAM and volatile STT-RAM configuration. Based on the principle of cache locality, data block is mapped to SRAM firstly to reduce write latency and write energy, and is moved to volatile STT-RAM to reduce leakage power consumption. After a time period when there is no access of a data block in the volatile STT-RAM, we then stop its refresh operations to further reduce power consumption. Experimental results show that in comparison with the SRAM only L1 cache configuration, our hybrid cache configuration and data migration methodology reduce energy consumption by about 15–20%, with only nearly to 5% of latency overhead. Also when comparing to the STT-RAM only L1 cache configuration, we reduce memory access latency nearly to 20% with close or even better energy consumption.

Shared cache in modern multi-core systems has been considered as one of the major factors that degrade system predictability and performance. How to manage the shared cache for real-time multi-core systems in order to optimize the system performance while guaranteeing the system predictability is an open issue. In this paper, we present a reconfigurable cache architecture which supports dynamic cache partitioning at hardware level and a framework that can exploit cache management for real-time MPSoCs. The proposed reconfigurable cache allows cores to dynamically allocate cache resource with minimal timing overhead while guaranteeing strict cache isolation among the real-time tasks. The cache management framework automatically determines time-triggered schedule and cache configuration for each task to minimize cache misses while guarantee the real-time constraints. We evaluate the proposed framework with respect to different numbers of cores and cache modules and prototype the constructed MPSoCs on FPGA. Our experiments show that, our automatic framework brings significant benefits over the state-of-the-art cache management strategies when testing 27 benchmark programs on the constructed MPSoCs.

This paper investigates cache placement on a cooperative cache built from individual client caches in an online social network or web service. We use a service that maintains a mapping between content and the clients that cache it, and propose cache placement schemes that leverage relationships between clients (for example, social links) and workload statistics, proactively placing content on clients that are likely to access it. We evaluate efficacy through simulation, comparing our schemes against commonly used cache placement algorithms as well as optimal placement. We synthesize a workload to match characteristics of online social networks. Simulation results of our proposed caching schemes impose moderate network overhead and show considerable improvement to the client's cache hit ratio, even under churn.

File I/O buffer caching plays an important role to narrow the wide speed gap between main memory and secondary storage. However, data loss or inconsistency may occur if the system crashes before updated data in the buffer cache is flushed to storage. Thus, most operating systems adopt a daemon that periodically flushes dirty data to secondary storage. This periodic flush degrades the caching efficiency seriously because most write requests lead to direct storage accesses. We show that periodic flush accounts for 30-70 percent of the total write traffic to storage. To remove this inefficiency, this paper presents a new buffer cache architecture that adopts a small amount of non-volatile memory to maintain modified data. This novel buffer cache architecture removes almost all storage accesses due to periodic flush operations without any loss of reliability. It also improves the buffer cache performance through space-efficient management techniques, such as delta-write and fragment-grouping. Our experimental results show that the proposed buffer cache reduces the storage write traffic by 44.3 percent and also improves the throughput by 23.6 percent on average.

Domain wall memory (DWM) is a recently developed spin-based memory technology in which several bits of data are densely packed into the domains of a ferromagnetic wire. DWM has shown great promise in enabling non-volatile memory with very high density and energy efficiency, and has been explored for secondary storage and off-chip memory. In this work, we explore the use of DWM within the on-chip cache hierarchy of general purpose computing platforms. Our work is motivated by the fact that DWMs enable much higher density compared to SRAM, DRAM, and other spin-based memory technologies such as STT-MRAM. However, DWMs also pose the unique challenge of serial access to the bits stored in a cell, leading to large and variable access latencies. In addition, DWMs share the inherent write inefficiency of other spin-based memories. We propose TapeCache, a DWM-based cache design that employs device, circuit, and architectural techniques to address these challenges. At the device level, we perform write optimization by employing a new write mechanism based on domain wall shifts to achieve fast, energy-efficient writes in DWM. At the circuit level, we propose different DWM bit-cell designs that are tailored to the distinct architectural requirements of different levels in the cache hierarchy. At the architecture level, we propose a new cache organization and suitable management policies that mitigate the performance penalty arising from serial access to bits in a DWM cell. We show that the holistic device-circuit-architecture co-design enables all the levels in the cache hierarchy to be realized using DWM and benefit from its improved density. Over a wide range of SPEC CPU 2006 benchmarks, TapeCache achieves an average energy improvement of 7.5 $\times$ , with virtually identical performance and 7.8 $\times$ improvement in area, compared to an iso-capacity SRAM cache. Compared to an iso-capacity STT-MRAM cache, TapeCache obtains 3.1 $\times$ improvement in area and 2 $\times$ average energy savings along with 1.1 percent performance improvement.

We propose FusedCache , a two-level set-associative Racetrack memory (RM) cache design that utilizes RM’s high density for providing fast uniform access at one level, and non-uniform access at the next. FusedCache is well suited for private L1/L2 caches enforcing alignment of L1 data with the RM access points with the remaining non-aligned data serving as L2. It uses traditional LRU eviction for L1 misses. Promotion and demotion between L1 and L2 are performed through shifts and, when necessary, background swap operations. These swap operations do not require physical stores or loads, making accesses both faster and more energy efficient. Further, unlike a traditional inclusive cache hierarchy, fused L1 cache lines naturally exist in L2 avoiding duplicated storage and tag structures, promotions, and evictions. L1 status on each track is strictly enforced by track LRU maintenance and background swapping. Our results demonstrate that compared to an iso-area L1 SRAM cache replacement, FusedCache improves application performance by 7 percent while reducing cache energy by 33 percent. Compared to an iso-capacity two level (L1/L2) SRAM cache replacement, FusedCache provides similar performance with a dramatic 69 percent cache energy reduction. Compared to a TapeCache L1 scheme, FusedCache gains a 7 percent performance improvement with a 6 percent cache energy saving which translates to a 13 percent improvement in energy-delay product.

On-chip dynamic random access memory (DRAM) cache has been recently employed in the memory hierarchy to mitigate the widening latency gap between high-speed cores and off-chip memory. Two important parameters are the DRAM cache miss rate (D$-MR) and the DRAM cache hit latency (D$-HL), as they strongly influence the performance. These parameters depend upon the DRAM set mapping policy. Recently proposed DRAM set mapping policies are predominantly optimized for either D$-MR or D$-HL. We propose novel DRAM set mapping policies that simultaneously reduce D$-MR (via high associativity) and D$-HL (via improved row buffer hit rates). To further improve the D$-HL, we propose a small and low latency DRAM Tag cache (DTC) structure that can quickly determine whether an access to the DRAM cache will be a hit or a miss. The performance of the proposed DTC depends upon the DTC hit rate. To increase it, we present a novel DTC insertion policy that also increases the DTC hit rate. We investigate the latency and miss rate tradeoffs when designing a DRAM cache hierarchy and analyze the effects of different policies on the overall performance. We evaluate our policies on a wide variety of workloads and compare its performance with three recent proposals for on-chip DRAM caches. For a 16-core system, our set mapping policy along with our DTC and its adaptive DTC insertion policy improve the harmonic mean instruction per cycle throughput by 25.4%, 15.5%, and 7.3% compared to state-of-the-art, while requiring 55% less storage overhead for DRAM cache hit/miss prediction.

In this paper, we investigate the feasibility of voltage adjustment in a large capacity cache, and propose the architecture of voltage-adaptable nonuniform cache access (VANUCA) that exploits near-threshold computing and multivoltage domain to approach the limit of $V_{\rm dd}$ in a low-power cache. However, the adoption of near-threshold voltage (NTV) leads to a rocketing error probability in SRAM arrays, which has to be addressed by effective fault-tolerant techniques. Instead of using error correction code or data duplication, the VANUCA exploits the natural data redundancy across the whole memory hierarchy to enable fast fault recovery in the NTV cache. Based on the discovered data resilience and the multi- $V_{\rm dd}$ architecture, the VANUCA is able to match vulnerable/invulnerable data clusters to available high-/low-voltage domains by utilizing the data migration mechanism in dynamic NUCA. The proposed VANUCA includes two important architectural techniques: 1) static assignment that assumes a fixed voltage domain partitioning and 2) DataMotion that dynamically fits the working set into heterogeneous cache banks through $V_{\rm dd}$ switching. Experimental results show that the VANUCA achieves considerable improvements in energy efficiency over the conventional single-voltage domain NUCA cache.

An effective design of cache memory is an important aspect in computer architecture to improve the system performance. In this work, we study the effect of reducing the cache comparisons to map the cache address on the performance experimentally and analytically. Cache miss penalties have drastic impact on the systems’ performance. To overcome this, we propose a novel tag access scheme, which uses a partial comparison unit called n-bit comparator and use multiple search methods inside the data cache to improve the cache performance by reducing cache access time. Partial tag comparison (PTC) enables the cache to compare the tag in multi-stage techniques starting with the least significant bits (LSBs). Thus, useless tag comparison and number of tag bits being compared can be effectively reduced, hence reaching the requested tag is faster and the cache hit time is reduced. Simulation results show that the proposed approach outperforms conventional mapping techniques. The PTC technique improves the hit time in 2-bank and 4-bank fully associative caches by 70–96% and 67–88% over a cache with full tag comparison. Moreover, the proposed technique provides the minimum hit time when using a hash searching method rather than other searching methods: linear and binary.

Emerging nonvolatile memory technologies, such as spin-transfer torque RAM or resistive RAM, can increase the capacity of the last-level cache (LLC) in a latency and power-efficient manner. These technologies endure $10^{9}$ – $10^{12}$ writes per cell, making a nonvolatile cache (NV-cache) with a lifetime of dozens of years under ideal working conditions. However, nonuniformity in writes to different cache lines considerably reduces the NV-cache lifetime to a few months. Writes to cache lines can be made uniformly by wear-leveling. A suitable wear-leveling for NV-cache should not incur high storage and performance overheads. We propose a novel, simple, and effective wear-leveling technique with negligible performance overhead of $13\times $ for different cache configurations.

Spin-transfer torque RAM (STT-RAM) has emerged as an energy-efficient and high-density alternative to SRAM for large on-chip caches. However, its high write energy has been considered as a serious drawback. Hybrid caches mitigate this problem by incorporating a small SRAM cache for write-intensive data along with an STT-RAM cache. In such architectures, choosing cache blocks to be placed into the SRAM cache is the key to their energy efficiency. This paper proposes a new hybrid cache architecture called prediction hybrid cache. The key idea is to predict write intensity of cache blocks at the time of cache misses and determine block placement based on the prediction. We design a write intensity predictor that realize the idea by exploiting a correlation between write intensity of blocks and memory access instructions that incur cache misses of those blocks. It includes a mechanism to dynamically adapt the predictor to application characteristics. We also design a hybrid cache architecture in which write-intensive blocks identified by the predictor are placed into the SRAM region. Evaluations show that our scheme reduces energy consumption of hybrid caches by 28 percent (31 percent) on average compared to the existing hybrid cache architecture in a single-core (quad-core) system.

In resource-constrained embedded systems, on-chip cache memories play an important role in both performance and energy consumption. In contrast to read operations, scant regard has been paid to optimizing write operations even though the energy consumed by write operations in the data cache constitutes a large portion of the total energy consumption. Consequently, this paper proposes a write buffer-oriented (WO) cache architecture that reduces energy consumption in the L1 data cache. Observing that write operations are very likely to be merged in the write buffer because of their high localities, we construct the proposed WO cache architecture to utilize two schemes. First, the write operations update the write buffer but not the L1 data cache, which is updated later by the write buffer after the write operations are merged. Write merging significantly reduces write accesses to the data cache and, consequently, energy consumption. Second, we further reduce energy consumption in the write buffer by filtering out unnecessary read accesses to the write buffer using a read hit predictor. In this paper, we show that the proposed WO cache architecture is applicable to the conventional embedded processors that support both write-through and write-back policies. Further, the experimental results verify that the proposed cache architecture reduces energy consumption in data caches up to 14%.

Thanks to superior leakage energy efficiency compared to SRAM cells, STTRAM cells are considered as a promising alternative for a memory element in on-chip caches. However, the main disadvantage of STTRAM cells is high write energy and latency. In this paper, we propose a lowcost write filter (WF) cache which resides between the load/store queue and STTRAM-based L1 data cache. To maximize efficiency of the WF cache, the line allocation and access policies are optimized for reducing energy consumption of STTRAM-based L1 data cache. By efficiently filtering the write operations in the STTRAM-based L1 data cache, our proposed WF cache reduces energy consumption of the STTRAM-based L1 data cache by up to 43.0% compared to the case without the WF cache. In addition, thanks to the fast hit latency of the WF cache, it slightly improves performance by 0.2%.

With the development of wireless cloud computing, video caching in the radio access network (RAN) of cellular networks has attracted extensive attention due to its lower delay and higher resource utilization efficiency. Nevertheless, existing video cache management mostly made decisions only according to the video coding requirements, without considering users' individual requirements for the video service and without making full use of the abundant network-side information in real time or from statistics. In this paper, we propose a new QoE (quality of experience)-driven video cache management scheme with the consideration of the parameters from three parties (i.e. client, base station, and RAN cache server) for video provisioning, with statistics of video popularities and under limited cache capacity. Specifically, through experiments we establish the mapping relationship between the QoE value and the three key parameters (i.e. the request rate from the client, the bandwidth of air interface, and the response rate of the cache server). Firstly, we allocate different gross caches for different video clips according to their popularities. Secondly, we optimize the cache space allocation for each individual video clip based on the QoE mapping relationship and the different models of the request rate and the bandwidth, with the convex optimization method and the Lagrange multiplier solution. The experiments results indicate that the proposed video cache scheme has better QoE performance under the constraints of the total cache capacity, specific distributions of the request rate and the bandwidth.

Providing a real-time cloud service requires simultaneously retrieving a large amount of data. How to improve the performance of file access becomes a great challenge. This paper first addresses the preconditions of dealing with this problem considering the requirements of applications, hardware, software, and network environments in the cloud. Then, a novel distributed layered cache system named HDCache is proposed. HDCahe is built on the top of Hadoop Distributed File System (HDFS). Applications can integrate the client library of HDCache to access the multiple cache services. The cache services are built up with three access layers an in-memory cache, a snapshot of the local disk, and a network disk provided by HDFS. The files loaded from HDFS are cached in a shared memory which can be directly accessed by the client library. In order to improve robustness and alleviate workload, the cache services are organized in a peer-to-peer style using a distributed hash table and every cached file has three replicas scattered in different cache service nodes. Experimental results show that HDCache can store files with a wide range in their sizes and has the access performance in a millisecond level under highly concurrent environments. The tested hit ratio obtained from a real-world cloud serviced is higher than 95 %.

The accurate and quantitative analysis of the cache behavior in a Chip Multi-Core (CMP) machine has long been a challenging work. So far there has been no practical way to predict the cache allocation, i.e., allocated cache size, of a running program. Lots of applications, especially those that have many interactions with the users, cache allocation should be estimated with high accuracy since its variation is closely related to the stability of system performance which is important to the efficient operation of servers and has a great influence on user experience. For these interests, this paper proposes an accurate prediction model for the allocation of the last level cache (LLC) of the co-runners. With a precise cache allocation predicted, we further implemented a performance-stability-oriented co-runner scheduling algorithm which aims to maximize the number of co-runners running in performance-stable state and minimize the performance variation of the unstable ones. We demonstrate that the proposed prediction algorithm exhibits a high accuracy with an average error of 5.7 %; and the co-runner scheduling algorithm can find the optimal solution under the specified target with a time complexity of O(n).

Performance degradation tolerance (PDT) has been shown to be able to effectively improve the yield, reliability, and lifetime of an electronic product. The focus of PDT is on the particular performance degrading faults (pdef) that only incur some performance degradation of a system without inducing any computation errors. The basic idea is that as long as the defective chips containing only the pdef can provide acceptable performance for some applications, they may still be marketable. Critical issues of PDT to be addressed include the portion of the pdef in a faulty chip and their induced performance degradation. For a typical cache design, most of the possible faults are not pdef. In this brief, we propose a cache redesign method, called PDT cache, where all functional faults in the data-storage cells of a cache (major part of the cache) can be transformed into pdef. By transforming this large number of faults into pdef, a faulty cache becomes much more likely to be still marketable. The proposed design exploits the existing hardware resources and the inherent error resilience scheme to reduce the incurred hardware overhead. The logic synthesis results show that the incurred hardware overhead is only 6.29% for a 32-kB cache. We also evaluate the induced performance degradation under various fault densities using the CPU2000 and CPU2006 benchmark programs. The results show that for a 32-kB cache design, when the fault density is

The cache memory consumes a large proportion of the energy used by a processor. In the on-chip cache, the translation lookaside buffer (TLB) accounts for 20-50% of energy consumption of the on-chip cache. To reduce energy consumption caused by TLB accesses, a virtual cache can be accessed by virtual addresses which are issued by a processor directly. However, a virtual cache may result in the synonym problem. In this paper, we propose low-cost synonym detection hardware and a synonym data coherence mechanism. These reduce the energy consumption incurred by TLB lookups, and maintain synonym data consistency in the virtual cache. The proposed synonym detection hardware efficiently reduces the number of blocks that must be looked up in a virtual cache for saving energy. In addition, the proposed synonym data coherence mechanism also reduces the number of invalidated blocks in the virtual cache to prevent the destruction of cache locality. The simulation results show that our proposed energy-aware virtual cache consumes 51%, 27%, and 20% less energy than the traditional physical cache, traditional virtual cache, and synonym lookaside buffer (SLB), respectively. In addition, our proposed design shows almost the same static energy consumption as SLB, and reduces static energy consumption by about 20% compared with the traditional physical cache and virtual cache.

Most modern processors have some cache memories that are much faster than main memory. These cache memories function well if temporal or spatial localities in programs are enhanced; therefore, the continuous accesses to the same array that improves the localities can enhance effective utilization of the cache memories. In addition, because a multidimensional array can be regarded as an array of lower dimensional arrays, the continuous accesses to array references with the most similar indexes can enhance the utilization of them further. We propose a new code motion algorithm called MDGLIA that improves utilization of cache memory. MDGLIA moves each array reference immediately after the preceding references accessing the same array with the most similar indexes, and then delays it as late as possible without changing the access order. These two-step code motions contribute to not only improvement of the cache efficiency in the overall program but also suppression of register pressure. We implemented MDGLIA in a real compiler and evaluated it for matrix multiplication and SPEC benchmarks. The experimental results show that our algorithm reduces the number of cache misses in most cases.

This paper aims to tackle two fundamental memory bottlenecks: limited off-chip bandwidth and long access latency. Our approach exploits the inherent error resilience of a wide range of applications through an approximation technique, called Rollback-Free Value Prediction (RFVP). When certain safe-to- approximate load operations miss in the cache, RFVP predicts the requested values. However, RFVP does not check for or recover from load value mispredictions, hence avoiding the high cost of pipeline flushes and re-executions. RFVP mitigates long memory access latencies by enabling the execution to continue without stalling for these accesses. To mitigate the limited off-chip bandwidth, RFVP drops a fraction of load requests which miss in the cache after predicting their values. The drop rate then becomes a knob to control the tradeoff between performance/en- ergy efficiency and output quality. Our extensive evaluations show that RFVP, when used in GPUs, yields significant performance improvements and energy reductions for a wide range of quality loss levels.

Next generation large single-chip multicores will process massive data with varying degree of locality. Harnessing on-chip data locality to optimize the utilization of on-chip cache and network resources is of fundamental importance. We propose a locality-aware selective data replication protocol for the last-level cache (LLC). The goal is to lower memory access latency and energy by only replicating cache lines with high reuse in the LLC slice of the requesting core, while simultaneously keep the off-chip miss rate low. The approach relies on low-overhead yet highly accurate in-hardware runtime cache line level classifier that only allows replication of cache lines with high reuse. Furthermore, a classifier captures the LLC pressure at the existing replica locations and adapts its replication decision accordingly. On a set of parallel benchmarks, the proposed protocol reduces overall energy by 14.7, 10.7, 10.5, and 16.7 % and completion time by 2.5, 6.5, 4.5, and 9.5 % when compared to the previously proposed Victim Replication, Adaptive Selective Replication, Reactive-NUCA, and Static-NUCA LLC management schemes. An efficient classifier implementation is evaluated with an overhead of 5.44 KB, which translates to only 1.58 % on top of the Static-NUCA baseline's cache related per-core storage.

A program can benefit from improved cache block utilization when contemporaneously accessed data elements are placed in the same memory block. This can reduce the program's memory block working set and thereby, reduce the capacity miss rate. We formally define the problem of data packing for arbitrary number of blocks in the cache and packing factor (the number of data objects fitting in a cache block) and study how well the optimal solution can be approximated for two dual problems. On the one hand, we show that the cache hit maximization problem is approximable within a constant factor, for every fixed number of blocks in the cache. On the other hand, we show that unless P=NP, the cache miss minimization problem cannot be efficiently approximated.

The efficiency of the cache mapping technique depends on how the c ache lines are organized and the way that is used to look for and hit the target cache line. In this paper, an efficient technique is proposed to obtain a significant improvement in average hit time of a li ne in the cache. The paper presents Distributive Comparison Approach (DCA) that significantly mini mizes the hit time and improve cache hit ratio. The efficient of DCA is based on how the cache line s are compared and picked up the coveted one leading to a low cache hit ratio. In DCA, the cache line is assigned by multi tags where each individual tag is only one character. Then, instead of one line tag of complete characters per a comparison cycle, the comparator is flushed by multi tags of different lines i n the cache. Also the cache lines that are come from the main memory classified into two groups; even and od d line's tags to reject the unwanted lines form the multi-tag comparison. These two procedures practicall y speed up the repelling of misfit tagged lines and consequently the hitting of the target line in the ca che. Simulation results show that the DCA outperforms well-known mapping techniques including FAMT and SMT.

Named data networking (NDN) aims at efficient content delivery and reducing the redundancy of data transmission. In NDN, one of the most important issues is how to utilize the cached contents to reduce the user's response delay and improve the utilization of the cache resources. In this paper, an intelligent content discovery system to fully utilize the cache resources, called DENA, is proposed, which encompasses the deep exponential network-based cache announcement and cache exergy-based cache replacement algorithms. DENA provides shortcut paths for interest packets to access targeted data packets under instructions of AT (announcement table) constructed by a precision cached content announcement algorithm with low communication overhead. In the meantime, a cache replacement algorithm to assist the cache announcement is also proposed. The method uses a so-called cache exergy index to instruct the replacement of cached content. The simulation results indicate that the system improves the utilization of cache resources and the network capacity while keeping the network control traffic at a relatively low level.

We present a new parallel optimization method to optimize cache memory designs.It obtains optimal designs of cache memories with respect to designer's metrics.We also reduce the execution time and energy consumption up to 64.43% and 91.69%.As a result, we have outlined other interesting future research lines in this area. Current embedded systems are specifically designed to run multimedia applications. These applications have a big impact on both performance and energy consumption. Both metrics can be optimized selecting the best cache configuration for a target set of applications. Multi-objective optimization may help to minimize both conflicting metrics in an independent manner. In this work, we propose an optimization method that based on Multi-Objective Evolutionary Algorithms, is able to find the best cache configuration for a given set of applications. To evaluate the goodness of candidate solutions, the execution of the optimization algorithm is combined with a static profiling methodology using several well-known simulation tools. Results show that our optimization framework is able to obtain an optimized cache for Mediabench applications. Compared to a baseline cache memory, our design method reaches an average improvement of 64.43 and 91.69% in execution time and energy consumption, respectively.

With the widespread use of Internet, the scale of mobile data traffic grows explosively, which makes 5G networks in cellular networks become a growing concern. Recently, the ideas related to future network, for example, Software Defined Networking (SDN), Content-Centric Networking (CCN), and Big Data, have drawn more and more attention. In this paper, we propose a service-customized 5G network architecture by introducing the ideas of separation between control plane and data plane, in-network caching, and Big Data processing and analysis to resolve the problems traditional cellular radio networks face. Moreover, we design an optimal routing algorithm for this architecture, which can minimize average response hops in the network. Simulation results reveal that, by introducing the cache, the network performance can be obviously improved in different network conditions compared to the scenario without a cache. In addition, we explore the change of cache hit rate and average response hops under different cache replacement policies, cache sizes, content popularity, and network topologies, respectively.

This research proposes a phase-change memory (PCM) based main memory system with an effective combination of a superblock-based adaptive buffering structure and its associated set divisible last-level cache (LLC). To achieve high performance similar to that of dynamic random-access memory (DRAM) based main memory, the superblock-based adaptive buffer (SABU) is comprised of dual DRAM buffers, i.e., an aggressive superblock-based pre-fetching buffer (SBPB) and an adaptive sub-block reusing buffer (SBRB), and a set divisible LLC based on a cache space optimization scheme. According to our experiment, the longer PCM access latency can typically be hidden using our proposed SABU, which can significantly reduce the number of writes over the PCM main memory by 26.44%. The SABU approach can reduce PCM access latency up to 0.43 times, compared with conventional DRAM main memory. Meanwhile, the average memory energy consumption can be reduced by 19.7%.