Search

This paper presents an innovative auto batching scheme designed to optimize Long Short-Term Memory (LSTM) inference on Field-Programmable Gate Array (FPGA) platforms. Existing block batching methods face challenges with LSTM models that have large hidden sizes due to insufficient on-chip memory, which impedes prefetching and leads to repeated evictions and reloads, significantly reducing processing utilization. Our approach extends block batching with weight stationary block batching (WSBB), allowing computation without stalls regardless of prefetch availability.Additionally, bypass-enabled block batching (BEBB) ensures that even when on-chip memory is insufficient, it prevents contamination on-chip while fully leveraging off-chip memory bandwidth. Experimental results from both synthetic benchmarks (Deepbench suite) and real-world applications (RNN-T) validate the superior performance and efficiency of the proposed method. Our auto batching scheme demonstrates up to 3.7 times speedup over previous block batching while maintaining high computational efficiency, even with limited on-chip memory. Furthermore, the FPGA-based implementation of our scheme achieves a 5 times speedup over the CPU and 4.3 times higher energy efficiency (GFLOP/s/W) compared to the GPU.

To meet the performance demands of chip multiprocessors, chip designers have increased the capacity and hierarchy of cache memories. Accordingly, a shared lower-level cache reduces conflict misses by adopting a multi-way set-associative structure with high associativity. This structure allows fast access because it allows access to all the ways in the cache set in parallel. However, it consumes a large amount of dynamic energy. Therefore, various schemes have been proposed to increase the energy efficiency of the cache memory. These schemes use way prediction or partial comparison to reduce unnecessary way access. This paper proposes a way prediction algorithm suitable for a shared second-level cache with high associativity. This algorithm is based on real-time way access dominance detection (WADD). Through this detection, the proposed algorithm can determine the number and location of way candidates suitable for each partial access pattern among the fragmented access patterns owing to the first-level cache replacement policy and intermingled accesses by multiple cores. Through this process, the proposed algorithm can implement an efficient way prediction. Simulation results show that the WADD exhibits the highest energy efficiency among the comparison groups, thus reducing the energy-delay product by 13.5% compared with the conventional cache without way prediction. This result is achieved by reducing the way prediction penalty through fast detection and high prediction accuracy.

For timing closure of logic circuits, circuit designers must perform sign-offs on a variety of process, voltage, and temperature (PVT) conditions. Designs of advanced logic circuits involve a multitude of voltage islands and operating modes, each of which requires delay characterizations at nearby PVT corners. Furthermore, advanced technologies nodes suffer from corner explosion: while the impact of PVT variations is being exacerbated, process variations are also diversifying, increasing the number of operating conditions exponentially. This paper revisits the importance of cross-corner timing estimations and proposes a delay variation model to mitigate such corner explosion. Our objective is to reduce PVT corner characterization effort for timely static timing analysis on an exploding number of operating conditions. Our proposed Decomposed Propagation Vector Variation Model (DPVVM) decomposes propagation delay and timing constraints into the driving by the receiver cell and its driver cell; by scaling them separately, delay characterization effort is reduced to a fraction of time while realizing accurate timing estimations. We also propose Multi-Dimension Recomposition (MDR) scheme, which exploits a multitude of pre-characterized corners to further improve the consistency of cross-corner timing estimations. As a result, with only 8.0% of a corner characterization effort compared to the full-characterization, DPVVM combined with MDR achieves overall cross-corner timing estimation errors of 4.8% and 5.6% for single cells and complex logic circuits—or improvements of 69% and 61% over the conventional derating method, respectively. Our proposed method’s characterization overhead is 11% over the conventional derating method; the overhead is marginal, accounting for only 0.76% of a full-characterization time.

Recently, 3D-stacked dynamic random access memory (DRAM) has become a promising solution for ultra-high capacity and high-bandwidth memory implementations. However, it also suffers from memory wall problems due to long latency, such as with typical 2D-DRAMs. Although there are various cache management techniques and latency hiding schemes to reduce DRAM access time, in a high-performance system using high-capacity 3D-stacked DRAM, it is ultimately essential to reduce the latency of the DRAM itself. To solve this problem, various asymmetric in-DRAM cache structures have recently been proposed, which are more attractive for high-capacity DRAMs because they can be implemented at a lower cost in 3D-stacked DRAMs. However, most research mainly focuses on the architecture of the in-DRAM cache itself and does not pay much attention to proper management methods. In this paper, we propose two new management algorithms for the in-DRAM caches to achieve a low-latency and low-power 3D-stacked DRAM device. Through the computing system simulation, we demonstrate the improvement of energy delay product up to 67%.