Search

As data volume and complexity increase, conventional software‐based encryption methods face significant challenges, including vulnerability to attacks and high computational demands. This study proposes a hardware‐based cryptographic engine using a dual‐polarity memristive crossbar array with Ta/HfO2/RuO2 memristors, utilizing stochastic and deterministic operations for enhanced security and efficiency. The system integrates random key generation and XOR‐based logic operations within the memristive array, enabling robust encryption and decryption of binary data with a fully resistive data representation. Experimental validation of alphabet image data encryption and the array scale simulation of individual fingerprint authentication are conducted to validate the robustness of the proposed hardware and cryptographic method. The results demonstrate the potential of the proposed hardware for homomorphic encryption, enabling secure data processing without decryption, which can pave the way for memristive hardware to evolve into resistance‐based digital computing hardware. [ABSTRACT FROM AUTHOR]

Modern recommendation systems are widely used in modern data centers. The random and sparse embedding lookup operations are the main performance bottleneck for processing recommendation systems on traditional platforms as they induce abundant data movements between computing units and memory. ReRAM-based processing-in-memory (PIM) can resolve this problem by processing embedding vectors where they are stored. However, the embedding table can easily exceed the capacity limit of a monolithic ReRAM-based PIM chip, which induces off-chip accesses that may offset the PIM profits. Therefore, we deploy the decomposed model on-chip and leverage the high computing efficiency of ReRAM to compensate for the decompression performance loss. In this paper, we propose ARCHER, a ReRAM-based PIM architecture that implements fully on-chip recommendations under resource constraints. First, we make a full analysis of the computation pattern and access pattern on the decomposed table. Based on the computation pattern, we unify the operations of each layer of the decomposed model in multiply-and-accumulate operations. Based on the access observation, we propose a hierarchical mapping schema and a specialized hardware design to maximize resource utilization. Under the unified computation and mapping strategy, we can coordinate the inter-processing elements pipeline. The evaluation shows that ARCHER outperforms the state-of-the-art GPU-based DLRM system, the state-of-the-art near-memory processing recommendation system RecNMP, and the ReRAM-based recommendation accelerator REREC by 15.79×, 2.21×, and 1.21× in terms of performance and 56.06×, 6.45×, and 1.71× in terms of energy savings, respectively. [ABSTRACT FROM AUTHOR]

We introduce novel sparsity-aware in-DRAM matrix mapping techniques and a corresponding DRAM-based acceleration framework, termed SpDRAM, which utilizes a triple row activation scheme to efficiently handle sparse matrix-vector multiplication (SpMV). We found that reducing operations by sparsity relies heavily on how matrices are mapped into DRAM banks, which operate row by row. These banks operate row by row. From this insight, we developed two distinct matrix mapping techniques aimed at maximizing the reduction of row operations with minimal design overhead: Output-aware Matrix Permutation (OMP) and Zero-aware Matrix Column Sorting (ZMCS). Additionally, we propose a Multiplication Deferring (MD) scheme that leverages the prevalent bit-level sparsity in matrix values to decrease the effective bit-width required for in-bank multiplication operations. Evaluation results demonstrate that the combination of our in-DRAM acceleration methods outperforms the latest DRAM-based PIM accelerator for SpMV, achieving a performance increase of up to $7.54\times $ and a $22.4\times $ improvement in energy efficiency in a wide range of SpMV tasks.

Training of convolutional neural networks (CNN) consumes a lot of time and resources. While most previous works have focused on accelerating the convolutional (CONV) layer, the proportion of non-convolutional (non-CONV) layers, such as batch normalization, is gradually increasing during training. Non-CONV layers have low cache reuse and arithmetic intensity, thereby performance is limited by memory bandwidth. Processing-in-memory (PIM) can utilize wide memory bandwidth, making it suitable for acceleration of non-CONV layers. Therefore, it makes sense to perform the computationally complex CONV layer on the host and handle the memory bottleneck challenges of the non-CONV layer on the PIM. Further improved performance can be expected if they run simultaneously. However, memory access conflicts between the host and PIM are the biggest factors hindering performance improvement. Prior studies proposed bank partitioning to alleviate memory conflicts, but it is not effective because CNN training involves significant data sharing between CONV and non-CONV layers. In this paper, we propose a memory scheduling and CNN training flow for the pipelined execution of CONV layers on the host and non-CONV layers on PIM. First, instead of applying bank partitioning, the host and PIM exclusively access memory for a certain period to avoid the movement of shared data between host memory and PIM memory. The conditions for switching the memory access authority between the host and PIM are set per layer, taking into account memory access characteristics and the number of queued memory requests. Second, in the training flow, CONV and non-CONV layers are pipelined in units of output feature map channels. Specifically, for the backward pass, the non-CONV tasks of the feature map gradient calculation phase and the weight gradient update phase are rearranged so that they can be easily performed within CONV layers. Experimental results show that the proposed pipelined execution achieves an average speedup of 18.1% at the network level compared to the serial operation of the host and PIM.

Genetic algorithm (GA) has been extensively used for solving complex problems. Due to a high computational burden of finding solutions using GA, acceleration with hardware support has been a choice. In this paper, a GA accelerator based on the processing-in-memory (PIM) methodology to address the computational issue of GA is proposed. The proposed GA accelerator has a memristive crossbar array that can support parallelism with memory and computation combined. For letting the crossover operation for GA exploit massive parallelism provided by the array, a novel crossover scheme called aligned hybrid crossover is proposed, in which multiple multi-point crossovers coexist whose crossover bit positions are aligned. By using the memristive array, the mutation operation can also be done simultaneously for all required chromosome bits. Moreover, the fitness for weighted-sum computation-based 0-1 knapsack and subset-sum problems is shown to be evaluated in full parallel for the entire chromosomes in a population. The effects of memristance variation in the array on the fitness evaluation and the read margin are investigated. According to performance evaluation, the proposed GA accelerator having a $64\times 64$ memristive crossbar array is found to reduce the clock cycles significantly for performing operations like crossover, mutation, selection, and fitness evaluation. Specifically, for executing the generational GA with a chromosome population size of 64 with each chromosome having 64 bits, the total number of clock cycles required per generation is at least 10 times reduced as compared to conventional designs.

Analog and digital switching performances in a Ta/HfO2/RuO2 (THR) memristor are studied to implement a density‐based spatial clustering of applications with noise (DBSCAN) algorithm in a low‐power, parallel‐computing memristor crossbar structure. In the analog mode THR memristor, more than 256 states can be stored through a fine‐tuning process with a denoising scheme. The analog mode crossbar array facilitates Euclidean distance calculation between any points in the given graphic dataset. In the digital mode, the on/off ratio of more than three orders of magnitude between the binary states is achieved, providing functionality to cluster the data points with a reduced number of operations. The parallel computing capacity of the adopted crossbar decreases the time complexity of the original DBSCAN from O(n2) to O(n). Through array‐level simulations, the effectiveness of hardware functionality is validated using representative synthetic datasets and single‐cell RNA sequences datasets. [ABSTRACT FROM AUTHOR]

Today's systems are capable of storing large amounts of data in main memory. Particularly, in-memory DBMSs benefit from this development. However, the processing of data from the main memory necessarily has to run via the CPU. This creates a bottleneck, which affects the possible performance of the DBMS. Processing-In-Memory (PIM) is a paradigm to overcome this problem, which was not available in commercial systems for a long time. With the availability of UPMEM, a commercial product is finally available that provides PIM technology in hardware. In this work, we focus on the acceleration of the table scan, a fundamental database query operation. We show and investigate an approach that can be used to optimize this operation by using PIM. We evaluate the PIM scan in terms of parallelism and execution time in benchmarks with different table sizes and compare it to a traditional CPU-based table scan. The result is a PIM table scan that outperforms the CPU-based scan significantly. [ABSTRACT FROM AUTHOR]

The capabilities of artificial neural networks are rapidly evolving, so are the expectations for them to solve ever more challenging tasks in numerous everyday situations. Larger, more complex networks and the need to execute them efficiently on edge devices are the two counteracting requirements of this trend. Novel devices and computation techniques show promising characteristics to address this challenge. A huge design space covering different combinations of neural networks and hardware architectures using these technologies needs to be explored. An efficient design flow is, therefore, crucial for a good quality of service. This work reviews a wide range of simulation tools for novel memristive devices and analyzes their applicability for the design space exploration. A modular toolflow is proposed that shrinks down the large design space step-by-step using state-of-the-art optimization techniques and builds upon existing tools to find the best trade-offs between network accuracy and hardware requirements.

Summary: Non‐volatile memory (NVM) is an emerging technology being explored as an alternative to DRAM main memory in computing systems because of its persistence, higher storage density, lower energy consumption, and access latency close to DRAM. However, persistent memory systems must ensure data consistency on system failures, a property known as crash consistency. One of the main challenges in these systems is creating efficient checkpointing mechanisms in terms of performance and usability. Thus, it is necessary to remove persistence from the critical execution path and reduce the excessive number of writes to NVM caused by logging operations, resulting in increased memory bandwidth usage. Another limitation is that most proposed mechanisms restrict application source code to programming interfaces based on transactional models, typed as software‐based approaches. These factors make it challenging to adopt NVM systems. This article presents a software‐transparent mechanism based on dynamic epochs with logging operations via processing‐in‐memory and checkpoints integrated into the cache replacement policy. Compared to the previous best‐performing system, our strategy reduces 50.6% of writes to NVM. Furthermore, it does not increase the average memory bandwidth usage, providing crash consistency with less than 2% runtime overhead. [ABSTRACT FROM AUTHOR]

Processing-in-Memory (PiM), which combines a memory device with a Processing Unit (PU) into an integrated chip, has drawn special attention in the field of Artificial Intelligence semiconductors. Currently, in the development and commercialization of PiM's technology, there are challenges in the hegemony competition between the PU and memory device industries. In addition, there are challenges in finding strategic partnerships rather than independent development due to the complexity of technological development caused by heterogeneous chips. In this study, patent Main Path Analysis (MPA) is used to identify the majority and complementary groups between PU and memory devices for PiM. Subsequently, Document-to-Vector (Doc2Vec) and similarity-scoring analyses are used to determine the potential partners for technical cooperation required for PiM technology development for the majority group identified. According to the empirical results, PiM core technology is evolving from PU to memory device with an 'architecture-operation-architecture' design pattern. The ten ASIC candidates are identified for strategic partnerships with memory device suppliers. Those partnership candidates include several mobile AP firms, implying PiM's opportunities in the field of mobile applications. It suggests that memory device suppliers should prepare for different technology strategies for PiM technology development. This study contributes to the literature and high-tech industry via the proposed quantitative technology partnership model. [ABSTRACT FROM AUTHOR]

Stencil computations are commonly used in a wide variety of scientific applications, ranging from large-scale weather prediction to solving partial differential equations. Stencil computations are characterized by three properties: 1) low arithmetic intensity, 2) limited temporal data reuse, and 3) regular and predictable data access pattern. As a result, stencil computations are typically bandwidth-bound workloads, which experience only limited benefits from the deep cache hierarchy of modern CPUs. In this work, we propose Casper, a near-cache accelerator consisting of specialized stencil computation units connected to the last-level cache (LLC) of a traditional CPU. Casper is based on two key ideas: 1) avoiding the cost of moving rarely reused data throughout the cache hierarchy, and 2) exploiting the regularity of the data accesses and the inherent parallelism of stencil computations to increase overall performance. With small changes in LLC address decoding logic and data placement, Casper performs stencil computations at the peak LLC bandwidth. We show that by tightly coupling lightweight stencil computation units near LLC, Casper improves performance of stencil kernels by $1.65\times $ on average (up to $4.16\times $ ) compared to a commercial high-performance multi-core processor, while reducing system energy consumption by 35% on average (up to 65%). Casper provides $37\times $ (up to $190\times $ ) improvement in performance-per-area compared to a state-of-the-art GPU.

Copyright of Frontiers of Information Technology & Electronic Engineering is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The performance and efficiency of running large-scale datasets on traditional computing systems exhibit critical bottlenecks due to the existing “power wall” and “memory wall” problems. To resolve those problems, processing-in-memory (PIM) architectures are developed to bring computation logic in or near memory to alleviate the bandwidth limitations during data transmission. NAND-like spintronics memory (NAND-SPIN) is one kind of promising magnetoresistive random-access memory (MRAM) with low write energy and high integration density, and it can be employed to perform efficient in-memory computation operations. In this study, we propose a NAND-SPIN-based PIM architecture for efficient convolutional neural network (CNN) acceleration. A straightforward data mapping scheme is exploited to improve parallelism while reducing data movements. Benefiting from the excellent characteristics of NAND-SPIN and in-memory processing architecture, experimental results show that the proposed approach can achieve ∼2.6× speedup and ∼1.4× improvement in energy efficiency over state-of-the-art PIM solutions. [ABSTRACT FROM AUTHOR]

With the increasing amount of data, there is an urgent need for efficient sorting algorithms to process large data sets. Hardware sorting algorithms have attracted much attention because they can take advantage of different hardware’s parallelism. But the traditional hardware sort accelerators suffer “memory wall” problems since their multiple rounds of data transmission between the memory and the processor. In this paper, we utilize the in-situ processing ability of the ReRAM crossbar to design a new ReCAM array that can process the matrix-vector multiplication operation and the vector-scalar comparison in the same array simultaneously. Using this designed ReCAM array, we present ReCSA, which is the first dedicated ReCAM-based sort accelerator. Besides hardware designs, we also develop algorithms to maximize memory utilization and minimize memory exchanges to improve sorting performance. The sorting algorithm in ReCSA can process various data types, such as integer, float, double, and strings. We also present experiments to evaluate the performance and energy efficiency against the state-of-the-art sort accelerators. The experimental results show that ReCSA has 90.92×, 46.13×, 27.38×, 84.57×, and 3.36× speedups against CPU-, GPU-, FPGA-, NDP-, and PIM-based platforms when processing numeric data sets. ReCSA also has 24.82×, 32.94×, and 18.22× performance improvement when processing string data sets compared with CPU-, GPU-, and FPGA-based platforms. [ABSTRACT FROM AUTHOR]

Processing-in-memory (PIM) techniques have gained much attention from computer architecture researchers, and significant research effort has been invested in exploring and developing such techniques. Increasing the research activity dedicated to improving PIM techniques will hopefully help deliver PIM’s promise to solve or significantly reduce memory access bottleneck problems for memory-intensive applications. We also believe it is imperative to track the advances made in PIM research to identify open challenges and enable the research community to make informed decisions and adjust future research directions. In this survey, we analyze recent studies that explored PIM techniques, summarize the advances made, compare recent PIM architectures, and identify target application domains and suitable memory technologies. We also discuss proposals that address unresolved issues of PIM designs (e.g., address translation/mapping of operands, workload analysis to identify application segments that can be accelerated with PIM, OS/runtime support, and coherency issues that must be resolved to incorporate PIM). We believe this work can serve as a useful reference for researchers exploring PIM techniques.

The Discrete Fourier Transform (DFT) is essential for various applications ranging from signal processing to convolution and polynomial multiplication. The groundbreaking Fast Fourier Transform (FFT) algorithm reduces DFT time complexity from the naive O(n2)to O(nlogn), and recent works have sought further acceleration through parallel architectures such as GPUs. Unfortunately, accelerators such as GPUs cannot exploit their full computing capabilities since memory access becomes the bottleneck. Therefore, this paper accelerates the FFT algorithm using digital Processing-in-Memory (PIM) architectures that shift computation into the memory by exploiting physical devices capable of both storage and logic (e.g., memristors). We propose an O(logn)in-memory FFT algorithm that can also be performed in parallel across multiple arrays for high-throughput batched execution, supporting both fixed-point and floating-point numbers. Through the convolution theorem, we extend this algorithm to O(logn)polynomial multiplication – a fundamental task for applications such as cryptography. We evaluate FourierPIM on a publicly-available cycle-accurate simulator that verifies both correctness and performance, and demonstrate 5–15×throughput and 4–13×energy improvement over the NVIDIA cuFFT library on state-of-the-art GPUs for FFT and polynomial multiplication.

Resistive Random-Access Memory (ReRAM) based Processing-in-Memory (PIM) frameworks are proposed to accelerate the working process of DNN models by eliminating the data movement between the computing and memory units. To further mitigate the space and energy consumption, DNN model weight sparsity and weight pattern repetition are exploited to optimize these ReRAM-based accelerators. However, most of these works only focus on one aspect of this software/hardware co-design framework and optimize them individually, which makes the design far from optimal. In this paper, we propose PRAP-PIM, which jointly exploits the weight sparsity and weight pattern repetition by using a weight pattern reusing aware pruning method. By relaxing the weight pattern reusing precondition, we propose a similarity-based weight pattern reusing method that can achieve a higher weight pattern reusing ratio. Experimental results show that PRAP-PIM achieves 1.64× performance improvement and 1.51× energy efficiency improvement in popular deep learning benchmarks, compared with the state-of-the-art ReRAM-based DNN accelerators.

Copyright of Journal of National University of Defense Technology / Guofang Keji Daxue Xuebao is the property of NUDT Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Processing-in-Memory (PIM) devices have reemerged as a promise to mitigate the memory-wall and the limitations of transferring massive amount of data from main memories to the host processors. Novel memory technologies and the advent of 3D-stacked integration have provided means to compute data in-memory, either by exploring the inherent analog capabilities or by tight-coupling logic and memory. However, allowing the effective use of a PIM device demands significant and costly modifications on the host processor to support instructions offloading, cache coherence, virtual memory management, and communication between different PIM instances. This paper tackles these challenges by presenting a set of solutions to couple host and PIM with no modifications at host side. Moreover, we highlight the limitations presented on modern host processors that may prevent full performance extraction of the PIM devices. This work presents Plug N' PIM, a set of strategies and procedures to seamlessly couple host general-purpose processors and PIM devices. We show that with our techniques one can exploit the benefits of a PIM device with seamless integration between host and PIM, bypassing possible limitations on the host side. • Processing in-Memory devices must integrate with current host architectures. • The integration can be achieved without host modifications. • There is room for software optimization. • Processing in-Memory devices benefit from host features. [ABSTRACT FROM AUTHOR]

Data-intensive workloads and applications, such as machine learning (ML), are fundamentally limited by traditional computing systems based on the von-Neumann architecture. As data movement operations and energy consumption become key bottlenecks in the design of computing systems, the interest in unconventional approaches such as Near-Data Processing (NDP), machine learning, and especially neural network (NN)-based accelerators has grown significantly. Emerging memory technologies, such as ReRAM and 3D-stacked, are promising for efficiently architecting NDP-based accelerators for NN due to their capabilities to work as both high-density/low-energy storage and in/near-memory computation/search engine. In this paper, we present a survey of techniques for designing NDP architectures for NN. By classifying the techniques based on the memory technology employed, we underscore their similarities and differences. Finally, we discuss open challenges and future perspectives that need to be explored in order to improve and extend the adoption of NDP architectures for future computing platforms. This paper will be valuable for computer architects, chip designers, and researchers in the area of machine learning.

Many modern workloads, such as neural networks, databases, and graph processing, are fundamentally memory-bound. For such workloads, the data movement between main memory and CPU cores imposes a significant overhead in terms of both latency and energy. A major reason is that this communication happens through a narrow bus with high latency and limited bandwidth, and the low data reuse in memory-bound workloads is insufficient to amortize the cost of main memory access. Fundamentally addressing this data movement bottleneck requires a paradigm where the memory system assumes an active role in computing by integrating processing capabilities. This paradigm is known as processing-in-memory (PIM). Recent research explores different forms of PIM architectures, motivated by the emergence of new 3D-stacked memory technologies that integrate memory with a logic layer where processing elements can be easily placed. Past works evaluate these architectures in simulation or, at best, with simplified hardware prototypes. In contrast, the UPMEM company has designed and manufactured the first publicly-available real-world PIM architecture. The UPMEM PIM architecture combines traditional DRAM memory arrays with general-purpose in- order cores, called DRAM Processing Units (DPUs), integrated in the same chip. This paper provides the first comprehensive analysis of the first publicly-available real-world PIM architecture. We make two key contributions. First, we conduct an experimental characterization of the UPMEM-based PIM system using microbenchmarks to assess various architecture limits such as compute throughput and memory bandwidth, yielding new insights. Second, we present PrIM (Processing-In-Memory benchmarks), a benchmark suite of 16 workloads from different application domains (e.g., dense/sparse linear algebra, databases, data analytics, graph processing, neural networks, bioinformatics, image processing), which we identify as memory-bound. We evaluate the performance and scaling characteristics of PrIM benchmarks on the UPMEM PIM architecture, and compare their performance and energy consumption to their modern CPU and GPU counterparts. Our extensive evaluation conducted on two real UPMEM-based PIM systems with 640 and 2,556 DPUs provides new insights about suitability of different workloads to the PIM system, programming recommendations for software designers, and suggestions and hints for hardware and architecture designers of future PIM systems.

Processing-In-Memory (PIM) is an increasingly popular architecture aimed at addressing the 'memory wall' crisis by prioritizing the integration of processors within DRAM. It promotes low data access latency, high bandwidth, massive parallelism, and low power consumption. The skyline operator is a known primitive used to identify those multi-dimensional points offering optimal trade-offs within a given dataset. For large multidimensional dataset, calculating the skyline is extensively compute and data intensive. Although, PIM systems present opportunities to mitigate this cost, their execution model relies on all processors operating in isolation with minimal data exchange. This prohibits direct application of known skyline optimizations which are inherently sequential, creating dependencies and large intermediate results that limit the maximum parallelism, throughput, and require an expensive merging phase. In this work, we address these challenges by introducing the first skyline algorithm for PIM architectures, called DSky. It is designed to be massively parallel and throughput efficient by leveraging a novel work assignment strategy that emphasizes load balancing. Our experiments demonstrate that it outperforms the state-of-the-art algorithms for CPUs and GPUs, in most cases. DSky achieves 2× to 14× higher throughput compared to the state-of-the-art solutions on competing CPU and GPU architectures. Furthermore, we showcase DSky's good scaling properties which are intertwined with PIM's ability to allocate resources with minimal added cost. In addition, we showcase an order of magnitude better energy consumption compared to CPUs and GPUs.

Processing-In-Memory (PIM) is an increasingly popular architecture aimed at addressing the 'memory wall' crisis by prioritizing the integration of processors within DRAM. It promotes low data access latency, high bandwidth, massive parallelism, and low power consumption. The skyline operator is a known primitive used to identify those multi-dimensional points offering optimal trade-offs within a given dataset. For large multidimensional dataset, calculating the skyline is extensively compute and data intensive. Although, PIM systems present opportunities to mitigate this cost, their execution model relies on all processors operating in isolation with minimal data exchange. This prohibits direct application of known skyline optimizations which are inherently sequential, creating dependencies and large intermediate results that limit the maximum parallelism, throughput, and require an expensive merging phase. In this work, we address these challenges by introducing the first skyline algorithm for PIM architectures, called DSky. It is designed to be massively parallel and throughput efficient by leveraging a novel work assignment strategy that emphasizes load balancing. Our experiments demonstrate that it outperforms the state-of-the-art algorithms for CPUs and GPUs, in most cases. DSky achieves 2× to 14× higher throughput compared to the state-of-the-art solutions on competing CPU and GPU architectures. Furthermore, we showcase DSky's good scaling properties which are intertwined with PIM's ability to allocate resources with minimal added cost. In addition, we showcase an order of magnitude better energy consumption compared to CPUs and GPUs.

Though ReRAM has been greatly successful in reducing energy consumption of various neural networks, it still suffers write amplification in energy, which impedes ReRAM to provide efficient storage for the ubiquitous streaming data in CNNs, such as feature-maps. Racetrack memory, an emerging magnetic memory technique, is a proper candidate to hold streaming data since it enjoys fast sequential-access with ultra-low operating energy in read and write. In this work, we propose a hybrid processing-in-memory architecture, called MemUnison, that coordinates ReRAM and racetrack to overcome the expenditure storage of streaming data in ReRAM. By placing feature-maps in racetrack and leaving weights in ReRAM, a datapath is constructed between the two sides to form a fetch-process-writeback pipeline. As the invalid-shifts of the racetrack memory incurs a large amount of pipeline bubble, we propose a row-based access that can read and write a feature-map without any invalid-shifts. For the row-based operation, a cohesive controlling method is proposed to coordinate racetrack and ReRAM. In runtime, convolution kernels are scheduled in ReRAM banks for cross-channel calculations of one row, by which computing complexity of a convolutional layer can be reduced by 4 orders of magnitude, excessing the 2 order of reduction by traditional ReRAM. [ABSTRACT FROM AUTHOR]

This letter proposes a novel capacitor-less dynamic random-access memory composed of a 3D stacked cell array. To perform selective program and erase operations in the 3D cell array, a back-gate-sharing and channel-separating architecture are adopted. The operation of the unit cell is demonstrated through 3D technology computer-aided design simulations. The proposed device uses a simple process to optimize the back gate (e.g., modifying the thickness of materials or using different types of materials). Through optimization of the back gate, a sensing margin and retention time of 52 $\mu \text{A}$ and 475 ms, respectively, are achieved at 300 K in a scaled cell with a gate length of 30 nm and channel radius of 30 nm. Because this device has a cell array similar to that of a 3D NAND flash memory, it can be manufactured using the same process and equipment in a cost-effective manner. In addition, it can be adopted in processing-in-memory applications to provide high-performance and high-density hybrid memory functionality when used in conjunction with NAND flash memories. [ABSTRACT FROM AUTHOR]

Processing-in-memory (PIM) is a promising architecture to design various types of neural network accelerators as it ensures the efficiency of computation together with Resistive Random Access Memory (ReRAM). ReRAM has now become a promising solution to enhance computing efficiency due to its crossbar structure. In this paper, a ReRAM-based PIM neural network accelerator is addressed, and different kinds of methods and designs of various schemes are discussed. Various models and architectures implemented for a neural network accelerator are determined for research trends. Further, the limitations or challenges of ReRAM in a neural network are also addressed in this review. [ABSTRACT FROM AUTHOR]

Triangles are the basic substructure of networks and triangle counting (TC) has been a fundamental graph computing problem in numerous fields such as social network analysis. Nevertheless, like other graph computing problems, due to the high memory-computation ratio and random memory access pattern, TC involves a large amount of data transfers thus suffers from the bandwidth bottleneck in the traditional Von-Neumann architecture. To overcome this challenge, in this paper, we propose to accelerate TC with the emerging processing-in-memory (PIM) architecture through an algorithm-architecture co-optimization manner. To enable the efficient in-memory implementations, we come up to reformulate TC with bitwise logic operations (such as AND), and develop customized graph compression and mapping techniques for efficient data flow management. With the emerging computational Spin-Transfer Torque Magnetic RAM (STT-MRAM) array, which is one of the most promising PIM enabling techniques, the device-to-architecture co-simulation results demonstrate that the proposed TC in-memory accelerator outperforms the state-of-the-art GPU and FPGA accelerations by $12.2\times$ 12. 2 × and $31.8\times$ 31. 8 × , respectively, and achieves a $34\times$ 34 × energy efficiency improvement over the FPGA accelerator. [ABSTRACT FROM AUTHOR]

Hyperdimensional computing (HDC) is an emerging computational framework inspired by the brain that operates on vectors with thousands of dimensions to emulate cognition. Unlike conventional computational frameworks that operate on numbers, HDC, like the brain, uses high-dimensional random vectors and is capable of one-shot learning. HDC is based on a well-defined set of arithmetic operations and is highly error resilient. The core operations of HDC manipulate HD vectors in bulk bit-wise fashion, offering many opportunities to leverage parallelism. Unfortunately, on conventional von Neumann architectures, the continuous movement of HD vectors among the processor and the memory can make the cognition task prohibitively slow and energy intensive. Hardware accelerators only marginally improve related metrics. In contrast, even partial implementations of an HDC framework inside memory can provide considerable performance/energy gains as demonstrated in prior work using memristors. This article presents an architecture based on racetrack memory (RTM) to conduct and accelerate the entire HDC framework within memory. The proposed solution requires minimal additional CMOS circuitry by leveraging a read operation across multiple domains in RTMs called transverse read (TR) to realize exclusive-or (XOR) and addition operations. To minimize the CMOS circuitry overhead, an RTM nanowire-based counting mechanism is proposed. Using language recognition as the example workload, the proposed RTM HDC system reduces the energy consumption by 8.6× compared to the state-of-the-art in-memory implementation. Compared to dedicated hardware design realized with an FPGA, RTM-based HDC processing demonstrates 7.8× and 5.3× improvements in the overall runtime and energy consumption, respectively. [ABSTRACT FROM AUTHOR]

Today's systems rely on sending all the data to the cloud and then using complex algorithms, such as Deep Neural Networks, which require billions of parameters and many hours to train a model. In contrast, the human brain can do much of this learning effortlessly. Hyperdimensional (HD) Computing aims to mimic the behavior of the human brain by utilizing high-dimensional representations. This leads to various desirable properties that other Machine Learning (ML) algorithms lack, such as robustness to noise in the system and simple, highly parallel operations. In this article, we propose HyDREA, a HyperDimensional Computing system that is Robust, Efficient, and Accurate. We propose a Processing-in-Memory (PIM) architecture thatworks in a federated learning environment with challenging communication scenarios that cause errors in the transmitted data. HyDREA adaptively changes the bitwidth of the model based on the signal-to-noise ratio (SNR) of the incoming sample to maintain the accuracy of the HD model while achieving significant speedup and energy efficiency. Our PIM architecture is able to achieve a speedup of 28× and 255× better energy efficiency compared to the baseline PIM architecture for Classification and achieves 32× speed up and 289× higher energy efficiency than the baseline architecture for Clustering. HyDREA is able to achieve this by relaxing hardware parameters to gain energy efficiency and speedup while introducing computational errors. We show experimentally, HD Computing is able to handle the errors without a significant drop in accuracy due to its unique robustness property. For wireless noise, we found that HyDREA is 48× more robust to noise than other comparable ML algorithms. Our results indicate that our proposed system loses less than 1% Classification accuracy, even in scenarios with an SNR of 6.64. We additionally test the robustness of using HD Computing for Clustering applications and found that our proposed system also looses less than 1% in the mutual information score, even in scenarios with an SNR under 7 dB, which is 57× more robust to noise than K-means. [ABSTRACT FROM AUTHOR]

With the rapid development of 5G, artificial intelligence (AI), and high-performance computing (HPC), there is a huge increase in the data exchanged between the processor and memory. However, the "storage wall" caused by the von Neumann architecture severely limits the computational performance of the system. To efficiently process such large amounts of data and break up the "storage wall", it is necessary to develop processing-in-memory (PIM) technology. Chiplet combines processor cores and memory chips with advanced packaging technologies, such as 2.5D, 3 dimensions (3D), and fan-out packaging. This improves the quality and bandwidth of signal transmission and alleviates the "storage wall" problem. This paper reviews the Chiplet packaging technology that has achieved the function of PIM in recent years and analyzes some of its application results. First, the research status and development direction of PIM are presented and summarized. Second, the Chiplet packaging technologies that can realize the function of PIM are introduced, which are divided into 2.5D, 3D packaging, and fan-out packaging according to their physical form. Further, the form and characteristics of their implementation of PIM are summarized. Finally, this paper is concluded, and the future development of Chiplet in the field of PIM is discussed. [ABSTRACT FROM AUTHOR]

In Analog Computing-in-Memory (CIM) neural network accelerators, analog-to-digital converters (ADCs) are required to convert the analog partial sums generated from a CIM array to digital values. The overhead from ADCs substantially degrades the energy efficiency of CIM accelerators so that previousworks attempted to lower the ADC resolution considering the distribution of the partial sums. Despite the efforts, the required ADC resolution still remains relatively high. In this article, we propose the data-driven partial sum quantization scheme, which exhaustively searches for the optimal quantization range with little computational burden. We also report that analyzing the characteristics of the partial sum distributions at each layer gives an additional information to further reduce the ADC resolution compared to previous works that mostly used the characteristics of the partial sum distributions of the entire network. Based on the finer-level data-driven approach combined with retraining, we present a methodology for extreme partial-sum quantization. Experimental results show that the proposed method can reduce the ADC resolution to 2 to 3 bits for CIFAR-10 dataset, which is the smaller ADC bit resolution than any previous CIM-based NN accelerators. [ABSTRACT FROM AUTHOR]

Processing-in-memory (PIM) architectures have demonstrated great potential in accelerating numerous deep learning tasks. Particularly, resistive random-access memory (RRAM) devices provide a promising hardware substrate to build PIM accelerators due to their abilities to realize efficient in-situ vector-matrix multiplications (VMMs). However, existing PIM accelerators suffer from frequent and energy-intensive analog-to-digital (A/D) conversions, severely limiting their performance. This paper presents a new PIM architecture to efficiently accelerate deep learning tasks by minimizing the required A/D conversions with analog accumulation and neural approximated peripheral circuits. We first characterize the different dataflows employed by existing PIM accelerators, based on which a new dataflow is proposed to remarkably reduce the required A/D conversions for VMMs by extending shift and add (S+A) operations into the analog domain before the final quantizations. We then leverage a neural approximation method to design both analog accumulation circuits (S+A) and quantization circuits (ADCs) with RRAM crossbar arrays in a highly-efficient manner. Finally, we apply them to build a RRAM-based PIM accelerator (i.e., Neural-PIM) upon the proposed analog dataflow and evaluate its system-level performance. Evaluations on different benchmarks demonstrate that Neural-PIM can improve energy efficiency by $5.36\times$ 5. 36 × ($1.73\times$ 1. 73 × ) and speed up throughput by $3.43\times$ 3. 43 × ($1.59\times$ 1. 59 × ) without losing accuracy, compared to the state-of-the-art RRAM-based PIM accelerators, i.e., ISAAC [1] (CASCADE [2]). [ABSTRACT FROM AUTHOR]

The recent advances in Artificial Intelligence (AI) achieving "better-than-human" accuracy in a variety of tasks such as image classification and the game of Go have come at the cost of exponential increase in the size of artificial neural networks. This has lead to AI hardware solutions becoming severely memory-bound and scrambling to keep-up with the ever increasing "von Neumann bottleneck". Processing-in-Memory (PiM) architectures offer an excellent solution to ease the von Neumann bottleneck by embedding compute capabilities inside the memory and reducing the data traffic between the memory and the processor. But PiM accelerators break the standard von Neumann programming model by fusing memory and compute operations together which impedes their integration in the standard computing stack. There is an urgent requirement for system-level solutions to take full advantage of PiM accelerators for end-to-end acceleration of AI applications. This article presents AI-PiM as a solution to bridge this research gap. AIPiM proposes a hardware, ISA and software co-design methodology which allows integration of PiM accelerators in the RISC-V processor pipeline as functional execution units. AI-PiM also extends the RISC-V ISA with custom instructions which directly target the PiM functional units resulting in their tight integration with the processor. This tight integration is especially important for edge AI devices which need to process both AI and non-AI tasks on the same hardware due to area, power, size and cost constraints. AI-PiM ISA extensions expose the PiM hardware functionality to software programmers allowing efficient mapping of applications to the PiM hardware. AI-PiM adds support for custom ISA extensions to the complete software stack including compiler, assembler, linker, simulator and profiler to ensure programmability and evaluation with popular AI domain-specific languages and frameworks like TensorFlow, PyTorch, MXNet, Keras etc. AI-PiM improves the performance for vector-matrix multiplication (VMM) kernel by 17.63x and provides a mean speed-up of 2.74x for MLPerf Tiny benchmark compared to RV64IMC RISC-V baseline. AI-PiM also speeds-up MLPerf Tiny benchmark inference cycles by 2.45x (average) compared to state-of-the-art Arm Cortex-A72 processor. [ABSTRACT FROM AUTHOR]

Neural networks have been widely used for advanced tasks from image recognition to natural language processing. Many recent works focus on improving the efficiency of executing neural networks in diverse applications. Researchers have advocated processing‐in‐memory (PIM) architecture as a promising candidate for training and testing neural networks because PIM design can reduce the communication cost between storage and computing units. However, there exist noises in the PIM system generated from the intrinsic physical properties of both memory devices and the peripheral circuits. The noises introduce challenges in stably training the systems and achieving high test performance, e.g., accuracy in classification tasks. This review discusses the current approaches to tolerating noise effects for both training and inference in PIM systems and provides an analysis from a hardware–software codesign perspective. Noise‐tolerant strategies for PIM systems based on resistive random‐access memory (ReRAM), including circuit‐level, algorithm‐level, and system‐level solutions are explained. In addition, we also present some selected noise‐tolerate cases in PIM systems for generative adversarial networks and physical neural networks. [ABSTRACT FROM AUTHOR]

In-memory computing (IMC) is recognized as one of the most promising architecture solution to realize energy-efficient neural network inference. Amongst many memory technology, resistive RAM (RRAM) is a very attractive device to implement the IMC-based neural network accelerator architecture, which is particularly suitable for power-constrained IoT systems. Due to the nature of low leakage and in-situ computing, the dynamic power consumption of dot-production operations in RRAM crossbars dominates the chip power, especially when applied to low-precision neural networks. This work investigates the correlation between the cell resistance state and the crossbar operation power, and proposes a state-aware RRAM accelerator (SARA) architecture for energy-efficient low-precision neural networks. With the proposed state-aware network training and mapping strategy, crossbars in the RRAM accelerator can perform in a lower power state. Furthermore, we also leverage the proposed RRAM accelerator architecture to reduce the power consumption of high-precision network inference with both single-level or multilevel RRAM. The evaluation results show that for binary neural networks, our design saves 40.53% RRAM computing energy on average over the baseline. For high precision neural networks, the proposed method reduces 11.67% computing energy on average without any accuracy loss. [ABSTRACT FROM AUTHOR]

Processing-in-memory (PIM) provides promising solutions to the main memory bottleneck by placing computational logic in or near memory devices to reduce data movement overheads. Recent work explored how commercial DRAM can feature digital PIM logic while meeting fab-level energy and area constraints, and showed a significant speedup in the inference time of data-intensive deep learning models. However, convolutional neural network (CNN) models were not considered as main targets for the commercial DRAM-PIM due to their compute-intensive convolution layers. Moreover, recent studies revealed that the area and power constraints on memory die prevent DRAM-PIM from competing with GPUs and specialized accelerators in accelerating them. Recently, mobile CNN models have increasingly adopted a composition of depthwise and pointwise convolutions instead of such compute-intensive convolutions to reduce computation cost without accuracy drop. In this paper, we show that 1x1 convolution can be offloaded for PIM acceleration with integrated runtime support and without any hardware or algorithm changes. We provide further speedup with parallel execution on GPU and DRAM-PIM and code generation optimizations. Our solution achieves up to 35.2% (31.6% on average) speedup for all 1x1 convolutions for mobile CNN models against GPU. [ABSTRACT FROM AUTHOR]

Neural networks have been widely used for advanced tasks from image recognition to natural language processing. Many recent works focus on improving the efficiency of executing neural networks in diverse applications. Researchers have advocated processing‐in‐memory (PIM) architecture as a promising candidate for training and testing neural networks because PIM design can reduce the communication cost between storage and computing units. However, there exist noises in the PIM system generated from the intrinsic physical properties of both memory devices and the peripheral circuits. The noises introduce challenges in stably training the systems and achieving high test performance, e.g., accuracy in classification tasks. This review discusses the current approaches to tolerating noise effects for both training and inference in PIM systems and provides an analysis from a hardware–software codesign perspective. Noise‐tolerant strategies for PIM systems based on resistive random‐access memory (ReRAM), including circuit‐level, algorithm‐level, and system‐level solutions are explained. In addition, we also present some selected noise‐tolerate cases in PIM systems for generative adversarial networks and physical neural networks.

The recent advances in Artificial Intelligence (AI) achieving “better-than-human” accuracy in a variety of tasks such as image classification and the game of Go have come at the cost of exponential increase in the size of artificial neural networks. This has lead to AI hardware solutions becoming severely memory-bound and scrambling to keep-up with the ever increasing “von Neumann bottleneck”. Processing-in-Memory (PiM) architectures offer an excellent solution to ease the von Neumann bottleneck by embedding compute capabilities inside the memory and reducing the data traffic between the memory and the processor. But PiM accelerators break the standard von Neumann programming model by fusing memory and compute operations together which impedes their integration in the standard computing stack. There is an urgent requirement for system-level solutions to take full advantage of PiM accelerators for end-to-end acceleration of AI applications. This article presents AI-PiM as a solution to bridge this research gap. AI-PiM proposes a hardware, ISA and software co-design methodology which allows integration of PiM accelerators in the RISC-V processor pipeline as functional execution units. AI-PiM also extends the RISC-V ISA with custom instructions which directly target the PiM functional units resulting in their tight integration with the processor. This tight integration is especially important for edge AI devices which need to process both AI and non-AI tasks on the same hardware due to area, power, size and cost constraints. AI-PiM ISA extensions expose the PiM hardware functionality to software programmers allowing efficient mapping of applications to the PiM hardware. AI-PiM adds support for custom ISA extensions to the complete software stack including compiler, assembler, linker, simulator and profiler to ensure programmability and evaluation with popular AI domain-specific languages and frameworks like TensorFlow, PyTorch, MXNet, Keras etc. AI-PiM improves the performance for vector-matrix multiplication (VMM) kernel by 17.63x and provides a mean speed-up of 2.74x for MLPerf Tiny benchmark compared to RV64IMC RISC-V baseline. AI-PiM also speeds-up MLPerf Tiny benchmark inference cycles by 2.45x (average) compared to state-of-the-art Arm Cortex-A72 processor.

The processing in-memory (PIM) approach that combines memory and processor appears to solve the memory wall problem. NAND flash memory, which is widely adopted in edge devices, is one of the promising platforms for PIM with its high-density property and the intrinsic ability for analog vector-matrix multiplication. Despite its potential, the domain conversion process, which converts an analog current to a digital value, accounts for most energy consumption on the NAND flash-based accelerator. It restricts the NAND flash memory usage for PIM compared to the other platforms. In this article, we propose a NAND flash-based DNN accelerator to achieve both large memory density and energy efficiency among various platforms. As the NAND flash memory already shows higher memory density than other memory platforms, we aim to enhance energy efficiency by reducing the domain conversion process burden. First, we optimize the bit width of partial multiplication by considering the analog-to-digital converter (ADC) resource. For further optimization, we propose a methodology to exploit many zero partial multiplication results for enhancing both energy efficiency and throughput. The proposed work successfully exploits the bit-level sparsity of DNN, which results in achieving up to 8.6×/8.2× larger energy efficiency/throughput over the provisioned baseline. [ABSTRACT FROM AUTHOR]

Due to the need for computing models that can process large quantities of data efficiently and with high throughput in many state-of-the-art machine learning algorithms, the processing-in-memory (PIM) paradigm is emerging as a potential replacement for standard digital architectures on these workloads. In this tutorial, we review the progress of PIM technology in recent years, at both the circuit and architecture level. We further present an analysis of when and how PIM technology surpasses the performance of conventional architectures. Finally, we outline our vision for the future of PIM technology. [ABSTRACT FROM AUTHOR]