Search

This work demonstrates a programmable in-memory-computing (IMC) inference accelerator for scalable execution of neural network (NN) models, leveraging a high-signal-to-noise ratio (SNR) capacitor-based analog technology. IMC accelerates computations and reduces memory accessing for matrix-vector multiplies (MVMs), which dominate in NNs. The accelerator architecture focuses on scalable execution, addressing the overheads of state swapping and the challenges of maintaining high utilization across highly dense and parallel hardware. The architecture is based on a configurable on-chip network (OCN) and scalable array of cores, which integrate mixed-signal IMC with programmable near-memory single-instruction multiple-data (SIMD) digital computing, configurable buffering, and programmable control. The cores enable flexible NN execution mappings that exploit data- and pipeline-parallelism to address utilization and efficiency across models. A prototype is presented, incorporating a $4 \times 4$ array of cores demonstrated in 16 nm CMOS, achieving peak multiply-accumulate (MAC)-level throughput of 3 TOPS and peak MAC-level energy efficiency of 30 TOPS/W, both for 8-b operations. The measured results shows high accuracy of the analog computations, matching bit-true simulations. This enables the abstractions required for robust and scalable architectural and software integration. Developed software libraries and NN-mapping tools are used to demonstrate CIFAR-10 and ImageNet classification, with an 11-layer CNN and ResNet-50, respectively, achieving accuracy, throughput, and energy efficiency of 91.51% and 73.33%, 7815 and 581 image/s, 51.5 k and 3.0 k image/s/W, with 4-b weights and activations. [ABSTRACT FROM AUTHOR]

In-memory computing (IMC) addresses the cost of accessing data from memory in a manner that introduces a tradeoff between energy/throughput and computation signal-to-noise ratio (SNR). However, low SNR posed a primary restriction to integrating IMC in larger, heterogeneous architectures required for practical workloads due to the challenges with creating robust abstractions necessary for the hardware and software stack. This work exploits recent progress in high-SNR IMC to achieve a programmable heterogeneous microprocessor architecture implemented in 65-nm CMOS and corresponding interfaces to the software that enables mapping of application workloads. The architecture consists of a 590-Kb IMC accelerator, configurable digital near-memory-computing (NMC) accelerator, RISC-V CPU, and other peripherals. To enable programmability, microarchitectural design of the IMC accelerator provides the integration in the standard processor memory space, area- and energy-efficient analog-to-digital conversion for interfacing to NMC, bit-scalable computation (1–8 b), and input-vector sparsity-proportional energy consumption. The IMC accelerator demonstrates excellent matching between computed outputs and idealized software-modeled outputs, at 1b TOPS/W of 192|400 and 1b-TOPS/mm2 of 0.60|0.24 for MAC hardware, at VDD of 1.2|0.85 V, both of which scale directly with the bit precision of the input vector and matrix elements. Software libraries developed for application mapping are used to demonstrate CIFAR-10 image classification with a ten-layer CNN, achieving accuracy, throughput, and energy of 89.3%|92.4%, 176|23 images/s, and 5.31|105.2 μJ/image, for 1|4 b quantization levels. [ABSTRACT FROM AUTHOR]

Large-scale matrix-vector multiplications, which dominate in deep neural networks (DNNs), are limited by data movement in modern VLSI technologies. This paper addresses data movement via an in-memory-computing accelerator that employs charged-domain mixed-signal operation for enhancing compute SNR and, thus, scalability. The architecture supports analog/binary input activation (IA)/weight first layer (FL) and binary/binary IA/weight hidden layers (HLs), with batch normalization and input–output (IO) (buffering) circuitry to enable cascading, if desired, for realizing different DNN layers. The architecture is arranged as $8\times 8=64$ in-memory-computing neuron tiles, supporting up to 512, $3\times 3\times 512$ -input HL neurons and 64, $3\times 3\times 3$ -input FL neurons, configurable via tile-level clock gating. In-memory computing is achieved using an 8T bit cell with overlaying metal-oxide-metal (MOM) capacitor, yielding a structure having $1.8\times $ the area of a standard 6T bit cell. Implemented in 65-nm CMOS, the design achieves HLs/FL energy efficiency of 866/1.25 TOPS/W and throughput of 18876/43.2 GOPS (1498/3.43 GOPS/mm2), when implementing convolution layers; and 658/0.95 TOPS/W, 9438/10.47 GOPS (749/0.83 GOPS/mm2), when implementing convolution followed by batch normalization layers. Several large-scale neural networks are demonstrated, showing performance on standard benchmarks (MNIST, CIFAR-10, and SVHN) equivalent to ideal digital computing. [ABSTRACT FROM AUTHOR]

This paper presents a 3-D integrated circuit (3-D IC) for heterogeneous domain-specific streaming architectures. In such architectures, an array of fine-grained accelerators is provided for executing kernels, and applications are mapped via configuration of the accelerators into a desired computation pipeline. The two-layer 3-D IC addresses architectures for different application domains, through a generic routing-and-memory (RM) layer and a separate compute-accelerator (CA) layer, which could ultimately be selected at assembly time for different application domains. The RM layer provides a configurable routing network, as well as memory for pipeline buffering and computation scratch pad. The routing network is based on a 2-D mesh with low-swing signaling. The memory is organized as 32 fine-grained (1-kB) SRAM tiles for increased interface parallelism, reduced access energy, and modularity, to interface with different accelerators in the CA layer. Memory-driver and sensing circuits are reused by the low-swing routing network, both for repeaters and to directly load pipeline data into accelerator input buffers. For the prototype, the CA layer is implemented as an array of multiplexers, providing off-chip interfacing to any memory title, thereby enabling different accelerators to be emulated by an off-chip field-programmable gate array (FPGA). The 3-D interconnection is achieved by 8- $\mu \text{m}$ -pitch face-to-face (F2F) vias and wafer-level assembly. For the $2.47\times 3.38$ mm2 two-layer die, implemented in 130-nm CMOS, the total peak memory bandwidth is 9.2 GB/s/mm2. A compute pipeline for computational photography is demonstrated, with the total energy of the accelerators reduced by over 2 $\times $ , by exploiting parallelism enabled by interfaces to fine-grained RM-layer memory. [ABSTRACT FROM AUTHOR]

In-memory computing is an emerging approach for overcoming memory-accessing bottlenecks, by eliminating the costs of explicitly moving data from point of storage to point of computation outside the array. However, computation increases the dynamic range of signals, such that performing it via the existing structure of dense memory substantially squeezes the signal-to-noise ratio (SNR). In this brief, we explore how computations can be scaled up, to jointly optimize energy/latency/bandwidth gains with SNR requirements. We employ algorithmic techniques to decompose computations so that they can be mapped to multiple parallel memory banks operating at chosen optimal points. Specifically focusing on in-memory classification, we consider a custom IC in 130-nm CMOS IC and demonstrate an algorithm combining error-adaptive classifier boosting and multi-armed bandits, to enable segmentation of a feature vector into multiple subsets. The measured performance of 10-way MNIST digit classification, using images downsampled to $16{\times }16$ pixels (mapped across four separate banks), is 91%, close to that simulated using full unsegmented feature vectors. The energy per classification is 879.7 pJ, $14.3{\times }$ lower than that of a system based on separated memory and digital accelerator. [ABSTRACT FROM AUTHOR]

Hybrid systems combine large-area electronics (LAE) with silicon-CMOS ICs for sensing and computation, respectively. In such systems, interfacing a large number of distributed LAE sensors with the CMOS domain poses a key limitation. This paper presents an architecture that aims to greatly reduce both the number of physical connections and the time for accessing all of the sensors. Each sensor modulates the amplitude of a thin-film transistor (TFT) digitally controlled oscillator (DCO). All DCO outputs are combined, but each follows a unique frequency-hopping pattern (controlled by a code from CMOS), allowing recovery of the individual sensors. The architecture enables much greater scalability of sensors for a given number of connections than active-matrix and binary-addressing schemes. For demonstration, an 18-element large-area force-sensing system is demonstrated based on zinc-oxide (ZnO) TFT DCOs with a frequency-hopping rate of 4.2 kHz. Acquisition error $\leq $ 62 mVrms is achieved over 30 weight patterns. [ABSTRACT FROM AUTHOR]

This paper presents a machine-learning classifier where computations are performed in a standard 6T SRAM array, which stores the machine-learning model. Peripheral circuits implement mixed-signal weak classifiers via columns of the SRAM, and a training algorithm enables a strong classifier through boosting and also overcomes circuit nonidealities, by combining multiple columns. A prototype 128 $\times $ 128 SRAM array, implemented in a 130-nm CMOS process, demonstrates ten-way classification of MNIST images (using image-pixel features downsampled from 28 $\times $ 28 = 784 to 9 $\times $ 9 = 81, which yields a baseline accuracy of 90%). In SRAM mode (bit-cell read/write), the prototype operates up to 300 MHz, and in classify mode, it operates at 50 MHz, generating a classification every cycle. With accuracy equivalent to a discrete SRAM/digital-MAC system, the system achieves ten-way classification at an energy of 630 pJ per decision, 113 times lower than a discrete system with standard training algorithm and 13 times lower than a discrete system with the proposed training algorithm. [ABSTRACT FROM AUTHOR]

By enabling diverse and large-scale transducers, large-area electronics raises the potential for electronic systems to interact much more extensively with the physical world than is possible today. This can substantially expand the scope of applications, both in number and in value. But first, translation into applications requires a base of system functions (instrumentation, computation, power management, communication). These cannot be realized on the desired scale by large-area electronics alone. It is necessary to combine large-area electronics with high-performance, high-efficiency technologies, such as crystalline silicon CMOS, within hybrid systems. Scalable hybrid systems require rethinking the subsystem architectures from the start by considering how the technologies should be interfaced, on both a functional and physical level. To explore platform architectures along with the supporting circuits and devices, we consider as an application driver, a self-powered sheet for high-resolution structural health monitoring (of bridges and buildings). Top-down evaluation of design alternatives within the hybrid design space and pursuit of template architectures exposes circuit functions and device optimizations traditionally overlooked by bottom-up approaches alone. [ABSTRACT FROM PUBLISHER]

Computational requirements in highly energy constrained applications are driving the need for ultra-low-power processors. In such devices SRAMs pose a primary energy limitation. This paper analyzes SRAM energy in practical applications using state-of-the-art power-management techniques. The design targets and array biasing for energy minimization are developed. Compared with generic logic, these are characterized by the important difference that SRAMs generally need to retain data. This restricts the use of power-gating for leakage elimination, and thus this paper considers the application of low-leakage data-retention biasing during the idle-mode. The resulting energy tradeoffs have important distinctions, and these are analyzed in the presence of practical variation levels. [ABSTRACT FROM AUTHOR]

This tutorial paper examines architectural and circuit design techniques for a microsensor node operating at power levels low enough to enable the use of an energy harvesting source. These requirements place demands on all levels of the design. We propose an architecture for achieving the required ultra-low energy operation and discuss the circuit techniques necessary to implement the system. Dedicated hardware implementations improve the efficiency for specific functionality, and modular partitioning permits fine-grained optimization and power-gating. We describe modeling and operating at the minimum energy point in the subthreshold region for digital circuits. We also examine approaches for improving the energy efficiency of analog components like the transmitter and the ADC. A microsensor node using the techniques we describe can function in an energy-harvesting scenario. [ABSTRACT FROM AUTHOR]