Search

Reduced precision computation is a key enabling factor for energy-efficient acceleration of deep learning (DL) applications. This article presents a 7-nm four-core mixed-precision artificial intelligence (AI) chip that supports four compute precisions--FP16, Hybrid-FP8 (HFP8), INT4, and INT2--to support diverse application demands for training and inference. The chip leverages cutting-edge algorithmic advances to demonstrate leading-edge power efficiency for 8-bit floating-point (FP8) training and INT4 inference without model accuracy degradation. A new HFP8 format combined with separation of the floating- and fixed-point pipelines and aggressive circuit/architecture optimization enables performance improvements while maintaining high compute utilization. A high-bandwidth ring protocol enables efficient data communication, while power management using workload-aware clock throttling maximizes performance within a given power budget. The AI chip demonstrates 3.58-TFLOPS/W peak energy efficiency and 26.2-TFLOPS peak performance for HFP8 iso-accuracy training, and 16.9-TOPS/W peak energy efficiency and 104.9-TOPS peak performance for INT4 iso-accuracy inference.

Advances in deep neural networks (DNNs) and the availability of massive real-world data have enabled superhuman levels of accuracy on many AI tasks and ushered the explosive growth of AI workloads across the spectrum of computing devices. However, their superior accuracy comes at a high computational cost, which necessitates approaches beyond traditional computing paradigms to improve their operational efficiency. Leveraging the application-level insight of error resilience, we demonstrate how approximate computing (AxC) can significantly boost the efficiency of AI platforms and play a pivotal role in the broader adoption of AI-based applications and services. To this end, we present RaPiD, a multi-tera operations per second (TOPS) AI hardware accelerator core (fabricated at 14-nm technology) that we built from the ground-up using AxC techniques across the stack including algorithms, architecture, programmability, and hardware. We highlight the workload-guided systematic explorations of AxC techniques for AI, including custom number representations, quantization/pruning methodologies, mixed-precision architecture design, instruction sets, and compiler technologies with quality programmability, employed in the RaPiD accelerator.

This survey paper summarizes recent progress of deep learning circuits and systems technologies and contains four topics: hardware-centric deep learning algorithms, digital architectures, analog architectures, and system demonstrations. We present an overview of these four areas and introduce key contributions of papers in this special issue.

This letter presents a multi-TOPS AI accelerator core for deep learning training and inference. With a programmable architecture and custom ISA, this engine achieves >90% sustained utilization across the range of neural network topologies by employing a dataflow architecture to provide high throughput and an on-chip scratchpad hierarchy to meet the bandwidth demands of the compute units. A custom 16b floating point (fp16) representation with 1 sign bit, 6 exponent bits, and 9 mantissa bits has also been developed for high model accuracy in training and inference as well as 1b/2b (binary/ternary) integer for aggressive inference performance. At 1.5 GHz, the AI core prototype achieves 1.5 TFLOPS fp16, 12 TOPS ternary, or 24 TOPS binary peak performance in 14-nm CMOS.

This paper investigates degradation behavior induced by the self-heating effect for InGaZnO (IGZO) thin-film transistors (TFTs). Both the surrounding oxide and other thermal insulating materials, as well as the low thermal conductivity of the InGaZnO layer itself, cause the self-heating effect in InGaZnO TFTs. The heated channel layer enhances the threshold voltage shift, and the evolution of threshold voltage shift is found to be dominated by charge-trapping effects. Moreover, a nonuniform distribution of channel carrier concentration leads to an uneven temperature distribution throughout the IGZO active layer, which results in the asymmetrical degradation behavior after the self-heating operation. Further verifications indicate that the degree of the threshold voltage shift is only dependent on stress power, regardless of stress Vg, Vd, and channel length. Further, two-stage dependence of the threshold voltage shift on dynamic stress frequency is found.

The plasma-process-induced damage (PPID) of low-temperature polycrystalline silicon (poly-Si) thin-film transistors (TFTs) during the etching of the poly-Si film was investigated in this paper. The results reveal the relationship between the device degradation and the PPID during TFT liquid-crystal-display fabrication. This degradation is caused in part by the damage at the edge of the poly-Si film in plasma exposure. The trapped-state densities Ntrap are measured to clarify the relationship between instability and plasma etching damages. The plasma-process condition substantially affects the PPID of the poly-Si etching process. The main mechanism is the generation of charge trapping states at the poly-Si grain boundary in the damaged edge of the TFT channel active region. The electrical recovery from the plasma damage is also studied with various postetching treatments. Hydrogen-base plasma treatment and laser anneal process are revealed to improve the device characteristics due to reduction of charge trapping states.

Advanced harmonic balance simulation capabilities have been used to investigate linearity behavior in RF power laterally diffused metal-oxide semiconductor (LDMOS) devices with different laterally graded channel doping profiles. Harmonic balance analysis provides relevant device information from device-level simulations and yields performance metrics of RF circuits. The linearity behavior of simple quasi-1-D structures with different graded channel doping profiles was investigated; the analysis was then extended to a more realistic power LDMOS device. The third-order intermodulation distortion product reveals the important role of lateral channel doping. The analysis lays groundwork for device optimization for improved linearity.

Device-level simulation capabilities have been developed to investigate low-frequency noise behavior in p-type Si0.7Ge0.3/Si heterostructure MOS (SiGe p-HMOS) transistors. The numerical model is based on the impedance field method; it accounts for a trap-induced carrier number fluctuation, a layer-dependent correlated mobility fluctuation, and a Hooge mobility fluctuation in the buried and parasitic surface channels, respectively. Simulations based on such models have been conducted for SiGe p-HMOS transistors, and the results have been carefully correlated with experimental data. Quantitative agreement has been obtained in terms of the noise level dependence on gate biases, drain currents, and body biases, revealing the important role of the dual channels in the low-frequency noise behavior of SiGe p-HMOS devices.