Search

Transformer models are difficult to employ in mobile devices due to their memory- and computation-intensive properties. Accordingly, there is ongoing research on various methods for compressing transformer models, such as pruning and quantization. However, general computing platforms, such as central processing units (CPUs) and graphics processing units (GPUs), are not energy-efficient to accelerate the pruned model because the unstructured sparsity they exhibit causes degradation of parallelism. In this article, we propose a low-power accelerator for transformers that can handle various levels of structured sparsity induced by line pruning with different granularity. Our approach accelerates pruned transformers in a head-wise and line-wise manner. We present a head reorganization and shuffling method that supports head-wise skip operations and resolves the load imbalance problem among processing engines (PEs) caused by the varying number of operations in each head. Furthermore, we implemented a sparse quantized general matrix-to-matrix multiplication (SQ-GEMM) module that supports line-wise skipping and on-the-fly tile-based dynamic quantization of activations. As a result, compared to mobile GPU and CPU, the proposed accelerator improved the energy efficiency by $2.9\times $ and $12.3\times $ for the detection transformer (DETR), and $3.0\times $ and $12.4\times $ for the vision transformer (ViT) models, respectively. In addition, our proposed mobile accelerator achieved the highest-energy efficiency among the current state-of-the-art FPGA-based transformer accelerators.

There is a need for electronic design automation (EDA) tools for analog circuit design since analog circuit design requires substantial human effort and expertise. Using reinforcement learning (RL)-inspired methodologies, this study presents MA-Opt, an analog circuit optimizer. We propose MA-Opt to provide multiple predictions of optimized circuit designs through the use of multiple actors. Multiple actors can be exploited effectively by sharing a memory that affects the loss function of network training, resulting in an accelerated optimization of circuits. Furthermore, we introduce a cooperative near-sampling method deploying a synergistic effect and then optimizing the design. The efficiency of MA-Opt was demonstrated by simulating three analog circuits and comparing the results to other methods. In the experiment, the use of multiple actors with a shared elite solution set and the cooperative near-sampling method proved to be effective. MA-Opt achieved minimum target metrics up to 34% better than DNN-Opt within the same number of simulations while satisfying all given constraints. Moreover, at identical runtime, MA-Opt exhibited better Figure of Merits (FoMs) in comparison to DNN-Opt.

Many design optimization methods using machine learning (ML) techniques have been investigated to reduce the number of design iterations in the physical design flow. The demand for big data to support ML research has been increasing, but the lack of place-and-route (P&R) benchmarks is one of the major problems. We propose a framework to construct realistic P&R benchmarks for use in training ML applications. The framework can organize the P&R database using an artificial netlist generator, which can create any gate-level netlist from user-specified input parameters that represent the topological characteristics of the circuit. We show that a training dataset that contains many artificial gate-level netlists can improve the generalizability of the model to predict the routability for unseen real circuits without using expensive real-world data. Compared to the model that had been trained with real-world circuits, we improved the F1 score in predicting the timing and routing failure by 26.4% and 54.5%, respectively.