Search

Significant growth is seen in artificial intelligence (AI), in particular deep learning (DL), which has made remarkable progress in various areas such as computer vision, natural language processing, health care, autonomous driving, and surveillance. To accomplish this, AI technologies have broadened from a centralized fashion to mobile or distributed fashion, opening a new era called edge AI, with dramatic advancements that are substantially changing everyday technology, social behavior, and lifestyles. Edge AI couples intelligence and analysis to a broad collection of connected devices and systems for data collection, caching, and processing. It enables a wide variety of new promising applications where data collection and analysis are combined. Billions of mobile users are exploiting various smartphone applications such as translation services, digital assistants, and health monitoring services.

Despite the great success of High-Level Synthesis (HLS) tools, we observe several unresolved challenges: 1) the high-level abstraction of programming styles in HLS conceals optimization opportunities; 2) existing HLS tools do not provide flexible trade-offs among different objectives and constraints; 3) the actual quality of the resulting RTL designs is hard to predict. To this end, we propose an end-to-end framework, IRONMAN. The primary goal is to enable a flexible and automated design space exploration (DSE), which can provide either optimized solutions under user-specified constraints, or Pareto trade-offs among different objectives (e.g., resource types, area, and latency). IronMan consists of three components: GPP (a highly accurate graph-neural-network-based performance predictor), RLMD (a reinforcement-learning-based DSE engine that explores the optimized resource allocation strategy), and CT (a code transformer that assists RLMD and GPP by extracting data flow graphs from original HLS C/C++). Experimental results show that, 1) GPP achieves high prediction accuracy, reducing prediction errors of HLS tools by 10.9X in resource usage and 5.7X in timing; 2) RLMD obtains optimized or Pareto solutions outperforming genetic algorithm and simulated annealing by 12.7% and 12.9%, respectively; 3) IronMan can find optimized solutions perfectly matching various DSP constraints, with 2.54X fewer DSPs and up to 6X shorter latency than those of HLS tools. IronMan is also up to 400X faster than meta-heuristic techniques and HLS tools.

The deep neural network (DNN) based AI applications on the edge require both low-cost computing platforms and high-quality services. However, the limited memory, computing resources, and power budget of the edge devices constrain the effectiveness of the DNN algorithms. Developing edge-oriented AI algorithms and implementations (e.g., accelerators) is challenging. In this paper, we summarize our recent efforts for efficient on-device AI development from three aspects, including both training and inference. First, we present on-device training with ultra-low memory usage. We propose a novel rank-adaptive tensor-based tensorized neural network model, which offers orders-of-magnitude memory reduction during training. Second, we introduce an ultra-low bitwidth quantization method for DNN model compression, achieving the state-of-the-art accuracy under the same compression ratio. Third, we introduce an ultra-low latency DNN accelerator design, practicing the software/hardware co-design methodology. This paper emphasizes the importance and efficacy of training, quantization and accelerator design, and calls for more research breakthroughs in the area for AI on the edge.

High quality AI solutions require joint optimization of AI algorithms, such as deep neural networks (DNNs), and their hardware accelerators. To improve the overall solution quality as well as to boost the design productivity, efficient algorithm and accelerator co-design methodologies are indispensable. In this paper, we first discuss the motivations and challenges for the Algorithm/Accelerator co-design problem and then provide several effective solutions. Especially, we highlight three leading works of effective co-design methodologies: 1) the first simultaneous DNN/FPGA co-design method; 2) a bi-directional lightweight DNN and accelerator co-design method; 3) a differentiable and efficient DNN and accelerator co-search method. We demonstrate the effectiveness of the proposed co-design approaches using extensive experiments on both FPGAs and GPUs, with comparisons to existing works. This paper emphasizes the importance and efficacy of algorithm-accelerator co-design and calls for more research breakthroughs in this interesting and demanding area., GLSVLSI, September 7-9, 2020

Along with higher and higher integration of intellectual properties (IPs) on a single chip, traditional bus-based system-on-chips (SoC) meets several design difficulties (such as low scalability, high power consumption, packet latency and clock tree problem). As a promising solution, network-on-chips (NoC) has been proposed and widely studied. In this work, a novel algorithm for NoC topology synthesis, which is decomposing and cluster refinement (DCR) algorithm, has been proposed to minimize the total power consumption of application-specific NoC. This algorithm is composed of two stages: decomposing with cluster generation, and cluster refinement. For partitioning and cluster generation, an initial low-power solution for NoC topology is generated. For cluster refinement, the clustering is optimized by performing floorplan to further reduce power consumption. Meanwhile, a good tradeoff between power consumption and CPU time can be achieved. Experimental results show that the proposed method outperforms the existing work.

In this paper, we focus on the low-power scheduling problem with multiple threshold and/or supply voltage technologies in high-level synthesis. We propose a unified scheduling approach which is applicable to various optimization problems, including: 1) dynamic power and resource usage co-optimization; 2) leakage power optimization; and 3) dynamic power and leakage power co-optimization. To deal with different objectives with high flexibility, three problems are divided into two common subproblems including delay assignment and resource density variance minimization, then a vertex potential-based mobility allocation model is proposed to solve two subproblems simultaneously. Experimental results show that, for dynamic power and resource co-optimization, our scheduling approach produces optimum solutions for all six benchmarks with 15 groups of data; for leakage power optimization it also greatly excels the latest existing work, by 20% leakage power reduction and 52 times speedup. Besides, for dynamic and leakage power co-optimization, the Pareto solutions are studied.

Data path interconnection on VLSI chips usually consumes a significant amount of both power and area. In this paper, we focus on the port assignment problem for binary commutative operators for interconnection complexity reduction. First, the port assignment problem is formulated on a constraint graph, and a practical method is proposed to find a valid and initial solution. For solution optimization, an elementary spanning-tree-transformation-based local search algorithm is proposed. To improve the efficiency of optimization, a matrix formulation, which meets the simplex tabuleau format, is proposed and thus the simplex method is adopted for optimization. Moreover, operation pivoting and successive pivoting are discussed for algorithm speedup. The experimental results show that on the randomly generated test cases, the matrix-based algorithm shows the highest solution optimality and is five times faster than the elementary transformation method. On the real high-level synthesis benchmarks, the matrix-based method reduced 14% interconnections, while the previous greedy algorithm reduced 8% on average.

Quantization has been proven to be an effective method for reducing the computing and/or storage cost of DNNs. However, the trade-off between the quantization bitwidth and final accuracy is complex and non-convex, which makes it difficult to be optimized directly. Minimizing direct quantization loss (DQL) of the coefficient data is an effective local optimization method, but previous works often neglect the accurate control of the DQL, resulting in a higher loss of the final DNN model accuracy. In this paper, we propose a novel metric called Vector Loss. Based on this new metric, we develop a new quantization solution called VecQ, which can guarantee minimal direct quantization loss and better model accuracy. In addition, in order to speed up the proposed quantization process during model training, we accelerate the quantization process with a parameterized probability estimation method and template-based derivation calculation. We evaluate our proposed algorithm on MNIST, CIFAR, ImageNet, IMDB movie review and THUCNews text data sets with numerical DNN models. The results demonstrate that our proposed quantization solution is more accurate and effective than the state-of-the-art approaches yet with more flexible bitwidth support. Moreover, the evaluation of our quantized models on Saliency Object Detection (SOD) tasks maintains comparable feature extraction quality with up to 16$\times$ weight size reduction., Comment: 14 pages, 9 figures, Journal

Recent breakthroughs in Deep Neural Networks (DNNs) have fueled a growing demand for DNN chips. However, designing DNN chips is non-trivial because: (1) mainstream DNNs have millions of parameters and operations; (2) the large design space due to the numerous design choices of dataflows, processing elements, memory hierarchy, etc.; and (3) an algorithm/hardware co-design is needed to allow the same DNN functionality to have a different decomposition, which would require different hardware IPs to meet the application specifications. Therefore, DNN chips take a long time to design and require cross-disciplinary experts. To enable fast and effective DNN chip design, we propose AutoDNNchip - a DNN chip generator that can automatically generate both FPGA- and ASIC-based DNN chip implementation given DNNs from machine learning frameworks (e.g., PyTorch) for a designated application and dataset. Specifically, AutoDNNchip consists of two integrated enablers: (1) a Chip Predictor, built on top of a graph-based accelerator representation, which can accurately and efficiently predict a DNN accelerator's energy, throughput, and area based on the DNN model parameters, hardware configuration, technology-based IPs, and platform constraints; and (2) a Chip Builder, which can automatically explore the design space of DNN chips (including IP selection, block configuration, resource balancing, etc.), optimize chip design via the Chip Predictor, and then generate optimized synthesizable RTL to achieve the target design metrics. Experimental results show that our Chip Predictor's predicted performance differs from real-measured ones by < 10% when validated using 15 DNN models and 4 platforms (edge-FPGA/TPU/GPU and ASIC). Furthermore, accelerators generated by our AutoDNNchip can achieve better (up to 3.86X improvement) performance than that of expert-crafted state-of-the-art accelerators., Accepted by 28th ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA'2020)

Deep Neural Networks (DNNs) have become promising solutions for data analysis especially for raw data processing from sensors. However, using DNN-based approaches can easily introduce huge demands of computation and memory consumption, which may not be feasible for direct deployment onto the Internet of Thing (IoT) devices, since they have strict constraints on hardware resources, power budgets, response latency, and manufacturing cost. To bring DNNs into IoT devices, embedded FPGA can be one of the most suitable candidates by providing better energy efficiency than GPU and CPU based solutions, and higher flexibility than ASICs. In this paper, we propose a systematic solution to deploy DNNs on embedded FPGAs, which includes a ternarized hardware Deep Learning Accelerator (T-DLA), and a framework for ternary neural network (TNN) training. T-DLA is a highly optimized hardware unit in FPGA specializing in accelerating the TNNs, while the proposed framework can significantly compress the DNN parameters down to two bits with little accuracy drop. Results show that our training framework can compress the DNN up to 14.14x while maintaining nearly the same accuracy compared to the floating point version. By illustrating our proposed design techniques, the T-DLA can deliver up to 0.4TOPS with 2.576W power consumption, showing 873.6x and 5.1x higher energy efficiency (fps/W) on ImageNet with Resnet-18 model comparing to Xeon E5-2630 CPU and Nvidia 1080 Ti GPU. To the best of our knowledge, this is the first instruction-based highly efficient ternary DLA design reported from the literature.

The efficacy and effectiveness of Convolutional Neural Networks (CNNs) have been proven in a wide range of machine learning applications. However, the high computational complexity of CNNs presents a critical challenge towards their broader adoption in real-time and power-efficient scenarios. FPGAs are poised to take a significant role for high-performance and energy-efficient computation of CNNs for both mobile (e.g., UAVs, self-driving cars, and IoT devices) and cloud computing domains. However, implementing an effective CNN system onto FPGAs efficiently remains problematic. The current cloud-based FPGAs with unique design constraints and architectural characteristics further increase the challenges. To address these challenges, we propose a novel open-source automated tool chain called Cloud-DNN. Our tool chain takes trained CNN models specified in Caffe as input, performs a set of transformations, and maps the model to a cloud-based FPGA. Cloud-DNN can significantly improve the overall design productivity of CNNs on FPGAs while satisfying the emergent computational requirements. Our design provides an alternative solution compared to other cloud-based options (e.g., GPUs or TPUs) while offering flexible, and high performance DNN inferences. The unique features of Cloud-DNN include the optimizations with cloud-platform characteristics and the support of easier and streamlined implementation. Experimental results demonstrate up to 104.55x performance improvement when compared to CPU implementation and comparable usability, flexibility, and strong quality compared to other state-of-the-art DNN inference implementations on standalone FPGAs.

The exponential increase in leakage power and the substantial power-saving opportunities provided by scheduling have made dual-threshold voltage (dual- $V_{\mathrm{ th}}$ ) an attractive choice for low-leakage-power designs. In this paper, we work under the assumption that functional units (FUs) are allocated after scheduling, and fully explore the solution space of scheduling with dual- $V_{\mathrm{ th}}$ operations to optimize the leakage power of the FUs. First, a binding conflict graph (BCG)-based scheduling method is presented to minimize the number of FUs. Second, the BCG-based method is extended to allow scheduling with dual- $V_{\mathrm{ th}}$ operation targeting the minimization of leakage power. In timing-constrained scheduling, each operation in the data flow is initialized with low- $V_{\mathrm{ th}}$ . Then, starting from an operation schedule with the timing constraint satisfied, we scale the sets of low- $V_{\mathrm{ th}}$ operations in the off-critical paths with high- $V_{\mathrm{ th}}$ so as to reduce the number of low- $V_{\mathrm{ th}}$ FUs without increasing the total delay. Finally, a scheduling method for minimizing the leakage power under both timing and resource constraints is presented. The results of benchmark tests show that the proposed algorithms can reduce the leakage power reported in previous works by 10.2% while maintaining high circuit performance.

The Visual Object Tracking challenge VOT2018 is the sixth annual tracker benchmarking activity organized by the VOT initiative. Results of over eighty trackers are presented; many are state-of-the-art trackers published at major computer vision conferences or in journals in the recent years. The evaluation included the standard VOT and other popular methodologies for short-term tracking analysis and a “real-time” experiment simulating a situation where a tracker processes images as if provided by a continuously running sensor. A long-term tracking subchallenge has been introduced to the set of standard VOT sub-challenges. The new subchallenge focuses on long-term tracking properties, namely coping with target disappearance and reappearance. A new dataset has been compiled and a performance evaluation methodology that focuses on long-term tracking capabilities has been adopted. The VOT toolkit has been updated to support both standard short-term and the new long-term tracking subchallenges. Performance of the tested trackers typically by far exceeds standard baselines. The source code for most of the trackers is publicly available from the VOT page. The dataset, the evaluation kit and the results are publicly available at the challenge website (http://votchallenge.net). Funding agencies: Slovenian research agencySlovenian Research Agency - Slovenia [P2-0214, P2-0094, J2-8175]; Czech Science FoundationGrant Agency of the Czech Republic [GACR P103/12/G084]; WASP; VR (EMC2); SSF (SymbiCloud); SNIC; AIT Strategic Research Programme 2017 Visua

With an explosive growth of various neural network algorithms, their high performance implementations on hardware platforms, such as GPUs and FPGAs, are becoming critical as well. Compared to widely used GPUs, FPGAs are considered to be harder for design and optimization even with the help of High Level Synthesis (HLS) tools. However, recent studies have shown that FPGAs can outperform GPUs in speed and power/energy efficiency; both factors are important in machine learning applications. In this paper, we will discuss a simultaneous DNN and hardware accelerator co-design method to push the DNN performance on FPGAs. We first summarize existing techniques and results along this direction, and then propose new ideas to further improve DNN development productivity and design quality. Finally we discuss the challenges we would face and propose some potential solutions.

Triangle counting and truss decomposition are two essential procedures in graph analysis. As the scale of graphs grows larger, designing highly efficient graph analysis systems with less power demand becomes more and more urgent. In this paper, we present triangle counting and truss decomposition using a Field-Programmable Gate Array (FPGA). We leverage the flexibility of FPGAs and achieve low-latency high-efficiency implementations. Evaluation on SNAP dataset shows that our triangle counting and truss decomposition implementations achieve 43.5× on average (up to 757.7×) and 6.4× on average (up to 68.0×) higher performance per Watt respectively over GPU solutions.

With the rapid development of electricity market reform and renewable energy in China, the trading system of green certificate based on Renewable Portfolio Standard (RPS) will be established in China. An equilibrium model of multi-periods electricity market considering green certificate trading is proposed in this paper, in which GenCos' (Generation Companies') strategic bidding behaviors are properly modeled using Cournot model and the transmission limit constraints, the renewable energy ratio constraints and the unit ramp rate constraints are simultaneously considered. Moreover, the nodal price is also decomposed into five components, including the unit marginal cost, the green certificate marginal cost and so on. As illustrated by the numerical tests, in order to reduce the cost of the system, the output of renewable energy unit in peak load periods is often greater than the required green certificate quota at the same time, and vice versa in valley load periods. In imperfectly competitive environment, the GenCo owning thermal units and renewable energy units may manipulate the nodal price and green certificate price simultaneously to earn higher profit.

The study on machine learning has been flourishing for several years, and machine learning algorithms are being applied to various fields with great achievements. In this paper, combining the on-line machine learning method into optimization algorithms is to be studied. In many heuristic optimization algorithms, one common way to reduce execution time and improve solution optimality is, first estimating the quality of a set of candidate solutions, and solving only promising candidates in detail. Currently most estimations are performed by empirical equations, whose accuracy greatly relies on the how well the equation is designed. In this paper, we propose an on-line learning based estimator to perform the solution estimation in heuristic algorithms to improve estimation accuracy. Then a simple case study is discussed, where a local search based heuristic with random start is used, and an on-line estimator considering the properties of local search is proposed. The experiments show that the accuracy of on-line estimator is much higher than the static estimator, and is also higher than a general off-line pre-trained learner. Even though the on-line estimator introduced special time for its training, the heuristic algorithm still speeds up by 3.7X without optimality sacrifice.

In 3D integrated circuit (3D-IC), there are two or more layers of active electronic components which are integrated both vertically and horizontally. Through-silicon-via (TSV) is used as the vertical electrical connection which enables a great deal of functionality packed into a small footprint. In this work we solve the signal TSV assignment problem in 3D-IC taking thermal problem into consideration, while only wire length is concerned in the previous work. Firstly, we propose a multilevel node-weight-oriented flow assignment algorithm to reduce wire length and temperature simultaneously. During the evaluation of wire length and temperature result, we use a thermal estimation model to evaluate temperature, where compression storage method with LU decomposition is adopted to improve algorithm efficiency. Moreover, we implement the remove and reassign optimization method which helps further optimize the TSV assignment results. The experimental results show that our algorithm provides better solution with wire length reduction and temperature decrease.

Due to the rise of utilization wall, a large portion of silicon chips become dark or dim silicon. A NoC-sprinting method is proposed to deal with this problem for instantaneous improvement and the key design constraint of these problems is thermal design power(TDP). In this work we propose a thermal-aware Modified Insert After Remove Floorplanning(MD-IARFP) algorithm for NoC-sprinting. Wire length is taken into consideration while only thermal behavior is concerned in the previous work. A thermal model is constructed to evaluate temperature, using relationship between heat transfer and electrical phenomena. Simulated Annealing(SA) based MD-IARFP algorithm is applied to optimize the distribution of active cores. In terms of perturbation of SA, Modified Insert After Remove(MD-IAR) method gives an efficient generation of new floorplan which helps to reduce iterate number and lead to less CPU time. Effective as the experimental results show, our proposal provides better solution with lower temperature and significant decrease of wire length.

Network-on-Chip (NoC) is a promising solution for System-on-Chip (SoC) challenges. In this work, we present a Decompose and Cluster generation Refinement (DCR) algorithm to find minimum power consumption simultaneously. A two-stage method is proposed for decompose and cluster generation step to generate solutions with lower power. Refinement step explores optimal positions and adjusts clusters for selected solutions to find balanced point between power consumption and CPU time. Experimental results show that the proposed method outperforms the existing work.

In this work we solve the 3D-IC TSV assignment which is formulated as an Integer Minimum Cost Multi Commodity (IMCMC) problem. We propose a multi-level algorithm including graph coarsening and un-coarsening, to greatly reduce the IMCMC problem size; benefiting from it, we transform the IMCMC problem into a mixed single and multi commodity problem and partially solved it using min-cost flow method. Further, we extend the layer by layer method for TSV assignment optimization. The experimental results shows that the our proposal generates initial solutions with 2.8% wire length reduction compared to the existing work, and our extended layer by layer method achieves another 1.7% improvement with slight time overhead.