Search

In this article, a novel interconnect technology is presented for the cost-effective and flexible design of asynchronous networks-on-chip. It delivers asynchrony in heterogeneous system integration while yielding low-energy on-chip data movement. The approach consists of both a lightweight asynchronous switch architecture (using transition-signaling protocols and bundled-data encoding) and a complete synthesis flow built on top of mainstream industrial CAD tools. For the first time, this article demonstrates compelling area, performance and power benefits when compared to a recent commercial synchronous switch, and the ability of the tool flow to correctly instantiate a complete and competitive network topology.

We introduce the “Formula-1” (F-1) roofline model to understand the role of computing in aerial autonomous machines. The model provides insights by exploiting the fundamental relationships between various components in an aerial robot, such as sensor framerate, compute performance, and body dynamics (physics). F-1 serves as a tool that can aid computer and cyber-physical system architects to understand the optimal design (or selection) of various components in the development of autonomous machines.

The modern system-on-chip (SoC) of the current exascale computing era is complex. These SoCs not only consist of several general-purpose processing cores but also integrate many specialized hardware accelerators. Three common coherency interfaces are used to integrate the accelerators with the memory hierarchy: non-coherent,coherent with the last-level cache (LLC), and fully-coherent.However, using a single coherence interface for all the accelerators in an SoC can lead to significant overheads: in the non-coherent model, accelerators directly access the main memory, which can have considerable performance penalty; whereas in the LLC-coherent model, the accelerators access the LLC but may suffer from performance bottleneck due to contention between several accelerators; and the fully-coherent model, that relies on private caches, can incur non-trivial power/area overheads. Given the limitations of each of these interfaces, this paper proposes a novel performance-aware hybrid coherency interface, where different accelerators use different coherency models, decided at design time based on the target applications so as to optimize the overall system performance. A new Bayesian optimization based framework is also proposed to determine the optimal hybrid coherency interface, i.e., use machine learning to select the best coherency model for each of the accelerators in the SoC in terms of performance. For image processing and classification workloads, the proposed framework determined that a hybrid interface achieves up to 23 percent better performance compared to the other ’homogeneous‘ interfaces, where all the accelerators use a single coherency model.

Modern systems-on-chip (SoCs) include not only general-purpose CPUs but also specialized hardware accelerators. Typically, there are three coherence model choices to integrate an accelerator with the memory hierarchy: no coherence, coherent with the last-level cache (LLC), and private cache based full coherence. However, there has been very limited research on finding which coherence models are optimal for the accelerators of a complex many-accelerator SoC. This paper focuses on determining a cost-aware coherence interface for an SoC and its target application: find the best coherence models for the accelerators that optimize their power and performance, considering both workload characteristics and system-level contention. A novel comprehensive methodology is proposed that uses Bayesian optimization to efficiently find the cost-aware coherence interfaces for SoCs that are modeled using the gem5-Aladdin architectural simulator. For a complete analysis, gem5-Aladdin is extended to support LLC coherence in addition to already-supported no coherence and full coherence. For a heterogeneous SoC targeting applications with varying amount of accelerator-level parallelism, the proposed framework rapidly finds cost-aware coherence interfaces that show significant performance and power benefits over the other commonly-used coherence interfaces.

In recent years, there has been tremendous advances in hardware acceleration of deep neural networks. However, most of the research has focused on optimizing accelerator microarchitecture for higher performance and energy efficiency on a per-layer basis. We find that for overall single-batch inference latency, the accelerator may only make up 25-40%, with the rest spent on data movement and in the deep learning software framework. Thus far, it has been very difficult to study end-to-end DNN performance during early stage design (before RTL is available) because there are no existing DNN frameworks that support end-to-end simulation with easy custom hardware accelerator integration. To address this gap in research infrastructure, we present SMAUG, the first DNN framework that is purpose-built for simulation of end-to-end deep learning applications. SMAUG offers researchers a wide range of capabilities for evaluating DNN workloads, from diverse network topologies to easy accelerator modeling and SoC integration. To demonstrate the power and value of SMAUG, we present case studies that show how we can optimize overall performance and energy efficiency for up to 1.8-5x speedup over a baseline system, without changing any part of the accelerator microarchitecture, as well as show how SMAUG can tune an SoC for a camera-powered deep learning pipeline., 14 pages, 20 figures

Multicast communication (1-to-many) is common in parallel architectures and emerging areas such as neuromorphic computing. However, there is very limited research in supporting multicast in asynchronous NoCs. This paper proposes a new parallel multicast asynchronous NoC with 2D-mesh topology. To the best of our knowledge, this is the first general-purpose asynchronous NoC to support multicast in 2D meshes. A critical feature of this NoC is the use of a new continuous-time replication strategy, where the flits of a multicast packet are routed through the distinct outputs of the router according to each output's own rate, in parallel and in continuous time. This unique asynchronous continuous-time replication, not discretized to clock cycles, can handle subtle variations in network congestion and exploit "sub-cycle" differentials in operating speeds. A new continuous-time multi-way read (CMR) buffer is proposed to enable this replication strategy. Only a single CMR buffer is used per input port, with multiple independent read pointers, which are accessed by different outputs. For diverse multicast benchmarks, the new parallel multicast network achieved significant latency and throughput gains over a serial baseline. Moderate energy overhead was seen for one benchmark with a small multicast portion, but major reductions were achieved for higher amounts of multicast. Interestingly, consistent latency improvements were observed for unicast, in spite of the extra instrumentation. Experiments on isolated multicast packet transmissions also showed over an order-of-magnitude improvement in delivery time.

We propose a lightweight parallel multicast targeting an asynchronous NoC with a variant Mesh-of-Trees topology. A novel strategy, local speculation, is introduced, where a subset of switches are speculative and always broadcast. These switches are surrounded by non-speculative switches, which throttle any redundant packets, restricting these packets to small regions. Speculative switches have simplified designs, thereby improving network performance. A hybrid network architecture is proposed to mix the speculative and non-speculative switches. For multicast benchmarks, significant performance improvements with small power savings are obtained by the new approach over a tree-based non-speculative approach. Interestingly, similar improvements are also shown for unicast. Finally, another benefit is to reduce the address field size in multicast packets.