NoM : Network-on-Memory for Inter-bank Data Transfer in Highly-banked Memories

Data copy is a widely-used memory operation in many programs and operating system services. In conventional computers, data copy is often carried out by two separate read and write transactions that pass data back and forth between the memory hierarchy and processor registers. Some prior mechanisms propose to avoid this unnecessary data movement by using the shared internal bus in DRAM chip to directly copy data between two DRAM banks. While these methods exhibit superior performance, compared to conventional techniques, this technique does not allow data copy over different DRAM channels. Hence, this technique has limited benefit for the emerging 3D stacked memories (such as HMC and HBM) that contains tens of banks across multiple memory controllers. In this paper, we present Network-on-Memory (NoM), a lightweight inter-bank communication scheme that enables direct data copy within memory. NoM adopts a TDM-based circuit-switching design, where circuit setup is done by the memory controller. Compared to previous state-of-the-art approaches, NoM enables both data copy over multiple DRAM channels and concurrent copy operation. Our evaluation shows that NoM improves the performance of data-intensive workloads by 3.8X on average compare to the state-of-the-art techniques, respectively. IEEE
QC 20200716