1. A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platforms
- Author
-
Silvano, Cristina, Ielmini, Daniele, Ferrandi, Fabrizio, Fiorin, Leandro, Curzel, Serena, Benini, Luca, Conti, Francesco, Garofalo, Angelo, Zambelli, Cristian, Calore, Enrico, Schifano, Sebastiano Fabio, Palesi, Maurizio, Ascia, Giuseppe, Patti, Davide, Petra, Nicola, De Caro, Davide, Lavagno, Luciano, Urso, Teodoro, Cardellini, Valeria, Cardarilli, Gian Carlo, Birke, Robert, and Perri, Stefania
- Subjects
Computer Science - Hardware Architecture ,Computer Science - Emerging Technologies ,Computer Science - Machine Learning - Abstract
Recent trends in deep learning (DL) imposed hardware accelerators as the most viable solution for several classes of high-performance computing (HPC) applications such as image classification, computer vision, and speech recognition. This survey summarizes and classifies the most recent advances in designing DL accelerators suitable to reach the performance requirements of HPC applications. In particular, it highlights the most advanced approaches to support deep learning accelerations including not only GPU and TPU-based accelerators but also design-specific hardware accelerators such as FPGA-based and ASIC-based accelerators, Neural Processing Units, open hardware RISC-V-based accelerators and co-processors. The survey also describes accelerators based on emerging memory technologies and computing paradigms, such as 3D-stacked Processor-In-Memory, non-volatile memories (mainly, Resistive RAM and Phase Change Memories) to implement in-memory computing, Neuromorphic Processing Units, and accelerators based on Multi-Chip Modules. Among emerging technologies, we also include some insights into quantum-based accelerators and photonics. To conclude, the survey classifies the most influential architectures and technologies proposed in the last years, with the purpose of offering the reader a comprehensive perspective in the rapidly evolving field of deep learning., Comment: Preprint version of our manuscript submitted to the journal @ ACM CSUR (58 pages including Appendix) on June 22nd, 2023. Major revision submitted on July 12th, 2024
- Published
- 2023