Back to Search
Start Over
Equivalent-accuracy accelerated neural-network training using analogue memory
- Source :
- Nature. June, 2018, Vol. 558 Issue 7708, p60, 8 p.
- Publication Year :
- 2018
-
Abstract
- Neural-network training can be slow and energy intensive, owing to the need to transfer the weight data for the network between conventional digital memory chips and processor chips. Analogue non-volatile memory can accelerate the neural-network training algorithm known as backpropagation by performing parallelized multiply-accumulate operations in the analogue domain at the location of the weight data. However, the classification accuracies of such in situ training using non-volatile-memory hardware have generally been less than those of software-based training, owing to insufficient dynamic range and excessive weight-update asymmetry. Here we demonstrate mixed hardware-software neural-network implementations that involve up to 204,900 synapses and that combine long-term storage in phase-change memory, near-linear updates of volatile capacitors and weight-data transfer with 'polarity inversion' to cancel out inherent device-to-device variations. We achieve generalization accuracies (on previously unseen data) equivalent to those of software-based training on various commonly used machine-learning test datasets (MNIST, MNIST-backrand, CIFAR-10 and CIFAR-100). The computational energy efficiency of 28,065 billion operations per second per watt and throughput per area of 3.6 trillion operations per second per square millimetre that we calculate for our implementation exceed those of today's graphical processing units by two orders of magnitude. This work provides a path towards hardware accelerators that are both fast and energy efficient, particularly on fully connected neural-network layers.Analogue-memory-based neural-network training using non-volatile-memory hardware augmented by circuit simulations achieves the same accuracy as software-based training but with much improved energy efficiency and speed.<br />Author(s): Stefano Ambrogio [sup.1] , Pritish Narayanan [sup.1] , Hsinyu Tsai [sup.1] , Robert M. Shelby [sup.1] , Irem Boybat [sup.2] [sup.3] , Carmelo di Nolfo [sup.1] [sup.3] , Severin [...]
- Subjects :
- Artificial neural networks -- Usage
Machine learning -- Technology application
Algorithms
Data processing services
Computer memory
Energy efficiency
Energy management
Energy conservation
Capacitors
Neural network
Technology application
Environmental issues
Science and technology
Zoology and wildlife conservation
Subjects
Details
- Language :
- English
- ISSN :
- 00280836
- Volume :
- 558
- Issue :
- 7708
- Database :
- Gale General OneFile
- Journal :
- Nature
- Publication Type :
- Academic Journal
- Accession number :
- edsgcl.572639550
- Full Text :
- https://doi.org/10.1038/s41586-018-0180-5