1. HeAT – a Distributed and GPU-accelerated Tensor Framework for Data Analytics
- Author
-
Björn Hagemeier, Claudia Comito, Kai Krajsek, Achim Streit, Simon Hanselmann, Daniel Coquelin, Martin Siggel, Achim Basermann, Philipp Knechtges, Michael Tarnawa, Charlotte Debus, and Markus Götz
- Subjects
FOS: Computer and information sciences ,Data Analysis ,0301 basic medicine ,Computer Science - Machine Learning ,Computer science ,Big data ,GPU ,G.1.3 ,02 engineering and technology ,Parallel computing ,Dask ,computer.software_genre ,Machine Learning (cs.LG) ,Machine Learning ,NumPy ,C.2.4 ,0202 electrical engineering, electronic engineering, information engineering ,Parallel Application Frameworks ,computer.programming_language ,I.5.5 ,Message Passing Interface ,High-performance Computing ,Computer Science - Distributed, Parallel, and Cluster Computing ,Parallel processing (DSP implementation) ,Data analysis ,Tensor Framework ,Distributed memory ,G.4 ,Model Parallelism ,Neural Networks ,03 medical and health sciences ,C.1.2 ,Big Data Analytics ,High-performanceComputing ,020204 information systems ,D.1.3 ,I.2.5 ,business.industry ,Node (networking) ,DATA processing & computer science ,I.2.0 ,HeAT ,Software framework ,030104 developmental biology ,PyTorch ,Computer Science - Mathematical Software ,Distributed, Parallel, and Cluster Computing (cs.DC) ,ddc:004 ,business ,Mathematical Software (cs.MS) ,computer - Abstract
To cope with the rapid growth in available data, the efficiency of data analysis and machine learning libraries has recently received increased attention. Although great advancements have been made in traditional array-based computations, most are limited by the resources available on a single computation node. Consequently, novel approaches must be made to exploit distributed resources, e.g. distributed memory architectures. To this end, we introduce HeAT, an array-based numerical programming framework for large-scale parallel processing with an easy-to-use NumPy-like API. HeAT utilizes PyTorch as a node-local eager execution engine and distributes the workload on arbitrarily large high-performance computing systems via MPI. It provides both low-level array computations, as well as assorted higher-level algorithms. With HeAT, it is possible for a NumPy user to take full advantage of their available resources, significantly lowering the barrier to distributed data analysis. When compared to similar frameworks, HeAT achieves speedups of up to two orders of magnitude., 10 pages, 8 figures, 5 listings, 1 table
- Published
- 2020