1. The Pangeo Ecosystem: Interactive Computing Tools for the Geosciences: Benchmarking on HPC
- Author
-
Tina Odaka, Jared Baker, Guillaume Eynard-Bontemps, Guillaume Maze, Ryan Abernathey, Anderson Banihirwe, Aurélien Ponte, and Kevin Paul
- Subjects
Interactive computing ,Scheme (programming language) ,Computer science ,business.industry ,Xarray ,Distributed computing ,Cloud computing ,02 engineering and technology ,Benchmarking ,interactive computing ,Dask ,Software ,020204 information systems ,Scalability ,HPC ,0202 electrical engineering, electronic engineering, information engineering ,Data_FILES ,Pangeo ,cloud ,020201 artificial intelligence & image processing ,benchmarking ,business ,computer ,Chunking (computing) ,computer.programming_language - Abstract
The Pangeo ecosystem is an interactive computing software stack for HPC and public cloud infrastructures. In this paper, we show benchmarking results of the Pangeo platform on two di erent HPC sys- tems. Four di erent geoscience operations were considered in this bench- marking study with varying chunk sizes and chunking schemes. Both strong and weak scaling analyses were performed. Chunk sizes between 64MB to 512MB were considered, with the best scalability obtained for 512MB. Compared to certain manual chunking schemes, the auto chunk- ing scheme scaled well.
- Published
- 2019