The Pangeo Ecosystem: Interactive Computing Tools for the Geosciences: Benchmarking on HPC

Type Proceedings paper
Date 2019
Language English
Other localization https://sc19.supercomputing.org/
Author(s) Odaka TinaORCID1, Banihirwe Anderson2, Eynard-Bontemps Guillaume3, Ponte AurelienORCID1, Maze GuillaumeORCID1, Paul Kevin2, Baker Jared2, Abernathey Ryan4
Affiliation(s) 1 : Laboratory for Ocean Physics and Satellite Remote Sensing UMR LOPS, Ifremer, Univ. Brest, CNRS, IRD, IUEM Brest, France
2 : National Center for Atmospheric Research, Boulder, CO, USA
3 : CNES Computing Center team Centre National d'Etudes Spatiales Toulouse, France
4 : Lamont Doherty Earth Observatory, Columbia University, New York, USA
Meeting SC ’19 workshops - The International Conference for High Performance Computing, Networking, Storage, and Analysis. 17-22 Novembrer 2019, Denver
Source Juckeland G., Chandrasekaran S. (eds) Tools and Techniques for High Performance Computing. HUST 2019, SE-HER 2019, WIHPC 2019. Communications in Computer and Information Science, vol 1190. Springer, Cham. Print ISBN 978-3-030-44727-4 Online ISBN 978-3-030-44728-1. https://doi.org/10.1007/978-3-030-44728-1_12. pp.190-204 .
DOI 10.1007/978-3-030-44728-1_12
Keyword(s) Pangeo, interactive computing, HPC, cloud, benchmarking, Dask, Xarray
Abstract

The Pangeo ecosystem is an interactive computing software stack for HPC and public cloud infrastructures. In this paper, we show benchmarking results of the Pangeo platform on two di erent HPC sys- tems. Four di erent geoscience operations were considered in this bench- marking study with varying chunk sizes and chunking schemes. Both strong and weak scaling analyses were performed. Chunk sizes between 64MB to 512MB were considered, with the best scalability obtained for 512MB. Compared to certain manual chunking schemes, the auto chunk- ing scheme scaled well.
Full Text
File Pages Size Access
Author's final draft 14 8 MB Open access
24 21 MB Access on demand
15 307 KB Access on demand
Top of the page

How to cite 

Odaka Tina, Banihirwe Anderson, Eynard-Bontemps Guillaume, Ponte Aurelien, Maze Guillaume, Paul Kevin, Baker Jared, Abernathey Ryan (2019). The Pangeo Ecosystem: Interactive Computing Tools for the Geosciences: Benchmarking on HPC. Juckeland G., Chandrasekaran S. (eds) Tools and Techniques for High Performance Computing. HUST 2019, SE-HER 2019, WIHPC 2019. Communications in Computer and Information Science, vol 1190. Springer, Cham. Print ISBN 978-3-030-44727-4 Online ISBN 978-3-030-44728-1. https://doi.org/10.1007/978-3-030-44728-1_12. pp.190-204 . https://archimer.ifremer.fr/doc/00597/70946/