Back to Search Start Over

Supercomputing with MPI meets the Common Workflow Language standards: An experience report

Authors :
Nash, Rupert W.
Crusoe, Michael R.
Kontak, Max
Brown, Nick
Nash, Rupert W.
Crusoe, Michael R.
Kontak, Max
Brown, Nick
Source :
Vrije Universiteit Amsterdam Repository
Publication Year :
2021

Abstract

Use of standards-based workflows is still somewhat unusual by high-performance computing users. In this paper we describe the experience of using the Common Workflow Language (CWL) standards to describe the execution, in parallel, of MPI-parallelised applications. In particular, we motivate and describe the simple extension to the specification which was required, as well as our implementation of this within the CWL reference runner. We discuss some of the unexpected benefits, such as simple use of HPC-oriented performance measurement tools, and CWL software requirements interfacing with HPC module systems. We close with a request for comment from the community on how these features could be adopted within versions of the CWL standards.

Details

Database :
OAIster
Journal :
Vrije Universiteit Amsterdam Repository
Notes :
2020 IEEE/ACM Workflows in Support of Large-Scale Science (WORKS): Proceedings, p.17-24. Institute of Electrical and Electronics Engineers Inc.., English
Publication Type :
Electronic Resource
Accession number :
edsoai.on1311081032
Document Type :
Electronic Resource
Full Text :
https://doi.org/10.1109.WORKS51914.2020.00008