1. Verification, Validation and Uncertainty Quantification of Large-Scale Applications with QCG-PilotJob
- Author
-
Erwan Raffin, Jalal Lakhlili, Bartosz Bosak, Tomasz Piontek, Paul Karlshoefer, and Piotr Kopta
- Subjects
Focus (computing) ,business.industry ,Computer science ,media_common.quotation_subject ,Distributed computing ,Scale (chemistry) ,Interdependence ,Software ,Workflow ,Computer cluster ,Scalability ,Uncertainty quantification ,business ,media_common - Abstract
Efficient execution of large-scale and extremely demanding computational scenarios is a challenge for both the infrastructure providers and end-users, usually scientists, that need to develop highly scalable computational codes. Nevertheless, at this time, on the eve of exa-scale supercomputers, the particular role has to be given also to the intermediate software that can help in the preparation of applications so they can be efficiently executed on the emerging HPC systems. The efficiency and scalability of such software can be seen as priorities, however, these are not the only elements that should be addressed. Equally important is to offer software that is elastic, portable between platforms of different sizes, and easy to use. Trying to fulfill all the above needs we present QCG-PilotJob, a tool designed to enable flexible execution of numerous potentially dynamic and interdependent computing tasks in a single allocation on a computing cluster. QCG-PilotJob is built on many years of collaboration with computational scientists representing various domains and it responses to the practical requirements of real scientific use-cases. In this paper, we focus on the recent integration of QCG-PilotJob with the EasyVVUQ library and its successful use for Uncertainty Quantification workflows of several complex multiscale applications being developed within the VECMA project. However, we believe that with a well-thought-out design that allows for fully user-space execution and straightforward installation, QCG-PilotJob may be easily exploited in many other application scenarios, even by inexperienced users.
- Published
- 2021