40 results on '"Bartosz Bosak"'
Search Results
2. The impact of uncertainty on predictions of the CovidSim epidemiological code.
- Author
-
Wouter Edeling, Hamid Arabnejad, Robert Sinclair, Diana Suleimenova, Krishnakumar Gopalakrishnan, Bartosz Bosak, Derek Groen, Imran Mahmood, Daan Crommelin, and Peter V. Coveney
- Published
- 2021
- Full Text
- View/download PDF
3. Introducing VECMAtk - Verification, Validation and Uncertainty Quantification for Multiscale and HPC Simulations.
- Author
-
Derek Groen, Robin A. Richardson, David W. Wright 0001, Vytautas Jancauskas, Robert Sinclair, Paul Karlshoefer, Maxime Vassaux, Hamid Arabnejad, Tomasz Piontek, Piotr Kopta, Bartosz Bosak, Jalal Lakhlili, Olivier Hoenen, Diana Suleimenova, Wouter Edeling, Daan Crommelin, Anna Nikishova, and Peter V. Coveney
- Published
- 2019
- Full Text
- View/download PDF
4. Patterns for High Performance Multiscale Computing.
- Author
-
Saad Alowayyed, Tomasz Piontek, James L. Suter, Olivier Hoenen, Derek Groen, Onnie Luk, Bartosz Bosak, Piotr Kopta, Krzysztof Kurowski, Oliver Perks, Keeran Brabazon, Vytautas Jancauskas, David Coster, Peter V. Coveney, and Alfons G. Hoekstra
- Published
- 2019
- Full Text
- View/download PDF
5. Towards Making Fusion Data FAIR.
- Author
-
David Coster, Shaun De Witt, Iraklis A. Klampanos, Marcin Plóciennik, Andreas Ikonomopoulos, Frederic Imbeaux, Nathan Cummings, Michal Owsiak, Agata Filipczak, Bartosz Bosak, Stasinos Konstantopoulos, and Pär Strand
- Published
- 2021
- Full Text
- View/download PDF
6. VECMAtk: A Scalable Verification, Validation and Uncertainty Quantification Toolkit for Scientific Simulations.
- Author
-
Derek Groen, Hamid Arabnejad, Vytautas Jancauskas, Wouter Edeling, Fredrik Jansson, Robin A. Richardson, Jalal Lakhlili, Linda van Veen, Bartosz Bosak, Piotr Kopta, D. W. Wright, Nicolas Monnier, Paul Karlshoefer, Diana Suleimenova, Robert Sinclair, Maxime Vassaux, Anna Nikishova, Mariusz Bieniek, Onnie O. Luk, Michal Kulczewski, Erwan Raffin, Daan Crommelin, Olivier Hoenen, David P. Coster, Tomasz Piontek, and Peter V. Coveney
- Published
- 2020
7. Localization of network service performance degradation in multi-tenant networks.
- Author
-
Pavle V. Vuletic, Bartosz Bosak, Marinos Dimolianis, Pascal Mérindol, David Schmitz, and Henrik Wessing
- Published
- 2020
- Full Text
- View/download PDF
8. MultiGrain/MAPPER: A distributed multiscale computing approach to modeling and simulating gene regulation networks.
- Author
-
Alexandru E. Mizeranschi, Martin T. Swain, Raluca Scona, Quentin Fazilleau, Bartosz Bosak, Tomasz Piontek, Piotr Kopta, Paul Thompson, and Werner Dubitzky
- Published
- 2016
- Full Text
- View/download PDF
9. Development of Science Gateways Using QCG - Lessons Learned from the Deployment on Large Scale Distributed and HPC Infrastructures.
- Author
-
Tomasz Piontek, Bartosz Bosak, Milosz Ciznicki, Piotr Grabowski, Piotr Kopta, Michal Kulczewski, Dawid Szejnfeld, and Krzysztof Kurowski
- Published
- 2016
- Full Text
- View/download PDF
10. Multiscale Computing with the Multiscale Modeling Library and Runtime Environment.
- Author
-
Joris Borgdorff, Mariusz Mamonski, Bartosz Bosak, Derek Groen, Mohamed Ben Belgacem, Krzysztof Kurowski, and Alfons G. Hoekstra
- Published
- 2013
- Full Text
- View/download PDF
11. Multiscale Modelling and Simulation, 14th International Workshop.
- Author
-
Derek Groen, Bartosz Bosak, Valeria V. Krzhizhanovskaya, Alfons G. Hoekstra, and Petros Koumoutsakos
- Published
- 2017
- Full Text
- View/download PDF
12. Highly Efficient Parallel Approach to the Next-Generation DNA Sequencing.
- Author
-
Jacek Blazewicz, Bartosz Bosak, Piotr Gawron, Marta Kasprzak, Krzysztof Kurowski, Tomasz Piontek, and Aleksandra Swiercz
- Published
- 2011
- Full Text
- View/download PDF
13. Distributed multiscale computing with MUSCLE 2, the Multiscale Coupling Library and Environment.
- Author
-
Joris Borgdorff, Mariusz Mamonski, Bartosz Bosak, Krzysztof Kurowski, Mohamed Ben Belgacem, Bastien Chopard, Derek Groen, Peter V. Coveney, and Alfons G. Hoekstra
- Published
- 2014
- Full Text
- View/download PDF
14. Multiscale Modelling and Simulation, 13th International Workshop.
- Author
-
Derek Groen, Valeria V. Krzhizhanovskaya, Bartosz Bosak, Timothy D. Scheibe, and Alfons G. Hoekstra
- Published
- 2016
- Full Text
- View/download PDF
15. A Distributed Multiscale Computation of a Tightly Coupled Model Using the Multiscale Modeling Language.
- Author
-
Joris Borgdorff, Carles Bona-Casas, Mariusz Mamonski, Krzysztof Kurowski, Tomasz Piontek, Bartosz Bosak, Katarzyna Rycerz, Eryk Ciepiela, Tomasz Gubala, Daniel Harezlak, Marian Bubak, Eric Lorenz, and Alfons G. Hoekstra
- Published
- 2012
- Full Text
- View/download PDF
16. VECMAtk: a scalable verification, validation and uncertainty quantification toolkit for scientific simulations
- Author
-
Linda van Veen, O. O. Luk, Peter V. Coveney, Robert C. Sinclair, Jalal Lakhlili, Maxime Vassaux, Tomasz Piontek, D. P. Coster, Paul Karlshoefer, David W. Wright, N. Monnier, Diana Suleimenova, Erwan Raffin, Piotr Kopta, Michal Kulczewski, Daan Crommelin, Robin A. Richardson, Fredrik Jansson, Mariusz Bieniek, Derek Groen, Hamid Arabnejad, Anna Nikishova, Vytautas Jancauskas, O. Hoenen, Bartosz Bosak, Wouter Edeling, Brunel University London [Uxbridge], Centrum Wiskunde & Informatica, Amsterdam (CWI), Department of Chemistry [UCL, London], University College of London [London] (UCL), Computational Science Lab (IVI, FNWI), Analysis (KDV, FNWI), KdV Other Research (FNWI), Faculty of Science, and Centrum Wiskunde & Informatica, Amsterdam (CWI), The Netherlands
- Subjects
FOS: Computer and information sciences ,010504 meteorology & atmospheric sciences ,uncertainty quantification ,General Mathematics ,General Physics and Astronomy ,[INFO.INFO-CG]Computer Science [cs]/Computational Geometry [cs.CG] ,01 natural sciences ,010305 fluids & plasmas ,[SPI.MAT]Engineering Sciences [physics]/Materials ,[PHYS.MECA.MEMA]Physics [physics]/Mechanics [physics]/Mechanics of materials [physics.class-ph] ,multiscale simulations ,Software ,[PHYS.MECA.STRU]Physics [physics]/Mechanics [physics]/Structural mechanics [physics.class-ph] ,0103 physical sciences ,Validation ,[PHYS.MECA.SOLID]Physics [physics]/Mechanics [physics]/Solid mechanics [physics.class-ph] ,[SPI.GCIV.RISQ]Engineering Sciences [physics]/Civil Engineering/Risques ,[MATH.MATH-AP]Mathematics [math]/Analysis of PDEs [math.AP] ,Sensitivity (control systems) ,Uncertainty quantification ,Reliability (statistics) ,Research Articles ,0105 earth and related environmental sciences ,validation ,business.industry ,[SPI.GCIV.GEOTECH]Engineering Sciences [physics]/Civil Engineering/Géotechnique ,General Engineering ,Verification ,Articles ,[PHYS.MECA.MSMECA]Physics [physics]/Mechanics [physics]/Materials and structures in mechanics [physics.class-ph] ,[INFO.INFO-NA]Computer Science [cs]/Numerical Analysis [cs.NA] ,cs.MS ,Range (mathematics) ,Scalability ,Key (cryptography) ,Computer Science - Mathematical Software ,[SPI.GCIV.STRUCT]Engineering Sciences [physics]/Civil Engineering/Structures ,Multiscale simulations ,[SPI.GCIV.MAT]Engineering Sciences [physics]/Civil Engineering/Matériaux composites et construction ,verification ,Software engineering ,business ,Focus (optics) ,Mathematical Software (cs.MS) ,[SPI.GCIV.GCN]Engineering Sciences [physics]/Civil Engineering/Génie civil nucléaire - Abstract
We present the VECMA toolkit (VECMAtk), a flexible software environment for single and multiscale simulations that introduces directly applicable and reusable procedures for verification, validation (V&V), sensitivity analysis (SA) and uncertainty quantification (UQ). It enables users to verify key aspects of their applications, systematically compare and validate the simulation outputs against observational or benchmark data, and run simulations conveniently on any platform from the desktop to current multi-petascale computers. In this sequel to our paper on VECMAtk which we presented last year, we focus on a range of functional and performance improvements that we have introduced, cover newly introduced components, and applications examples from seven different domains such as conflict modelling and environmental sciences. We also present several implemented patterns for UQ/SA and V&V, and guide the reader through one example concerning COVID-19 modelling in detail., 23 pages, 10 figures
- Published
- 2021
17. The impact of uncertainty on predictions of the CovidSim epidemiological code
- Author
-
Diana Suleimenova, Bartosz Bosak, Peter V. Coveney, Derek Groen, Daan Crommelin, Hamid Arabnejad, Robbie Sinclair, Imran Mahmood, Wouter Edeling, Krishnakumar Gopalakrishnan, Analysis (KDV, FNWI), KdV Other Research (FNWI), Faculty of Science, and Computational Science Lab (IVI, FNWI)
- Subjects
Structure (mathematical logic) ,Computational model ,Coronavirus disease 2019 (COVID-19) ,Computer Networks and Communications ,Computer science ,SARS-CoV-2 ,media_common.quotation_subject ,010103 numerical & computational mathematics ,infectious diseases ,01 natural sciences ,Computer Science Applications ,010101 applied mathematics ,computational models ,Computer Science (miscellaneous) ,Code (cryptography) ,Econometrics ,Computational models ,Infectious diseases ,Quality (business) ,Sensitivity (control systems) ,0101 mathematics ,Uncertainty quantification ,Parametric statistics ,media_common - Abstract
Data availability Figure 1a,b displays publicly available cumulative death count data for the UK, which were obtained from ref. 22. Source Data are available with this paper. Furthermore, the parameter list—with all input parameters, a description, their default values and reasons for inclusion or exclusion from the Imperial College London CovidSim team—is available as Supplementary Data. Supplementary information: https://static-content.springer.com/esm/art%3A10.1038%2Fs43588-021-00028-9/MediaObjects/43588_2021_28_MOESM1_ESM.pdf Supplementary discussion, Figs. 1–7 and Table 1. Supplementary Data 1: https://static-content.springer.com/esm/art%3A10.1038%2Fs43588-021-00028-9/MediaObjects/43588_2021_28_MOESM2_ESM.xlsx All input parameters, a description, their default values and reasons for inclusion or exclusion from the Imperial College London CovidSim team. Source data Source Data Fig. 1: https://static-content.springer.com/esm/art%3A10.1038%2Fs43588-021-00028-9/MediaObjects/43588_2021_28_MOESM3_ESM.xlsx The computed cumulative death results for Fig. 1 and the observed death count data. Source Data Fig. 2: https://static-content.springer.com/esm/art%3A10.1038%2Fs43588-021-00028-9/MediaObjects/43588_2021_28_MOESM4_ESM.xlsx The Sobol indices of Fig. 2. Copyright © 2022 The Autor(s). Epidemiological modelling has assisted in identifying interventions that reduce the impact of COVID-19. The UK government relied, in part, on the CovidSim model to guide its policy to contain the rapid spread of the COVID-19 pandemic during March and April 2020; however, CovidSim contains several sources of uncertainty that affect the quality of its predictions: parametric uncertainty, model structure uncertainty and scenario uncertainty. Here we report on parametric sensitivity analysis and uncertainty quantification of the code. From the 940 parameters used as input into CovidSim, we find a subset of 19 to which the code output is most sensitive—imperfect knowledge of these inputs is magnified in the outputs by up to 300%. The model displays substantial bias with respect to observed data, failing to describe validation data well. Quantifying parametric input uncertainty is therefore not sufficient: the effect of model structure and scenario uncertainty must also be properly understood. Royal Society’s RAMP initiative: European Union Horizon 2020 research and innovation programme under grant agreement nos. 800925 (VECMA project; www.vecma.eu) and 823712 (CompBioMed2 Centre of Excellence, www.compbiomed.eu); UK EPSRC for the UK High-End Computing Consortium (grant no. EP/R029598/1).
- Published
- 2021
18. Distributed Multiscale Computing with MUSCLE 2, the Multiscale Coupling Library and Environment.
- Author
-
Joris Borgdorff, Mariusz Mamonski, Bartosz Bosak, Krzysztof Kurowski, Mohamed Ben Belgacem, Bastien Chopard, Derek Groen, Peter V. Coveney, and Alfons G. Hoekstra
- Published
- 2013
19. Verification, Validation and Uncertainty Quantification of Large-Scale Applications with QCG-PilotJob
- Author
-
Erwan Raffin, Jalal Lakhlili, Bartosz Bosak, Tomasz Piontek, Paul Karlshoefer, and Piotr Kopta
- Subjects
Focus (computing) ,business.industry ,Computer science ,media_common.quotation_subject ,Distributed computing ,Scale (chemistry) ,Interdependence ,Software ,Workflow ,Computer cluster ,Scalability ,Uncertainty quantification ,business ,media_common - Abstract
Efficient execution of large-scale and extremely demanding computational scenarios is a challenge for both the infrastructure providers and end-users, usually scientists, that need to develop highly scalable computational codes. Nevertheless, at this time, on the eve of exa-scale supercomputers, the particular role has to be given also to the intermediate software that can help in the preparation of applications so they can be efficiently executed on the emerging HPC systems. The efficiency and scalability of such software can be seen as priorities, however, these are not the only elements that should be addressed. Equally important is to offer software that is elastic, portable between platforms of different sizes, and easy to use. Trying to fulfill all the above needs we present QCG-PilotJob, a tool designed to enable flexible execution of numerous potentially dynamic and interdependent computing tasks in a single allocation on a computing cluster. QCG-PilotJob is built on many years of collaboration with computational scientists representing various domains and it responses to the practical requirements of real scientific use-cases. In this paper, we focus on the recent integration of QCG-PilotJob with the EasyVVUQ library and its successful use for Uncertainty Quantification workflows of several complex multiscale applications being developed within the VECMA project. However, we believe that with a well-thought-out design that allows for fully user-space execution and straightforward installation, QCG-PilotJob may be easily exploited in many other application scenarios, even by inexperienced users.
- Published
- 2021
20. Model uncertainty and decision making: Predicting the Impact of COVID-19 Using the CovidSim Epidemiological Code
- Author
-
Wouter Edeling, Peter V. Coveney, Imran Mahmood, Robert C. Sinclair, Arabnejad Hamid, Daan Crommelin, Bartosz Bosak, Derek Groen, Krishnakumar Gopalakrishnan, and Diana Suleimenova
- Subjects
Coronavirus disease 2019 (COVID-19) ,Computer science ,Code (cryptography) ,Data mining ,computer.software_genre ,computer - Abstract
The severe acute respiratory syndrome coronavirus 2 (SARS-CoV2) virus has rapidly spread worldwide since December 2019, and early modelling work of this pandemic has assisted in identifying effective government interventions. The UK government relied in part on the CovidSim model developed by the MRC Centre for Global Infectious Disease Analysis at Imperial College London, to model various non-pharmaceutical intervention strategies, and guide its government policy in seeking to deal with the rapid spread of the COVID-19 pandemic during March and April 2020. CovidSim is subject to different sources of uncertainty, namely parametric uncertainty in the inputs, model structure uncertainty (i.e., missing epidemiological processes) and scenario uncertainty, which relates to uncertainty in the set of conditions under which the model is applied. We have undertaken an extensive parametric sensitivity analysis and uncertainty quantification of the current CovidSim code. From the over 900 parameters that are provided as input to CovidSim, we identified a key subset of 19 parameters to which the code output is most sensitive. We find that the uncertainty in the code is substantial, in the sense that imperfect knowledge in these inputs will be magnified to the outputs, up to the extent of ca. 300%. Most of this uncertainty can be traced back to the sensitivity of three parameters. Compounding this, the model can display significant bias with respect to observed data, such that the output variance does not capture this validation data with high probability. We conclude that quantifying the parametric input uncertainty is not sufficient, and that the effect of model structure and scenario uncertainty cannot be ignored when validating the model in a probabilistic sense.
- Published
- 2020
21. Building Confidence in Simulation: Applications of EasyVVUQ
- Author
-
Derek Groen, Hamid Arabnejad, Peter V. Coveney, Irina Chirca, Piotr Kopta, Michal Kulczewski, Robin A. Richardson, Daan Crommelin, Tomasz Piontek, Diana Suleimenova, Jan Węglarz, Bartosz Bosak, Jalal Lakhlili, O. O. Luk, David W. Wright, Wouter Edeling, O. Hoenen, Robert C. Sinclair, Vytautas Jancauskas, Centrum Wiskunde & Informatica, Amsterdam (CWI), The Netherlands, Analysis (KDV, FNWI), and Computational Science Lab (IVI, FNWI)
- Subjects
Statistics and Probability ,Numerical Analysis ,Multidisciplinary ,010504 meteorology & atmospheric sciences ,Computer science ,uncertainty quantification ,high‐performance computing ,high-performance computing ,Python (programming language) ,Supercomputer ,01 natural sciences ,Multiscale modeling ,010305 fluids & plasmas ,Workflow ,multiscale simulations ,Application areas ,Modeling and Simulation ,0103 physical sciences ,Systems engineering ,Uncertainty quantification ,computer ,0105 earth and related environmental sciences ,computer.programming_language - Abstract
© 2020 The Authors. Validation, verification, and uncertainty quantification (VVUQ) of simulation workflows are essential for building trust in simulation results, and their increased use in decision‐making processes. The EasyVVUQ Python library is designed to facilitate implementation of advanced VVUQ techniques in new or existing workflows, with a particular focus on high‐performance computing, middleware agnosticism, and multiscale modeling. Here, the application of EasyVVUQ to five very diverse application areas is demonstrated: materials properties, ocean circulation modeling, fusion reactors, forced human migration, and urban air quality prediction. Horizon 2020 Framework Programme. Grant Number: 800925; Engineering and Physical Sciences Research Council
- Published
- 2020
22. The ESCAPE project: Energy-efficient Scalable Algorithms for Weather Prediction at Exascale
- Author
-
Milosz Ciznicki, Bent Hansen Sass, Geert Smet, Sami Saarinen, Christian Kühnlein, Gianmarco Mengaldo, Valentin Clément, Xavier Vigouroux, Michal Kulczewski, Michael Glinton, Carlos Osuna, Nils Wedi, Willem Deconinck, Alan Gray, Mats Hamrud, Philippe Marguinaud, Pierre Bénard, Alexander J. Macfaden, Nick New, Parijat Shukla, David Guibert, Erwan Raffin, Bartosz Bosak, George Mozdzynski, Fabrice Voitus, Joanna Szmelter, Yongjun Zheng, Sarah-Jane Lock, Alastair McKinstry, Cyril Mazauric, Charles Colavolpe, Marcin Procyk, Per Berg, Louis Douriez, Michael Lysaght, Sebastian Ciesielski, Mike Gillard, Pawel Spychala, Michail Diamantakis, Michael Lange, Kristian Pagh Nielsen, Jacob Weismann Poulsen, Wojciech Piątek, Piotr K. Smolarkiewicz, Krzysztof Kurowski, Peter Messmer, Oisín Robinson, Daniel Thiemert, Oliver Fuhrer, Andreas Müller, Piet Termonia, Peter Bauer, Daan Degrauwe, Andrzej A. Wyszogrodzki, Joris van Bever, Enda O’Brien, Michael Baldauf, Zbigniew P. Piotrowski, and Marek Błażewicz
- Subjects
010504 meteorology & atmospheric sciences ,Computer science ,Distributed computing ,Weather and climate ,02 engineering and technology ,01 natural sciences ,7. Clean energy ,Software portability ,0202 electrical engineering, electronic engineering, information engineering ,0105 earth and related environmental sciences ,020203 distributed computing ,lcsh:QE1-996.5 ,Solver ,PERFORMANCE ,Numerical weather prediction ,Supercomputer ,CLIMATE ,lcsh:Geology ,13. Climate action ,Earth and Environmental Sciences ,SIMULATION ,Programming paradigm ,ATMOSPHERIC MODELS ,Xeon Phi ,Efficient energy use ,GENERATION - Abstract
In the simulation of complex multi-scale flows arising in weather and climate modelling, one of the biggest challenges is to satisfy strict service requirements in terms of time to solution and to satisfy budgetary constraints in terms of energy to solution, without compromising the accuracy and stability of the application. These simulations require algorithms that minimise the energy footprint along with the time required to produce a solution, maintain the physically required level of accuracy, are numerically stable, and are resilient in case of hardware failure. The European Centre for Medium-Range Weather Forecasts (ECMWF) led the ESCAPE (Energy-efficient Scalable Algorithms for Weather Prediction at Exascale) project, funded by Horizon 2020 (H2020) under the FET-HPC (Future and Emerging Technologies in High Performance Computing) initiative. The goal of ESCAPE was to develop a sustainable strategy to evolve weather and climate prediction models to next-generation computing technologies. The project partners incorporate the expertise of leading European regional forecasting consortia, university research, experienced high-performance computing centres, and hardware vendors. This paper presents an overview of the ESCAPE strategy: (i) identify domain-specific key algorithmic motifs in weather prediction and climate models (which we term Weather & Climate Dwarfs), (ii) categorise them in terms of computational and communication patterns while (iii) adapting them to different hardware architectures with alternative programming models, (iv) analyse the challenges in optimising, and (v) find alternative algorithms for the same scheme. The participating weather prediction models are the following: IFS (Integrated Forecasting System); ALARO, a combination of AROME (Application de la Recherche à l'Opérationnel à Meso-Echelle) and ALADIN (Aire Limitée Adaptation Dynamique Développement International); and COSMO–EULAG, a combination of COSMO (Consortium for Small-scale Modeling) and EULAG (Eulerian and semi-Lagrangian fluid solver). For many of the weather and climate dwarfs ESCAPE provides prototype implementations on different hardware architectures (mainly Intel Skylake CPUs, NVIDIA GPUs, Intel Xeon Phi, Optalysys optical processor) with different programming models. The spectral transform dwarf represents a detailed example of the co-design cycle of an ESCAPE dwarf. The dwarf concept has proven to be extremely useful for the rapid prototyping of alternative algorithms and their interaction with hardware; e.g. the use of a domain-specific language (DSL). Manual adaptations have led to substantial accelerations of key algorithms in numerical weather prediction (NWP) but are not a general recipe for the performance portability of complex NWP models. Existing DSLs are found to require further evolution but are promising tools for achieving the latter. Measurements of energy and time to solution suggest that a future focus needs to be on exploiting the simultaneous use of all available resources in hybrid CPU–GPU arrangements.
- Published
- 2019
23. Application of the extreme scaling computing pattern on multiscale fusion plasma modelling
- Author
-
Alberto Bottino, Oliver Perks, Bruce D. Scott, D. P. Coster, O. O. Luk, O. Hoenen, Tomasz Piontek, Piotr Kopta, Bartosz Bosak, and Keeran Brabazon
- Subjects
Profiling (computer programming) ,Fusion ,Computer science ,Turbulence ,General Mathematics ,General Engineering ,Fusion plasma ,General Physics and Astronomy ,Articles ,Supercomputer ,01 natural sciences ,7. Clean energy ,010305 fluids & plasmas ,Computational science ,Workflow ,0103 physical sciences ,010306 general physics ,Scaling - Abstract
The extreme scaling pattern of the ComPat project is applied to a multi-scale workflow relevant to the magnetically confined fusion problem. This workflow combines transport, turbulence and equilibrium codes (together with additional auxiliaries such as initial conditions and numerical module), which aims at calculating the behaviour of a fusion plasma on long (transport) time scales based on information from much faster (turbulence) time scales. Initial findings of profile measurements are reported in this paper and indicate that, depending on the chosen performance metric for defining ‘cost’, such as time to completion, efficiency and total energy consumption of the mutliscale workflow, different choices on the number of cores would be made when determining the optimal execution configuration. A variant of the workflow which increases the inherent parallelism is presented, and shown to produce equivalent results at (typically) lower cost compared with the original workflow. This article is part of the theme issue ‘Multiscale modelling, simulation and computing: from the desktop to the exascale’.
- Published
- 2019
24. Patterns for High Performance Multiscale Computing
- Author
-
Derek Groen, Vytautas Jancauskas, Saad Alowayyed, Peter V. Coveney, Piotr Kopta, Bartosz Bosak, Oliver Perks, O. O. Luk, Tomasz Piontek, O. Hoenen, James L. Suter, Krzysztof Kurowski, Alfons G. Hoekstra, Keeran Brabazon, D. P. Coster, and Computational Science Lab (IVI, FNWI)
- Subjects
Modelling methodology ,Computer Networks and Communications ,Computer science ,business.industry ,Distributed computing ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,020206 networking & telecommunications ,02 engineering and technology ,Load balancing (computing) ,Supercomputer ,Model coupling ,Multiscale computing ,Software ,Hardware and Architecture ,Middleware ,0202 electrical engineering, electronic engineering, information engineering ,Graph (abstract data type) ,Leverage (statistics) ,020201 artificial intelligence & image processing ,High performance computing ,business - Abstract
We describe our Multiscale Computing Patterns software for High Performance Multiscale Computing. Following a short review of Multiscale Computing Patterns, this paper introduces the Multiscale Computing Patterns Software, which consists of description, optimisation and execution components. First, the description component translates the task graph, representing a multiscale simulation, to a particular type of multiscale computing pattern. Second, the optimisation component selects and applies algorithms to find the most suitable mapping between submodels and available HPC resources. Third, the execution component which a middleware layer maps submodels to the number and type of physical resources based on the suggestions emanating from the optimisation part together with infrastructure-specific metrics such as queueing time and resource availability. The main purpose of the Multiscale Computing Patterns software is to leverage the Multiscale Computing Patterns to simplify and automate the execution of complex multiscale simulations on high performance computers, and to provide both application-specific and pattern-specific performance optimisation. We test the performance and the resource usage for three multiscale models, which are expressed in terms of two Multiscale Computing Patterns. In doing so, we demonstrate how the software automates resource selection and load balancing, and delivers performance benefits from both the end-user and the HPC system level perspectives.
- Published
- 2019
25. Development of Science Gateways Using QCG — Lessons Learned from the Deployment on Large Scale Distributed and HPC Infrastructures
- Author
-
Bartosz Bosak, Piotr Grabowski, Krzysztof Kurowski, Tomasz Piontek, Dawid Szejnfeld, Piotr Kopta, Michal Kulczewski, and M. CiźNicki
- Subjects
Communication design ,Computer Networks and Communications ,Computer science ,business.industry ,020206 networking & telecommunications ,Cloud computing ,02 engineering and technology ,Supercomputer ,Grid ,computer.software_genre ,World Wide Web ,Software ,User experience design ,Hardware and Architecture ,Software deployment ,Middleware (distributed applications) ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,business ,computer ,Information Systems - Abstract
Today, various Science Gateways created in close collaboration with scientific communities provide access to remote and distributed HPC, Grid and Cloud computing resources and large-scale storage facilities. However, as we have observed there are still many entry barriers for new users and various limitations for active scientists. In this paper we present our latest achievements and software solutions that significantly simplify the use of large scale and distributed computing. We describe several Science Gateways that have been successfully created with the help of our application tools and the QCG (Quality in Cloud and Grid) middleware, in particular Vine Toolkit, QCG-Portal and QCG-Now, and make the use of HPC, Grid and Cloud more straightforward and transparent. Additionally, we share the best practices and lessons learned after creating jointly with user communities many domain-specific Science Gateways, e.g. dedicated for physicists, medical scientists, chemists, engineers and external communities performing multi-scale simulations. As our deployed software solutions have reached recently a critical mass of active users in the PLGrid e-infrastructure in Poland, we also discuss in this paper how changing technologies, visual design and user experience could impact the way we should re-design Science Getaways or even develop new attractive tools, e.g. desktop or mobile-based applications in the future. Finally, we present information and statistics regarding the behaviour of users to help readers understand how new capabilities and functionalities may influence the growth of user interest in Science Gateways and HPC technologies.
- Published
- 2016
26. MultiGrain/MAPPER: A distributed multiscale computing approach to modeling and simulating gene regulation networks
- Author
-
Tomasz Piontek, Martin T. Swain, Alexandru Mizeranschi, Werner Dubitzky, Bartosz Bosak, Paul Thompson, Quentin Fazilleau, Raluca Scona, and Piotr Kopta
- Subjects
0301 basic medicine ,Regulation of gene expression ,Distributed Computing Environment ,Theoretical computer science ,Computational complexity theory ,Computer Networks and Communications ,business.industry ,Computer science ,Distributed computing ,Systems biology ,Gene regulatory network ,Particle swarm optimization ,02 engineering and technology ,Modeling and simulation ,03 medical and health sciences ,030104 developmental biology ,Software ,Hardware and Architecture ,0202 electrical engineering, electronic engineering, information engineering ,Key (cryptography) ,020201 artificial intelligence & image processing ,business - Abstract
Modeling and simulation of gene-regulatory networks (GRNs) has become an important aspect of modern systems biology investigations into mechanisms underlying gene regulation. A key task in this area is the automated inference or reverse-engineering of dynamic mechanistic GRN models from gene expression time-course data. Besides a lack of suitable data (in particular multi-condition data from the same system), one of the key challenges of this task is the computational complexity involved. The more genes in the GRN system and the more parameters a GRN model has, the higher the computational load. The computational challenge is likely to increase substantially in the near future when we tackle larger GRN systems. The goal of this study was to develop a distributed computing framework and system for reverse-engineering of GRN models. We present the resulting software called MultiGrain/MAPPER. This software is based on a new architecture and tools supporting multiscale computing in a distributed computing environment. A key feature of MultiGrain/MAPPER is the realization of GRN reverse-engineering based on the underlying distributed computing framework and multi-swarm particle swarm optimization. We demonstrate some of the features of MultiGrain/MAPPER and evaluate its performance using both real and artificial gene expression data.
- Published
- 2016
27. Distributed multiscale computing with MUSCLE 2, the Multiscale Coupling Library and Environment
- Author
-
Krzysztof Kurowski, Alfons G. Hoekstra, Derek Groen, Mariusz Mamonski, Bastien Chopard, M. Ben Belgacem, Bartosz Bosak, Joris Borgdorff, Peter V. Coveney, and Computational Science Lab (IVI, FNWI)
- Subjects
FOS: Computer and information sciences ,General Computer Science ,Java ,Modeling language ,Computer science ,Fortran ,Message Passing Interface ,02 engineering and technology ,Parallel computing ,GridFTP ,Software_PROGRAMMINGTECHNIQUES ,01 natural sciences ,010305 fluids & plasmas ,Theoretical Computer Science ,Computational Engineering, Finance, and Science (cs.CE) ,Multiscale modelling ,Model coupling ,Modelling and Simulation ,0103 physical sciences ,0202 electrical engineering, electronic engineering, information engineering ,ddc:025.063 ,Computer Science - Computational Engineering, Finance, and Science ,computer.programming_language ,Computer Science - Performance ,Execution environment ,Distributed multiscale computing ,Python (programming language) ,MUSCLE ,Canal system ,Performance (cs.PF) ,Computer Science - Distributed, Parallel, and Cluster Computing ,Multiscale coupling ,Modeling and Simulation ,020201 artificial intelligence & image processing ,Distributed, Parallel, and Cluster Computing (cs.DC) ,computer ,Computer Science(all) - Abstract
We present the Multiscale Coupling Library and Environment: MUSCLE 2. This multiscale component-based execution environment has a simple to use Java, C++, C, Python and Fortran API, compatible with MPI, OpenMP and threading codes. We demonstrate its local and distributed computing capabilities and compare its performance to MUSCLE 1, file copy, MPI, MPWide, and GridFTP. The local throughput of MPI is about two times higher, so very tightly coupled code should use MPI as a single submodel of MUSCLE 2; the distributed performance of GridFTP is lower, especially for small messages. We test the performance of a canal system model with MUSCLE 2, where it introduces an overhead as small as 5% compared to MPI., Comment: 18 pages, 22 figures, submitted to journal
- Published
- 2014
28. Multiscale modelling and simulation, 13th international workshop
- Author
-
Alfons G. Hoekstra, Derek Groen, Timothy D. Scheibe, Bartosz Bosak, Valeria V. Krzhizhanovskaya, and Computational Science Lab (IVI, FNWI)
- Subjects
Multiscale ,Computer science ,Management science ,Multiphysics ,simulation ,Multiscale modeling ,Modelling ,Bridging (programming) ,modelling ,Scientific analysis ,Coupling ,multiphysics ,multiscale ,Multidisciplinary approach ,Software deployment ,Systems engineering ,General Earth and Planetary Sciences ,coupling ,Simulation ,General Environmental Science - Abstract
Multiscale Modelling and Simulation (MMS) is a cornerstone in the today's research in computational science. Simulations containing multiple models, with each model operating at a different temporal or spatial scale, are a challenging setting that frequently require innovative approaches in areas such as scale bridging, code deployment, error quantification, and scientific analysis. The aim of the MMS workshop is to encourage and consolidate the progress in this multidisciplinary research field, both in the areas of the scientific applications and the underlying infrastructures that enable these applications. Here we briefly introduce the scope of the workshop and highlight some of the key aspects of this year's submissions.
- Published
- 2016
29. A Distributed Multiscale Computation of a Tightly Coupled Model Using the Multiscale Modeling Language
- Author
-
Mariusz Mamonski, Krzysztof Kurowski, Eric Lorenz, Marian Bubak, Katarzyna Rycerz, Alfons G. Hoekstra, Eryk Ciepiela, Bartosz Bosak, Joris Borgdorff, Tomasz Piontek, Tomasz Gubała, Daniel Harezlak, Carles Bona-Casas, and Computational Science Lab (IVI, FNWI)
- Subjects
distributed multiscale computing ,Computer science ,Computation ,MML ,multiscale modeling language ,Gridspace ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,02 engineering and technology ,MUSCLE ,in-stent restenosis ,multiscale modeling ,01 natural sciences ,Multiscale modeling ,Toolchain ,010305 fluids & plasmas ,Computational science ,0103 physical sciences ,0202 electrical engineering, electronic engineering, information engineering ,General Earth and Planetary Sciences ,Workbench ,020201 artificial intelligence & image processing ,QCG-Broker ,General Environmental Science - Abstract
Nature is observed at all scales; with multiscale modeling, scientists bring together several scales for a holistic analysis of a phenomenon. The models on these different scales may require significant but also heterogeneous computational resources, creating the need for distributed multiscale computing. A particularly demanding type of multiscale models, tightly coupled, brings with it a number of theoretical and practical issues. In this contribution, a tightly coupled model of in-stent restenosis is first theoretically examined for its multiscale merits using the Multiscale Modeling Language (MML); this is aided by a toolchain consisting of MAPPER Memory (MaMe), the Multiscale Application Designer (MAD), and Gridspace Experiment Workbench. It is implemented and executed with the general Multiscale Coupling Library and Environment (MUSCLE). Finally, it is scheduled amongst heterogeneous infrastructures using the QCG-Broker. This marks the first occasion that a tightly coupled application uses distributed multiscale computing in such a general way.
- Published
- 2012
30. Multiscale Modelling and Simulation, 14th International Workshop
- Author
-
Alfons G. Hoekstra, Petros Koumoutsakos, Derek Groen, Bartosz Bosak, Valeria V. Krzhizhanovskaya, and Computational Science Lab (IVI, FNWI)
- Subjects
Scientific analysis ,Software deployment ,Computer science ,Multiphysics ,Systems engineering ,General Earth and Planetary Sciences ,General Environmental Science ,Bridging (programming) - Abstract
Multiscale Modelling and Simulation (MMS) is a cornerstone in today’s research in computational science. Simulations containing multiple models, with each model operating at a different temporal or spatial scale, are a challenging setting that frequently require innovative approaches in areas such as scale bridging, code deployment, error quantification, and scientific analysis. The aim of the MMS workshop is to encourage and consolidate the progress in this multi-disciplinary research field, both in the areas of the scientific applications and the underlying infrastructures that enable these applications. Here we briefly introduce the scope of the workshop and highlight some of the key aspects of this year’s submissions.
- Published
- 2017
31. Reservations for Compute Resources in Federated e-Infrastructure
- Author
-
Mariusz Mamonski, Rafał Kluszczyński, Tadeusz Szymocha, Marcin Radecki, Bartosz Bosak, Krzysztof Benedyczak, Tomasz Piontek, and Pawel Wolniewicz
- Subjects
Identifier ,Resource (project management) ,Database ,Computer science ,Middleware ,Batch processing ,Reservation ,Normalization (sociology) ,E infrastructure ,Context (language use) ,computer.software_genre ,computer - Abstract
This paper presents work done to prepare compute resource reservations in the PL-Grid Infrastructure. A compute resource reservation allows a user to allocate some fraction of resources for exclusive access, when reservation is prepared. That way the user is able to run his/her job without waiting for allocating resources in a batch system. In the PL-Grid Infrastructure reservations can be allocated up to amount negotiated in a PL-Grid grant. One way of getting reservation is allocation by a resource administrator. Another way is to use predefined pool of resources accessible by various middleware. In both approaches once obtained, reservations identifiers can be used by middleware during job submissions. Enabling reservations requires changes in middleware. The modifications needed in each middleware will be described. The possible extension of existing reservation model in the PL-Grid Infrastructure can be envisaged: reservation usage normalization and reservation accounting. The reservations are created and utilized in the user's context, so there must be a way to pass the reservation details from the user-level tools to a batch system. Each of PL-Grid supported middleware, namely gLite, UNICORE and QosCosGrid, required adaptations to implement this goal.
- Published
- 2014
32. Preprocessing and Storing High-Throughput Sequencing Data
- Author
-
Aleksandra Świercz, Bartosz Bosak, Marek Chłopkowski, Arkadiusz Hoffa, Marta Kasprzak, Krzysztof Kurowski, Tomasz Piontek, and Jacek Błażewicz
- Subjects
Theoretical computer science ,Computer science ,Preprocessor ,General Medicine ,Data mining ,computer.software_genre ,computer ,DNA sequencing - Published
- 2014
- Full Text
- View/download PDF
33. New QosCosGrid Middleware Capabilities and Its Integration with European e-Infrastructure
- Author
-
Mariusz Mamonski, Tomasz Piontek, Piotr Kopta, Krzysztof Kurowski, and Bartosz Bosak
- Subjects
Computer science ,End user ,business.industry ,Distributed computing ,Reservation ,Scenario ,computer.software_genre ,Workflow ,Grid computing ,Middleware (distributed applications) ,Message oriented middleware ,Resource management ,Software engineering ,business ,computer - Abstract
QosCosGrid QCG is an integrated system offering leading job and resource management capabilities in order to deliver supercomputer-like performance and structure to end users. By combining many distributed computing resources together, QCG offers highly efficient mapping, execution and monitoring capabilities for a variety of applications, such as parameter sweep, workflows, multi-scale, MPI or hybrid MPI-OpenMP. The QosCosGrid middleware also provides ai¾źset of unique features, such as advance reservation, co-allocation of distributed computing resources, support for interactive tasks and monitoring of ai¾źprogress of running applications. The middleware is offered to end users by well-designed and easy-to-use client tools. At the time of writing, QosCosGrid is the most popular middleware within the PL-Grid Infrastructure. After its successful adoption within the Polish research communities, it has been integrated with the EGI infrastructure and through a release in UMD and EGI-AppDB it is also available at European level. In this article, we focus on the extensions that were introduced to QosCosGrid during the period of the PL-Grid and PLGrid Plus projects in order to support advanced user scenarios and to integrate the stack with the Polish and European e-Infrastructures.
- Published
- 2014
34. Multiscale Computing with the Multiscale Modeling Library and Runtime Environment
- Author
-
Mohamed Ben Belgacem, Bartosz Bosak, Joris Borgdorff, Derek Groen, Krzysztof Kurowski, Mariusz Mamonski, Alfons G. Hoekstra, and Computational Science Lab (IVI, FNWI)
- Subjects
distributed multiscale computing ,Java ,Fortran ,Computer science ,Quantitative Biology::Tissues and Organs ,Physics::Medical Physics ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,02 engineering and technology ,computer.software_genre ,01 natural sciences ,010305 fluids & plasmas ,Computational science ,Computer Science::Robotics ,Component (UML) ,0103 physical sciences ,0202 electrical engineering, electronic engineering, information engineering ,Multiscale modeling ,ddc:025.063 ,General Environmental Science ,computer.programming_language ,ComputingMethodologies_COMPUTERGRAPHICS ,Distributed multiscale computing ,MUSCLE ,multiscale modeling ,General Earth and Planetary Sciences ,020201 artificial intelligence & image processing ,Data mining ,computer - Abstract
We introduce a software tool to simulate multiscale models: the Multiscale Coupling Library and Environment 2 (MUSCLE 2). MUSCLE 2 is a component-based modeling tool inspired by the multiscale modeling and simulation framework, with an easy-to-use API which supports Java, C++, C, and Fortran. We present MUSCLE 2's runtime features, such as its distributed computing capabilities, and its benefits to multiscale modelers. We also describe two multiscale models that use MUSCLE 2 to do distributed multiscale computing: an in-stent restenosis and a canal system model. We conclude that MUSCLE 2 is a notable improvement over the previous version of MUSCLE, and that it allows users to more flexibly deploy simulations of multiscale models, while improving their performance.
- Published
- 2013
35. QosCosGrid e-Science Infrastructure for Large-Scale Complex System Simulations
- Author
-
George Kampis, Thomas Herault, Camille Coti, Piotr Grabowski, Krzysztof Kurowski, Mariusz Mamonski, Bartosz Bosak, Tomasz Piontek, László Gulyás, and Franck Cappello
- Subjects
business.industry ,Computer science ,End user ,Distributed computing ,Quality of service ,Suite ,Cloud computing ,Troubleshooting ,computer.software_genre ,Resource (project management) ,Middleware (distributed applications) ,e-Science ,business ,computer - Abstract
This chapter focuses on two enhanced and widely used parallel computing environments: QosCosGrid-ProActive (QCG-ProActive) and QosCosGrid-OpenMPI (QCG-OMPI). QCG-OMPI is designed to enable parallel applications to run across geographically widely distributed computational resources owned and secured by different administrative organizations. QCG-ProActive was successfully integrated with the Repast Suite, a very popular Java-based, agent-based modeling and simulation platform. The chapter presents a short overview of validation scenarios. It discusses the main requirements of complex systems and other demanding parallel applications and classifies these into ?> templates. ?> The chapter introduces the QCG middleware services and their key capabilities relevant to end users. A number of useful libraries and frameworks for application developers and resource providers using QCG are described. The chapter provides additional Web-based monitoring and troubleshooting tools for QCG, which are already available on some production-level computational clusters and supercomputers. large-scale systems; middleware; modelling
- Published
- 2012
36. Highly Integrated Environment for Parallel Application Development Using QosCosGrid Middleware
- Author
-
Krzysztof Kurowski, Bartosz Bosak, ski, Jan Konczak, Mariusz Mamo, and Tomasz Piontek
- Subjects
business.industry ,Computer science ,media_common.quotation_subject ,Usability ,computer.software_genre ,Set (abstract data type) ,Development (topology) ,Debugging ,User experience design ,Middleware ,Operating system ,Architecture ,business ,computer ,Eclipse ,media_common - Abstract
QosCosGrid infrastructure has been prepared, introduced successfully on multiple clusters and comprehensively tested. In order to enhance user experience and increase usability, a set of additional tools must be prepared. Profiting from distributed software becomes more popular nowadays, but the number of tools supporting developers of parallel programs is still smaller than developers' needs. Eclipse PTP is the leading open source tool among Integrated Development Environments for distributed software. In this chapter we shortly present Eclipse PTP and related Eclipse plug-ins aiding programmers in developing parallel software. Later we describe how QosCosGrid middleware has been integrated with Eclipse PTP. We present the plug-in basic concepts, such as the Resource Manager or Remote Services, general architecture, functionality of the PTP QCG plug-in and advantages over other solutions of this kind.
- Published
- 2012
37. Highly Efficient Parallel Approach to the Next-Generation DNA Sequencing
- Author
-
Krzysztof Kurowski, Bartosz Bosak, Tomasz Piontek, Marta Kasprzak, Aleksandra Swiercz, Jacek Blazewicz, and Piotr Gawron
- Subjects
Sequence ,Single node ,Hierarchy (mathematics) ,Computer science ,Computation ,Sequence assembly ,Parallel computing ,DNA sequencing - Abstract
Due to the rapid development of the technology, next-generation sequencers can produce huge amount of short DNA fragments covering a genomic sequence of an organism in short time. There is a need for the time-efficient algorithms which could assembly these fragments together and reconstruct the examined DNA sequence. Previously proposed algorithm for de novo assembly, SR-ASM, produced results of high quality, but required a lot of time for computations. The proposed hybrid parallel programming strategy allows one to use the two-level hierarchy: computations in threads (on a single node with many cores) and computations on different nodes in a cluster. The tests carried out on real data of Prochloroccocus marinus coming from Roche sequencer showed, that the algorithm was speeded up 20 times in comparison to the sequential approach with the maintenance of the high accuracy and beating results of other algorithms.
- Published
- 2012
38. New Capabilities in QosCosGrid Middleware for Advanced Job Management, Advance Reservation and Co-allocation of Computing Resources – Quantum Chemistry Application Use Case
- Author
-
ski, Jacek Komasa, Tomasz Piontek, Krzysztof Kurowski, Piotr Kopta, Mariusz Mamo, and Bartosz Bosak
- Subjects
Workflow ,Utility computing ,Computer science ,Middleware ,Distributed computing ,Programming paradigm ,High-throughput computing ,Supercomputer ,Resource management (computing) ,Grid - Abstract
In this chapter we present the new capabilities of QosCosGrid (QCG) middleware for advanced job and resource management in the grid environment. By connecting many computing clusters together, QosCosGrid offers easy-to-use mapping, execution and monitoring capabilities for a variety of complex computations, such as parameter sweep, workflows, MPI or hybrid MPI-OpenMP as well as multiscale simulations. Thanks to QosCosGrid, large-scale programming models written in Fortran, C, C++ or Java can be automatically distributed over a network of computing resources with guaranteed Quality of Service --- for example guaranteed startup time of a job. Consequently, applications can be run at specified periods with reduced execution time and waiting times. This enables more complex problem instances to be addressed. In order to prove the usefulness of the new functionality of QosCosGrid a detailed description of the system along with a real use case scenario from the quantum chemistry science domain will be presented in this chapter.
- Published
- 2012
39. Parallel Large Scale Simulations in the PL-Grid Environment
- Author
-
Krzysztof Kurowski, Tomasz Piontek, Piotr Kopta, Mariusz Mamoński, and Bartosz Bosak
- Subjects
Java ,Fortran ,Computer science ,Distributed computing ,General Medicine ,Grid ,Supercomputer ,computer.software_genre ,Resource (project management) ,Semantic grid ,Grid computing ,Middleware ,computer ,computer.programming_language - Abstract
The growing demand for computational power causes that Grids are becoming mission-critical components in research and industry, offering sophisticated solutions in leveraging large-scale computing and storage resources. The nature a Grid in which resources are usually shared among multiple organizations offering resources under their control based on the "best effort" approach with no guarantee concerning the quality-of-service may be inadequate to support large-scale simulations. Requirements of such simulations often exceed capabilities of a single computing center causing the need to simultaneously allocate and synchronize resources belonging to many administrative domains whose functionality is missing in leading grid middlewares preventing researchers from executing large-scale simulations in grids. The paper presents tools and services that were designed to build multilayered infrastructure capable of dealing with computationally intensive large-scale simulations in the grid environment. The developed and deployed middleware enables computing clusters in different administrative domains to be virtually welded into a single powerful compute resource that can be treated as a quasi-opportunistic supercomputer. We describe the middleware developed in the QosCosGrid project and being enhanced under the PL-Grid national grid initiative, which provides advance reservation and resource co-allocation functionality as well as support for parallel large-scale applications based on OpenMPI (for C/C++ and Fortran) or ProActive for Java.
- Published
- 2010
- Full Text
- View/download PDF
40. Predicting queue wait time probabilities for multi-scale computing
- Author
-
Bartosz Bosak, Tomasz Piontek, Vytautas Jancauskas, and Piotr Kopta
- Subjects
Scale (ratio) ,Computer science ,multi-scale computing ,General Mathematics ,Distributed computing ,General Engineering ,General Physics and Astronomy ,020206 networking & telecommunications ,02 engineering and technology ,Articles ,Supercomputer ,Wait time ,Naive Bayes classifier ,Naive Bayes ,Resource (project management) ,machine learning ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Queue ,Selection (genetic algorithm) ,Research Article - Abstract
We describe a method for queue wait time prediction in supercomputing clusters. It was designed for use as a part of multi-criteria brokering mechanisms for resource selection in a multi-site High Performance Computing environment. The aim is to incorporate the time jobs stay queued in the scheduling system into the selection criteria. Our method can also be used by the end users to estimate the time to completion of their computing jobs. It uses historical data about the particular system to make predictions. It returns a list of probability estimates of the form ( t i , p i ), where p i is the probability that the job will start before time t i . Times t i can be chosen more or less freely when deploying the system. Compared to regression methods that only return a single number as a queue wait time estimate (usually without error bars) our prediction system provides more useful information. The probability estimates are calculated using the Bayes theorem with the naive assumption that the attributes describing the jobs are independent. They are further calibrated to make sure they are as accurate as possible, given available data. We describe our service and its REST API and the underlying methods in detail and provide empirical evidence in support of the method's efficacy. This article is part of the theme issue ‘Multiscale modelling, simulation and computing: from the desktop to the exascale’.
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.