Search

The LEXIS (Large-scale EXecution for Industry & Society) Weather and Climate Pilot is developing a system for the prediction of water-related phenomena and their associated socioeconomic impacts. The system is based on multiple models chained together, as global weather models, high-resolution regional weather models, domain-specific application models (hydrological and forest fire risk forecasts), and impact models providing information (such as air quality and rainfall intensity) for key decisions and policymakers. This chapter describes the key aspect of this pilot in terms of serving model output data and products with cloud and high-performance data analytics environments, on top of a Weather Climate Data API, as well as the porting of models on the LEXIS infrastructure via different virtualization strategies (as virtual machine and dockers).

The development of community-specific user interfaces of a science gateway can be a challenging task for non-IT experts. This contribution proposes an original, easy-to-use solution to tackle this issue based on EasyGateway. EasyGateway is a modern, lightweight solution for the development of science gateway able to interplay with most toolkits. In this paper we present how EasyGateway can “dress” Apache Airavata to manage experiment configurations and job submissions. We discuss the proposed approach considering a real case application represented by the Weather Research and Forecasting Model (WRF), a community model exploited by a large number of users in meteorology and climatology domains. The combined use of EasyGateway and Apache Airavata leads to an improved user experience, enhancing the model configuration phase with support in finding inconsistencies immediately, but also exploiting the possibility of accessing a potentially large set of computational resources to perform model execution.

The e-Science environment developed in the framework of the EU-funded DRIHM project was used to demonstrate its ability to provide relevant, meaningful hydrometeorological forecasts. This was illustrated for the tragic case of 4 November 2011, when Genoa, Italy, was flooded as the result of heavy, convective precipitation that inundated the Bisagno catchment. The Meteorological Model Bridge (MMB), an innovative software component developed within the DRIHM project for the interoperability of meteorological and hydrological models, is a key component of the DRIHM e-Science environment. The MMB allowed three different rainfall-discharge models (DRiFt, RIBS and HBV) to be driven by four mesoscale limited-area atmospheric models (WRF-NMM, WRF-ARW, Meso-NH and AROME) and a downscaling algorithm (RainFARM) in a seamless fashion. In addition to this multi-model configuration, some of the models were run in probabilistic mode, thus giving a comprehensive account of modelling errors and a very large amount of likely hydrometeorological scenarios (> 1500). The multi-model approach proved to be necessary because, whilst various aspects of the event were successfully simulated by different models, none of the models reproduced all of these aspects correctly. It was shown that the resulting set of simulations helped identify key atmospheric processes responsible for the large rainfall accumulations over the Bisagno basin. The DRIHM e-Science environment facilitated an evaluation of the sensitivity to atmospheric and hydrological modelling errors. This showed that both had a significant impact on predicted discharges, the former being larger than the latter. Finally, the usefulness of the set of hydrometeorological simulations was assessed from a flash flood early-warning perspective.

From 1970 to 2012, about 9,000 high-impact weather events were reported globally, causing the loss of 1.94 million lives and damage of $2.4 trillion (U.S. dollars). The scientific community is called to action to improve the predictive ability of such events and communicate forecasts and associated risks both to affected populations and to those making decisions. At the heart of this challenge lies the ability to have easy access to hydrometeorological data and models and to facilitate the necessary collaboration between meteorologists, hydrologists, and computer science experts to achieve accelerated scientific advances. Two European Union (EU)-funded projects, Distributed Research Infrastructure for Hydro-Meteorology (DRIHM) and DRIHM to United States of America (DRIHM2US), sought to help address this challenge by developing a prototype e-science environment providing advanced end-to-end services (models, datasets, and postprocessing tools), with the aim of paving the way to a step change in how scientists can approach studying these events, with a special focus on flood events in complex topographic areas. This paper describes the motivation and philosophy behind this prototype e-science environment together with certain key components, focusing on hydrometeorological aspects that are then illustrated through actionable research for a critical flash flood event that occurred in October 2014 in Liguria, Italy.

Complex distributed architectures, like Grid, supply effective platforms to solve computations on huge datasets, often at the cost of increased power consumption. This energy issue affects the sustainability of the infrastructures and increases their environmental impact. On the other hand, due to Grid heterogeneity and scalability, possible power savings could be achieved if effective energy-aware allocation policies were adopted. These policies are meant to implement a better coupling between application requirements and the Grid resources, also taking energy parameters into account. In this paper, we discuss different allocation strategies which address jobs submitted to Grid resources, subject to efficiency and energy constraints. Our aim is to analyze the potential benefits that can be obtained from the adoption of a metric able to capture both performance and energy-savings. Based on an experimental study, we simulated two alternative scenarios aimed at comparing the behavior of different strategies for allocating jobs to resources. Moreover we introduced the Performance/Energy Trade-off function as a useful means to evaluate the tendency of an allocation strategy toward efficiency or power consumption. Our conclusion seems to suggest that performance and energy-savings are not always enemies, and these objectives may be combined if suitable energy metrics are adopted.

Grid environments must provide effective mechanisms able to select the most adequate resources satisfying application requirements. A description of applications and resources, grounded on a common and shared basis, is crucial to favour an effective pairing. A suitable criterion to match demand with supply is to characterize resources by means of their performance evaluated through the execution of low-level and application-specific benchmarks. We present GREEN, a distributed Matchmaker, based on a two-level benchmarking methodology. GREEN facilitates the ranking of Grid resources and the submission of jobs to the Grid, through the specification of both syntactic and performance requirements, independently of the underlying middleware and thus fostering Grid interoperability.

The flexible and pay-as-you-go computing capabilities offered by Cloud infrastructures are very attractive for high-demanding e-Science applications like weather prediction simulators. For their ability to couple the scalability offered by public service provider with the greater control and customization provided by Private Clouds, Hybrid Clouds seem a particularly appealing solution to support meteorological researchers and weather departments in their every-day activity. Cloud Brokers interfacing customers with Cloud providers, may support scientists in the deployment and execution of demanding meteorological simulations, by hiding all the intricacies related to the management of powerful but often complex HPC systems. The paper presents a set of brokering strategies for Hybrid Clouds aimed at the execution of various instances of the weather prediction WRF model subject to different user requirements and computational conditions. A simulation-based analysis documents the performance of the different scheduling strategies at varying workloads and system configuration. © 2014 Elsevier B.V. All rights reserved.

Scientific image processing is a topic of interest for a broad scientific community since it is a mean of gaining understanding and insight into the data for a growing number of applications. Furthermore, the technological evolution permits large data acquisition, with sophisticated instruments, and their elaboration through complex multidisciplinary applications, resulting in datasets that are growing at an extremely rapid pace. This results in the need of huge computational power for the processing. It is necessary to move towards High Performance Computing (HPC) and to develop proper parallel implementations of image processing algorithms/operations. Modern HPC resources are typically highly heterogeneous systems, composed of multiple CPUs and accelerators such as Graphics Processing Units (GPUs) and Field-Programmable Gate Arrays (FPGAs). The actual barrier posed by heterogeneous HPC resources is the development and/or the performance efficient porting of software on such complex architectures. In this context, the aim of this work is to enable image processing on cluster of CPUs, through the use of PIMA(GE)(2) Lib, the Parallel IMAGE processing GEnoa Library. The library is able to exploit traditional clusters through MPI, GPU device through CUDA and a first experimentation is aimed to explore the use of CPU-clusters. Library operations are provided to the users through a sequential interface defined to hide the parallelism of the computation. The parallel computation, at each level, is managed employing specific policies designed to suitably coordinate the parallel processes/threads involved in the elaboration and their use is tightly coupled with the PIMA(GE)2 Lib interface. In this paper, we present the incremental approach adopted in the development of the library and the performance gains in each implementations: quite linear speedup is achieved on cluster architecture, about a 30% improvement in the execution time on a single GPU and the first results on cluster of GPUs are promising. (C) 2014 Elsevier B.V. All rights reserved.

The Poisson-Boltzmann equation models the electrostatic potential generated by fixed charges on a polarizable solute immersed in an ionic solution. This approach is often used in computational structural biology to estimate the electrostatic energetic component of the assembly of molecular biological systems. In the last decades, the amount of data concerning proteins and other biological macromolecules has remarkably increased. To fruitfully exploit these data, a huge computational power is needed as well as software tools capable of exploiting it. It is therefore necessary to move towards high performance computing and to develop proper parallel implementations of already existing and of novel algorithms. Nowadays, workstations can provide an amazing computational power: up to 10 TFLOPS on a single machine equipped with multiple CPUs and accelerators such as Intel Xeon Phi or GPU devices. The actual obstacle to the full exploitation of modern heterogeneous resources is efficient parallel coding and porting of software on such architectures. In this paper, we propose the implementation of a full Poisson-Boltzmann solver based on a finite-difference scheme using different and combined parallel schemes and in particular a mixed MPI-CUDA implementation. Results show great speedups when using the two schemes, achieving an 18.9x speedup using three GPUs. © 2014 José Colmenares et al.

Current workstations can offer really amazing raw computational power, in the order of TFlops on a single machine equipped with multiple CPUs and accelerators, which means less than half a dollar for a GFlop. Such result can only be achieved with a massive parallelism of the computational devices, but unfortunately not every application is able to fully exploit them. In this paper we analyze the performances of some widely used, computational intensive, applications, like FFT, convolution and n-body simulation, comparing the performance of a multi-core cluster node, with or without the contribution of GPUs. We aim to provide clear measure of the benefit of a heterogeneous architecture, in terms of time and cost, with a stress on the technology adopted at different levels of the software stack for the application parallelization.

The allocation of resources to jobs running on e-Science infrastructures is a key issue for scientific communities. In order to provide a better efficiency of computational jobs we propose an SLA-aware architecture. The core of this architecture is a scheduler relying on resource performance information. For performance characterization we propose a two-level benchmark that includes tests corresponding to specific e-Science applications. In order to evaluate the proposal we present simulation results for the proposed architecture.

Due to their complexity, the exploitation of Grid environments is not a trivial activity for many users, and a key factor is to enable a simplified and transparent orchestration of resources and jobs. Particularly critical is the deployment of matching procedures capable to effectively meet user's requirements with resources offer. We introduce GREEN a management tool primarily devoted to the matchmaking process, based on a performance characterization of both resources and job requirements. Leveraging on a two-level benchmarking methodology, GREEN allows users to express performance preference through an appropriate extension to Grid submission and description languages such as JSDL and Glue. Operating at intermediate level between applications and Grid middleware, GREEN reduces the gap between users' needs and available resources thus enabling a seamless exploitation of the Grid.

Image processing applications are computing demanding and since a long time much attention has been paid to the use of parallel processing. Emerging distributed and Grid based architectures represent new and well suited platforms that promise the availability of the required computational power. In this direction image processing has to evolve to heterogeneous environments, and a crucial aspect is represented by the interoperability and reuse of available and high performance code. This paper describes our experience in the development of PIMA(GE)2, Parallel IMAGE processing GEnoa server, obtained wrapping a library using the CORBA framework. Our aim is to obtain a high level of flexibility and dynamicity in the server architecture with a possible limited overhead. The design of a hierarchy of image processing operation objects and the development of the server interface are discussed.