Search

The LEXIS (Large-scale EXecution for Industry & Society) Weather and Climate Pilot is developing a system for the prediction of water-related phenomena and their associated socioeconomic impacts. The system is based on multiple models chained together, as global weather models, high-resolution regional weather models, domain-specific application models (hydrological and forest fire risk forecasts), and impact models providing information (such as air quality and rainfall intensity) for key decisions and policymakers. This chapter describes the key aspect of this pilot in terms of serving model output data and products with cloud and high-performance data analytics environments, on top of a Weather Climate Data API, as well as the porting of models on the LEXIS infrastructure via different virtualization strategies (as virtual machine and dockers).

Extreme hydrometeorological events such as flash floods have caused considerable loss of life and damage to infrastructure over recent years. Flood events in the Mediterranean region between 1990 and 2006 caused over 4,500 fatalities and cost over EUR29 billion in damage, with Italy one of the worst affected countries. The Distributed Computing Infrastructure for Hydro-Meteorology (DRIHM) project is a European initiative aiming at providing an open, fully integrated eScience environment for predicting, managing, and mitigating the risks related to such extreme weather phenomena. Incorporating both modeled and observational data sources, it enables seamless access to a set of computing resources with the objective of providing a collection of services for performing experiments with numerical models in meteorology, hydrology, and hydraulics. The purpose of this article is to demonstrate how this flexible modeling architecture has been constructed using a set of standards including the NetCDF and WaterML2 file formats, in-memory coupling with OpenMI, controlled vocabularies such as CF Standard Names, ISO19139 metadata, and a Model MAP (Metadata, Adaptors, Portability) gateway concept for preparing numerical models for standardized use. Hydraulic results, including the impact to buildings and hazards to people, are given for the use cases of the severe and fatal flash floods, which occurred in Genoa, Italy in November 2011 and October 2014.

A full hydro-meteorological (HM) simulation, from rainfall to impact on urban areas, is a multidisciplinary activity which consists in the execution of a workflow composed by complex and heterogeneous model engines. Moreover an extensive set of configuration parameters have to be selected consistently among the models, otherwise the simulation can fail or produce unreliable results. The DRIHM portal is a Web-based science gateway aiming to support HM researchers in designing, executing and managing HM simulations. The first version of the portal was developed during the DRIHM project using the gUSE science gateway toolkit. The lesson we learned is guiding a refactoring process that, together with a review of the most relevant technologies for the development of a science gateway, represent the focus of this paper. Beside the technological aspects, the need of a strong interplay between ICT and other domain-specific communities clearly emerged, together with coherent policies in the management of data, computational resources and software components that represent the ecosystem of a science gateways.

The flexible and pay-as-you-go computing capabilities offered by Cloud infrastructures are very attractive for high-demanding e-Science applications like weather prediction simulators. For their ability to couple the scalability offered by public service provider with the greater control and customization provided by Private Clouds, Hybrid Clouds seem a particularly appealing solution to support meteorological researchers and weather departments in their every-day activity. Cloud Brokers interfacing customers with Cloud providers, may support scientists in the deployment and execution of demanding meteorological simulations, by hiding all the intricacies related to the management of powerful but often complex HPC systems. The paper presents a set of brokering strategies for Hybrid Clouds aimed at the execution of various instances of the weather prediction WRF model subject to different user requirements and computational conditions. A simulation-based analysis documents the performance of the different scheduling strategies at varying workloads and system configuration. © 2014 Elsevier B.V. All rights reserved.

Scientific image processing is a topic of interest for a broad scientific community since it is a mean of gaining understanding and insight into the data for a growing number of applications. Furthermore, the technological evolution permits large data acquisition, with sophisticated instruments, and their elaboration through complex multidisciplinary applications, resulting in datasets that are growing at an extremely rapid pace. This results in the need of huge computational power for the processing. It is necessary to move towards High Performance Computing (HPC) and to develop proper parallel implementations of image processing algorithms/operations. Modern HPC resources are typically highly heterogeneous systems, composed of multiple CPUs and accelerators such as Graphics Processing Units (GPUs) and Field-Programmable Gate Arrays (FPGAs). The actual barrier posed by heterogeneous HPC resources is the development and/or the performance efficient porting of software on such complex architectures. In this context, the aim of this work is to enable image processing on cluster of CPUs, through the use of PIMA(GE)(2) Lib, the Parallel IMAGE processing GEnoa Library. The library is able to exploit traditional clusters through MPI, GPU device through CUDA and a first experimentation is aimed to explore the use of CPU-clusters. Library operations are provided to the users through a sequential interface defined to hide the parallelism of the computation. The parallel computation, at each level, is managed employing specific policies designed to suitably coordinate the parallel processes/threads involved in the elaboration and their use is tightly coupled with the PIMA(GE)2 Lib interface. In this paper, we present the incremental approach adopted in the development of the library and the performance gains in each implementations: quite linear speedup is achieved on cluster architecture, about a 30% improvement in the execution time on a single GPU and the first results on cluster of GPUs are promising. (C) 2014 Elsevier B.V. All rights reserved.

The Poisson-Boltzmann equation models the electrostatic potential generated by fixed charges on a polarizable solute immersed in an ionic solution. This approach is often used in computational structural biology to estimate the electrostatic energetic component of the assembly of molecular biological systems. In the last decades, the amount of data concerning proteins and other biological macromolecules has remarkably increased. To fruitfully exploit these data, a huge computational power is needed as well as software tools capable of exploiting it. It is therefore necessary to move towards high performance computing and to develop proper parallel implementations of already existing and of novel algorithms. Nowadays, workstations can provide an amazing computational power: up to 10 TFLOPS on a single machine equipped with multiple CPUs and accelerators such as Intel Xeon Phi or GPU devices. The actual obstacle to the full exploitation of modern heterogeneous resources is efficient parallel coding and porting of software on such architectures. In this paper, we propose the implementation of a full Poisson-Boltzmann solver based on a finite-difference scheme using different and combined parallel schemes and in particular a mixed MPI-CUDA implementation. Results show great speedups when using the two schemes, achieving an 18.9x speedup using three GPUs. © 2014 José Colmenares et al.

Resources benchmarking is a promising though challenging opportunity for research communities to better exploit e-Science infrastructures. The paper investigates the suitability of an integrated two-level benchmarking approach in supporting resources ranking on a performance basis. The coupling of different benchmarking techniques (i.e. kernel and application benchmarks) is intended to grasp the behaviour of computational environments also with respect to real workloads so as to address a more suitable and efficient assignment of resources to the applications submitted by users. To confirm the appropriateness of our approach, an experimentation on a test bed was carried out which highlighted significant differences amongst the performance figures of the two levels of benchmarks. Some useful rules of thumb emerged from our experimentation that could be taken into account to fruitfully adopt this double-space approach to similar scenarios of use. © 2014 Inderscience Enterprises Ltd.

Current workstations can offer really amazing raw computational power, in the order of TFlops on a single machine equipped with multiple CPUs and accelerators, which means less than half a dollar for a GFlop. Such result can only be achieved with a massive parallelism of the computational devices, but unfortunately not every application is able to fully exploit them. In this paper we analyze the performances of some widely used, computational intensive, applications, like FFT, convolution and n-body simulation, comparing the performance of a multi-core cluster node, with or without the contribution of GPUs. We aim to provide clear measure of the benefit of a heterogeneous architecture, in terms of time and cost, with a stress on the technology adopted at different levels of the software stack for the application parallelization.

The distributed research infrastructure for hydrometeorology (DRIHM) project focuses on the development of an e-Science infrastructure to provide end-to-end hydro meteorological research (HMR) services (models, data, and post processing tools) by exploiting HPC, Grid and Cloud facilities. In particular, the DRIHM infrastructure supports the execution and analysis of high-resolution simulations through the definition of workflows composed by heterogeneous HMR models in a scalable and interoperable way, while hiding all the low level complexities. This contribution gives insights into best practices adopted to satisfy the requirements of an emerging multidisciplinary scientific community composed of earth and atmospheric scientists. To this end, DRIHM supplies innovative services leveraging high performance and distributed computing resources. Hydro meteorological requirements shape this IT infrastructure through an iterative "learning-by-doing" approach that permits tight interactions between the application community and computer scientists, leading to the development of a flexible, extensible, and interoperable framework.

Hybrid Clouds seems able to offer their customers with differentiate solutions capable of providing more and personalized guarantees with respect to the basic service availability generally supplied. In the context of an Italian research project aimed to transfer ICT advancements from research centers towards ICT SMEs, the paper focuses on the design of a brokering tool for hybrid clouds capable to adequately respond to specific Quality of Service (QoS) constraints. Aimed at satisfying the highest number of user requests while trying maximizing the profit of the private provider, in the context of a posted price economic model, the proposed brokering algorithm may apply different allocation policies, based on the reservation of a quota of private resources to high-level QoS applications.

Hybrid Clouds couple the scalability offered by public Clouds with the greater control supplied by private ones. A (hybrid) Cloud broker acting as an intermediary between users and providers of public Cloud services, may support customers in the selection of the most suitable offers, optionally adding the provisioning of dedicated services with higher levels of quality. The paper presents a Cloud brokering algorithm delivering services with different level of non-functional requirements, to the private or public resources, on the basis of different scheduling criteria. With the objective of maximize user satisfaction and broker’s revenues, the algorithm pursues profit increases by reducing energy costs, through the adoption of energy saving mechanisms. A simulation model is used to evaluate performance in terms of broker’s revenue, user satisfaction and energy behavior of various allocation policies. Simulation results show that differences among policies depend on system loads and that the use of turn on and off techniques greatly improves energy savings at low and medium load rates.

The allocation of jobs to a set of heterogeneous resources, as in common e-Science environments, is a key issue for their efficient use. To reach a better coupling and improve execution times of applications, a consolidated though challenging opportunity is the benchmarking of resources. In this direction, we propose to exploit two complementary approaches, the micro and application-driven benchmarks, which allow to grasp the behavior of computational contexts with respect to more realistic workloads and to lead to a more efficient assignment of the submitted jobs to resources. To evaluate the appropriateness of our methodology an experimentation on a test bed was carried-out. A first comparison of benchmarking results highlighted differences in performance amongst the two levels of benchmarks. Moreover the appropriateness of adopting benchmarking information to rank resources was also confirmed by examining different allocation strategies through a queuing network simulator.

The allocation of resources to jobs running on e-Science infrastructures is a key issue for scientific communities. In order to provide a better efficiency of computational jobs we propose an SLA-aware architecture. The core of this architecture is a scheduler relying on resource performance information. For performance characterization we propose a two-level benchmark that includes tests corresponding to specific e-Science applications. In order to evaluate the proposal we present simulation results for the proposed architecture.

Due to their complexity, the exploitation of Grid environments is not a trivial activity for many users, and a key factor is to enable a simplified and transparent orchestration of resources and jobs. Particularly critical is the deployment of matching procedures capable to effectively meet user's requirements with resources offer. We introduce GREEN a management tool primarily devoted to the matchmaking process, based on a performance characterization of both resources and job requirements. Leveraging on a two-level benchmarking methodology, GREEN allows users to express performance preference through an appropriate extension to Grid submission and description languages such as JSDL and Glue. Operating at intermediate level between applications and Grid middleware, GREEN reduces the gap between users' needs and available resources thus enabling a seamless exploitation of the Grid.

The grid represents a great opportunity in the scientific community to solve data/compute intensive problems and to share data. Using the grid, different application domains could achieve great advantages; however, the exploitation of the grid is not a trivial pursuit for many users. The key issue is to enable a simplified use of grid resources and tools that scientists usually employ. In this paper, we investigate these topics in bioinformatics community to allow the efficient elaboration on the grid of images obtained through the tissue microarray technique. We present a grid framework for tissue microarray analysis, GF4TMA, that allows the selection of TMA images and their efficient analysis by the exploitation of the grid architecture. In particular, we exploited PIMA(GE)2 Lib, the Parallel IMAGE processing GEnoa Library. A critical point was to enable parallel computations on the grid without compromising the efficiency and the user-friendliness of the native library. We tackled this issue by encapsulating the library in a grid service.

Image processing applications are computing demanding and since a long time much attention has been paid to the use of parallel processing. Emerging distributed and Grid based architectures represent new and well suited platforms that promise the availability of the required computational power. In this direction image processing has to evolve to heterogeneous environments, and a crucial aspect is represented by the interoperability and reuse of available and high performance code. This paper describes our experience in the development of PIMA(GE)2, Parallel IMAGE processing GEnoa server, obtained wrapping a library using the CORBA framework. Our aim is to obtain a high level of flexibility and dynamicity in the server architecture with a possible limited overhead. The design of a hierarchy of image processing operation objects and the development of the server interface are discussed.