Search

Accurate and comprehensive storage of provenance information is a basic requirement for modern scientific computing. A significant effort in recent years has developed robust theories and standards for the representation of these traces across a variety of execution platforms. Whilst these are necessary to enable repeatability they do not exploit the captured information to its full potential. This data is increasingly being captured from applications hosted on Cloud Computing platforms, which offer large scale computing resources without significant up front costs. Medical applications, which generate large datasets are also suited to cloud computing as the practicalities of storing and processing such data locally are becoming increasingly challenging. This paper shows how provenance can be captured from medical applications, stored using a graph database and then used to answer audit questions and enable repeatability. This static provenance will then be combined with performance data to predict future workloads, inform decision makers and reduce latency. Finally, cost models which are based on real world cloud computing costs will be used to determine optimum strategies for data retention over potentially extended periods of time.

In this article, we critically examine the role of semantic technology in data driven analysis. We explain why learning from data is more than just analyzing data, including also a number of essential synthetic parts that suggest a revision of George Boxs model of data analysis in statistics. We review arguments from statistical learning under uncertainty, workflow reproducibility, as well as from philosophy of science, and propose an alternative, synthetic learning model that takes into account semantic conflicts, observation, biased model and data selection, as well as interpretation into background knowledge. The model highlights and clarifies the different roles that semantic technology may have in fostering reproduction and reuse of data analysis across communities of practice under the conditions of informational uncertainty. We also investigate the role of semantic technology in current analysis and workflow tools, compare it with the requirements of our model, and conclude with a roadmap of 8 challenging research problems which currently seem largely unaddressed. We explain why learning from data is more than just analyzing data, including synthetic tasks.We provide arguments from statistical learning, workflow reproducibility, and philosophy.We propose a learning model that highlights the roles of semantic technology in data analysis.Based on this model, we review current analysis and workflow tools and Semantic Web research.We propose a roadmap of 8 challenging research problems which currently seem largely unaddressed.

This paper explores the potential for improving the performance of e-Science applications on commercial clouds through the detailed examination, and characterization, of the underlying cloud network using network tomography. Commercial cloud providers are increasingly offering high performance and GPU-enabled resources that are ideal for many e-Science applications. However, the opacity of the cloud's internal network, while a necessity for elasticity, limits the options for e-Science programmers to build efficient and high performance codes. We introduce health indicators, markers, metrics, and score as part of a network health system that provides a model for describing the overall network health of an e-Science application. We then explore the suitability of a range of tomographic techniques to act as health indicators using two testbeds-the second of which spanned one hundred AWS instances. Finally, we evaluate our work using a real-world medical image reconstruction application. We identify and characterize network performance in commercial clouds.An overall health system is constructed using tomographic probes to establish and compare an instance's network performance.We deploy the health system over a testbed of 100 AWS instances and explore its ability to scale.We apply the health system to a medical imaging e-Science application and demonstrate performance benefits.

This special issue contains extended versions of many of the top papers from the 10th IEEE International eScience Conference (eScience 2014), held in October 2014 in Guaruja, Brazil. The authors of strongly reviewed papers published in that conference were invited to extend their papers, which then went through a second peer review. This special issue contains the first set (seven) of the extended papers selected as the result of this review. Part 2, with three additional extended papers, will follow in a future issue of FGCS. eScience requires simultaneous advances in infrastructure and applications.Such advances are happening rapidly, and have been presented at the 2014 IEEE International Conference on eScience.This special issue highlights some of these advances, and the science they enable.

Our research is focused on developing an ecoinformatics platform to support climate change adaptation in Victoria. A multi-disciplinary, cross-organisational approach was taken in developing a platform of collaboration to support the understanding of climate change impact and the formulation of adaptation strategies.The platform comprises a number of components including: (i) a metadata discovery tool to support modelling, (ii) a workflow framework for connecting climate change models, (iii) geographical visualisation tools for communicating landscape and farm impacts, (iv) a landscape object library for storing and sharing digital objects, (v) a landscape constructor tool to support participatory decision-making, and (vi) an online collaboration space for supporting multi-disciplinary research and cross-organisational collaboration.In this paper we present the platform as it has been developed to support collaborative research and to inform stakeholders of the likely impacts of climate change in southwest Victoria, Australia. We discuss some of the drivers for research in developing the ecoinformatics platform and its components. We conclude by identifying some future research directions in better connecting researchers and communicating scientific outcomes in the context of climate change impact and adaptation. Highlights? VREs are useful to support climate change adaptation research. ? Visualisation is a powerful front-end to climate and biophysical models. ? Challenges remain in e-Science in supporting (near) real-time decision making. ? Data-driven' science is identified with a movement towards 'open science'. ? e-Science needs to consider both soft (people) and hard (technology) systems.

Scientific applications require large computing power, traditionally exceeding the amount that is available within the premises of a single institution. Therefore, clouds can be used to provide extra resources whenever required. For this vision to be achieved, however, requires both policies defining when and how cloud resources are allocated to applications and a platform implementing not only these policies but also the whole software stack supporting management of applications and resources. Aneka is a cloud application platform capable of provisioning resources obtained from a variety of sources, including private and public clouds, clusters, grids, and desktops grids. In this paper, we present Aneka's deadline-driven provisioning mechanism, which is responsible for supporting quality of service (QoS)-aware execution of scientific applications in hybrid clouds composed of resources obtained from a variety of sources. Experimental results evaluating such a mechanism show that Aneka is able to efficiently allocate resources from different sources in order to reduce application execution times.

In the business world, Service Oriented Architecture (SOA) has recently gained popularity as a new approach to integrate business applications. In a similar way, scientific workflows can accomplish the automation of large-scale e-Science applications. However, the use of workflows in scientific environments differs from that in business environments. Scientific workflows need to handle large amounts of data, deal with heterogeneous and distributed resources such as the Grid, and require specialized software applications that are written in diverse programming languages, most of which are not popular in business environments. In this paper, we analyze the preparedness and the shortcomings of the SOA paradigm in addressing the needs of e-Science and the extent to which this can be done. The paper identifies the characteristics of a Virtual Organization providing scientific services, and presents a model placing particular emphasis on BPEL processes as a mean for supporting the interaction with Web Services. We discuss the challenges encountered in the seamless integration of BPEL processes within an e-Science infrastructure and we propose a set of complementary infrastructural services. By providing business utilities and automation technology founded on Web Services, infrastructural services cooperate with BPEL in ensuring on-demand resource provisioning for the execution of scientific workflows, while addressing some critical issues such as security, access control and monitoring. Furthermore, the paper presents our experience in adopting the proposed approach within a collaborative environment for bioinformatics. To illustrate how a scientific experiment can be formalized and executed, we focus on micro-array data processing, a field that will be increasingly common in applications of machine learning to molecular biology.

This special issue contains extensions of work presented at the 2012 IEEE International Conference on eScience, held in Chicago, IL, in October 2012. For eScience to be successful, simultaneous advances in infrastructure and applications are required, which in turn requires research teams with widely varying expertise, from computer science to Earth sciences and biological sciences, as well as fora for different teams to interact and share knowledge and lessons learned, such as the eScience series of conferences. The papers in this special issue represent advances in both infrastructure and applications and how they can influence each other.

E-science projects of various disciplines generate large amounts of data and face a fundamental challenge: thousands of researchers want to obtain new scientific results by logically relating subsets of the total volume of data. Considering the huge and widely distributed amounts of data, e-science communities investigate different technologies to provide fast access to the growing data sets. Among these technologies, Peer-to-Peer (P2P) and Data Grid are two models that fit these requirements well, because of their potential to provide a high quality of service with low cost. In this paper, we explore the possibility of using the P2P paradigm for data-intensive e-science applications on the Grid. We argue that additional support is required to achieve fast access to the huge and widely distributed amounts of data and propose eSciGrid to overcome the scalability barriers in today's e-science communities. eSciGrid allows e-science communities to achieve a high query throughput through a decentralized protocol which integrates caching with query processing. The protocol takes into account the physical distance between peers and the amount of traffic carried by each node. The result of this integration is constant complexity for moderate queries and fast data transfers between Grid peers. Our results show that eSciGrid increases the performance of data access on e-science Grids.

International audience; Grid computing has been the subject of many large national and international IT projects. However, not all goals of these projects have been achieved. In particular, the number of users lags behind the initial forecasts laid out by proponents of grid technologies. This underachievement may have led to claims that the grid concept as a whole is on its way to being replaced by Cloud computing and various X-as-a-Service approaches. In this paper, we try to analyze the current situation and to identify promising directions for future grid development. Although there are shortcomings in current grid systems, we are convinced that the concept as a whole remains valid and can benefit from new developments, including Cloud computing. Furthermore, we strongly believe that some future applications will require the grid approach and that, as a result, further research is required in order to turn this concept into reliable, efficient and user-friendly computing platforms.

Today many significant scientific discoveries are achieved through complex and distributed scientific computations that are structured and represented as scientific workflows. Although atomicity is a well studied topic in transaction processing and business workflows, such an important capability needs to be revisited in a scientific workflow environment. Firstly, the semantics of atomicity needs to be defined in a dataflow-oriented scientific workflow model, particularly for pipelined execution of hierarchical scientific workflows. Secondly, in a scientific workflow environment, atomic regions are specified or inferred dynamically as needed and are committed implicitly, which are in contrast to a priori well-defined transaction boundaries and explicit commits in transaction processing and business workflows. Finally, although atomicity and provenance are related to each other, their interactions and relationships have never been explored in the literature. In this paper, we propose: (i) an architecture for scientific workflow management systems that supports both provenance and atomicity; (ii) a dataflow-oriented atomicity model that supports the notions of commit and abort; and (iii) a dataflow-oriented provenance model that supports querying and visualizing provenance.

Scientific workflow systems have become a necessary tool for many applications, enabling the composition and execution of complex analysis on distributed resources. Today there are many workflow systems, often with overlapping functionality. A key issue for potential users of workflow systems is the need to be able to compare the capabilities of the various available tools. There can be confusion about system functionality and the tools are often selected without a proper functional analysis. In this paper we extract a taxonomy of features from the way scientists make use of existing workflow systems and we illustrate this feature set by providing some examples taken from existing workflow systems. The taxonomy provides end users with a mechanism by which they can assess the suitability of workflow in general and how they might use these features to make an informed choice about which workflow system would be a good choice for their particular application.

This article offers an analysis of UK arts and humanities e-Science practices in order to identify current trends. It also considers challenges of how arts and humanities disciplines fit into the overall e-Science agenda. We will discuss a first phase of early experimentation projects in 2007 and continue with a second phase since 2007, which more systematically investigates methodologies and technologies that could provide answers to grand challenges in digital arts and humanities research.

E-science projects of various disciplines face a fundamental challenge: thousands of users want to obtain new scientific results by application-specific and dynamic correlation of data from globally distributed sources. Considering the involved enormous and exponentially growing data volumes, centralized data management reaches its limits. Since scientific data are often highly skewed and exploration tasks exhibit a large degree of spatial locality, we propose the locality-aware allocation of data objects onto a distributed network of interoperating databases. HiSbase is an approach to data management in scientific federated Data Grids that addresses the scalability issue by combining established techniques of database research in the field of spatial data structures (quadtrees), histograms, and parallel databases with the scalable resource sharing and load balancing capabilities of decentralized Peer-to-Peer (P2P) networks. The proposed combination constitutes a complementary e-science infrastructure enabling load balancing and increased query throughput.

As the practice of science moves beyond the single investigator due to the complexity of the problems that now dominate science, large collaborative and multi-institutional teams are needed to address these problems. In order to support this shift in science, the computing and data-handling infrastructure that is essential to most of modern science must also change in order to support this increased complexity. This is the goal of computing and data Grids: software infrastructure that facilitates solving large-scale problems by providing the mechanisms to access, aggregate, and manage the computer network-based infrastructure of science. This infrastructure includes computing systems, data archive systems, scientific instruments, and computer-mediated human collaborations. This paper examines several large-scale science problems, their requirements for computing and data Grid infrastructure, and the current approaches to providing the necessary functionality.

This paper describes the £ 120M UK 'e-Science' (http://www.research-councils.ac.uk/and http://www.escience-grid.org.uk) initiative and begins by defining what is meant by the term e-Science. The majority of the £ 120M, some £ 75M, is funding large-scale e-Science pilot projects in many areas of science and engineering. The infrastructure needed to support such projects must permit routine sharing of distributed and heterogeneous computational and data resources as well as supporting effective collaboration between groups of scientists. Such an infrastructure is commonly referred to as the Grid. Apart from £ 10M towards a Teraflop computer, the remaining funds, some £ 35M, constitute the e-Science 'Core Programme'. The goal of this Core Programme is to advance the development of robust and generic Grid middleware in collaboration with industry. The key elements of the Core Programme will be outlined including details of a UK e-Science Grid testbed. The pilot e-Science projects that have so far been announced are then briefly described. These projects span a range of disciplines from particle physics and astronomy to engineering and healthcare, and illustrate the breadth of the UK e-Science Programme. In addition to these major e-Science projects, the Core Programme is funding a series of short-term e-Science demonstrators across a number of disciplines as well as projects in network traffic engineering and some international collaborative activities. We conclude with some remarks about the need to develop a data architecture for the Grid that will allow federated access to relational databases as well as flat files.

Rapid development of Internet and Grid technologies has greatly extended the number of resources which scientists can apply for research. More and more scientific advances are achieved via experiments which consist of large volumes of computing tasks and data. Scientific Workflow Management Systems (SWMS) are a key technology to integrate computing and data analysis components, and to control the execution and logical sequences among them. By hiding the complexity in an underlying infrastructure, SWMSs allow scientists to design complex scientific experiments, access geographically distributed data files, and execute the experiments using computing resources at multiple organizations. In this way, domain scientists can effectively use available resources while focusing on the logic of the experiments instead of low level technical details. During the past few years, scientific workflow systems and applications have attracted enormous research interest. Fig. 1 shows an increasing number of workflow related publications in the journal of Future Generation Computer Systems (FGCS). A number of research foci can be found in recent publications. Running legacy applications on the Grid is essential for the domain scientists to extend execution scenarios of their experiments. Wrapping components in a scientific workflow using a standardized architecture, such as services, improves the reusability of the components and also makes the integration easy. McGough et al. [1] present a service architecture named GridSAM to manage the job submission onto Grid and tomanage different abstractions of tasks at runtime. Glatard et al. [2] demonstrate how services can be dynamically grouped and optimized in the workflow enactment and scheduling at runtime. Interoperability between different workflows is essential to realizing cooperation between different scientific experiments. Describing distributed Grid resources using semantic web technologies promotes high-level sharing of components and