Search

In this paper, we describe the findings of the NEON project - a cross-Nordic - Sweden, Norway, Denmark, Finland and Iceland - project evaluating the usefulness of private versus public cloud services for HPC users. Our findings are briefly that private cloud technology is not mature enough yet to provide a transparent user experience. It is expected that this will be the case mid 2012. The cost efficiency of both public and private cloud should be continuously monitored as there is a strong downward trend. This conclusion is supported by NEON experimenting as well as larger initiatives e.g. StratusLab reports. Public cloud technology is mature enough but lacks certain features that will be necessary to include cloud resources in a transparent manner in a national infrastructure like the Norwegian NOTUR (www.notur.no) case, e.g. with respect to quota management. These features are expected to emerge in 2011 via third party management software and in the best of breed public cloud services. Public clouds are competitive in the low end for non-HPC jobs (low memory, low number of cores) on price. A significant fraction (ca. 20%) of the jobs running on the current Nordic supercomputer infrastructure are potentially suitable for cloud-like technology. This holds in particular for singlethreaded or single-node jobs with small/medium memory requirements and non-intensive I/O. There is a backlog of real supercomputer jobs that suffers from the non-HPC jobs on the supercomputer infrastructure. Off-loading these non-HPC jobs to a public cloud would effectively add supercomputing capacity. Another finding is that available storage capacity is not accessible in a user-friendly way; most storage clouds are only accessible via programmable interfaces. A number of experiments and piloting are presented to support these claims., QC 20120130

In the last couple of years, many e-Science infrastructures have begun to offer production services to e- Scientists with an increasing number of applications that require access to different kinds of computational resources. Within Europe two rather different multi-national e-Science infrastructures evolved over time namely Distributed European Infrastructure for Supercomputing Applications (DEISA) and Enabling Grids for E-SciencE (EGEE). DEISA provides access to massively parallel systems such as supercomputers that are well suited for scientific applications that require many interactions between their typically high numbers of CPUs. EGEE on the other hand provides access to a world-wide Grid of university clusters and PC pools that are well suited for farming applications that require less or even no interactions between the distributed CPUs. While DEISA uses the HPC-driven Grid technology UNICORE, EGEE is based on the gLite Grid middleware optimized for farming jobs. Both have less adoption of open standards and therefore both systems are technically non-interoperable, which means that no e-Scientist can easily leverage the DEISA and EGEE infrastructure with one suitable client environment for scientific applications. This paper argues that future interoperability of such large e-Science infrastructures is required to improve e-Science in general and to increase the real scientific impact of world-wide Grids in particular. We discuss the interoperability achieved by the OMII-Europe project that fundamentally improved the interoperability between UNICORE and gLite by using open standards. We also outline one specific scientific scenario of the WISDOM initiative that actually benefits from the recently established interoperability., QC 20150417

In a distributed Grid environment with ambitious service demands the job submission and management interfaces provide functionality of major importance. Emerging e-Science and Grid infrastructures such as EGEE and DEISA rely on highly available services that are capable of managing scientific jobs. It is the adoption of emerging open standard interfaces which allows the distribution of Grid resources in such a way that their actual service implementation or Grid technologies are not isolated from each other, especially when these resources are deployed in different e-Science infrastructures that consist of different types of computational resources. This paper motivates the interoperability of these infrastructures and discusses solutions. We describe the adoption of various open standards that recently emerged from the Open Grid Forum (OGF) in the field of job submission and management by well-known Grid technologies, respectively gLite and UNICORE. This has a fundamental impact on the interoperability between these technologies and thus within the next generation e-Science infrastructures that rely on these technologies., QC 20141106