Search

This deliverable D2.7 describes the Blue Cloud architecture as it is known at Month 20. It is the second release of the architecture and an update of the earlier 1st release of the Blue-Cloud architecture document D2.6. It is expected that there will be further developments and refinements to the Blue Cloud system. For that purpose, one more release of the architecture document is planned, namely report D2.8 in Month 27. In order to make it easier for readers and reviewers, a table is included as part of Chapter 1, which indicates the elements and sections of this Deliverable 2.7, which have been updated or added in comparison with the earlier Deliverable 2.6. The technical framework of the pilot Blue-Cloud features: the Blue Cloud Data Discovery and Access service component to serve federated discovery and access to blue data infrastructures the Blue Cloud Virtual Research Environment (VRE) component to provide a Blue Cloud VRE as a federation of computing platforms and analytical services. In this report, the current architecture and functionalities of each of these components are described in detail as well as the roles of partners that are developing and hosting modules. The Blue Cloud Data Discovery and Access service architecture is based upon a combination of the DAB metadata broker service of CNR-IIA, and the SeaDataNet CDI service modules as developed by MARIS, IFREMER, and EUDAT in the framework of the EU SeaDataCloud project. For the Blue-Cloud Data Discovery and Access service and its modules, additional developments were needed such as adapting and upgrading of existing services, adding new services, testing modules, integrating modules, and testing the integrated service, in order to achieve the planned functionality. The Blue Cloud VRE is largely based upon the existing D4Science e-infrastructure as developed and managed by CNR-ISTI. This e-infrastructure from the start hosted already multiple Virtual Labs and offered a variety of services. These services have been adopted and adapted for the Blue Cloud and new services have been added, while also new Virtual Labs have been constructed and deployed as part of the Blue-Cloud Demonstrators. The D4Science e-infrastructure also already had proven solutions for connecting to external computing platforms and means for orchestrating distributed services, which are instrumental for smart connections to the other e-infrastructures in the Blue-Cloud system. The Blue Cloud demonstrators are developed as Virtual Labs embedded in the D4Science VRE e-infrastructure and supported by data input from the Blue Cloud Data Discovery and Access service and other data resources, and additional computing services. The demonstrators are worked out in a cooperation between WP3 and WP4, analysing their scientific workflows and technical set-up, and considering the D4Science VRE infrastructure and services that provide the basis platform. As part of their development, the demonstrators have required upgrading of existing functionality and development of additional services. This is described, where relevant, in this document. In addition, consideration is given to integration aspects, such as two-way linking between the 2 Blue Cloud components, and expanding the VRE with additional platforms for computing and algorithms, and where needed for specific demonstrators, direct access to data infrastructures. Moreover, aspects of authentication and monitoring are considered on full Blue Cloud scale. The Blue Cloud architecture as described in this report, is designed to be scalable and sustainable for near-future expansions, such as connecting additional blue data infrastructures, implementing more and advanced blue analytical services, configuring more dedicated Virtual Labs, and targeting more (groups of) users.

This report D2.6 describes the Blue Cloud architecture as it is known at Month 9. It is the first release of the architecture as it is expected that the actual development and implementation of the Blue Cloud system will provide further challenges and elaborated details. For that purpose, two more releases of the architecture document are planned, namely report D2.7 in Month 15 and report D2.8 in Month 27. The technical framework of the pilot Blue-Cloud will feature: the Blue Cloud Data Discovery and Access service component to serve federated discovery and access to blue data infrastructures the Blue Cloud Virtual Research Environment (VRE) component to provide a Blue Cloud VRE as a federation of computing platforms and analytical services. In this report, the architecture and functionalities of each of these components are described in detail as well as the roles of partners that will develop and host modules. The Blue Cloud Data Discovery and Access service architecture is based upon a combination of the GeoDab metadata broker service of CNR-IIA, and the SeaDataNet CDI service modules as developed by MARIS, IFREMER, and EUDAT in the framework of the EU SeaDataCloud project. For the Blue-Cloud Discovery and Access service and its modules, additional developments will be needed such as adapting and upgrading of existing services, adding new services, testing modules, integrating modules, and testing the integrated service, in order to achieve the planned functionality. The Blue Cloud VRE will be based upon the existing D4Science e-infrastructure as developed and managed by CNR-ISTI. This e-infrastructure already hosts multiple Virtual Labs and offers a variety of services, which can be adopted for the Blue Cloud. The D4Science e-infrastructure also has proven solutions for connecting to external computing platforms and means for orchestrating distributed services, which will be instrumental for smart connections to the other e-infrastructures in the Blue-Cloud system. The Blue Cloud demonstrators will be developed as Virtual Labs embedded in the D4Science VRE e-infrastructure and supported by data input from the Blue Cloud Data Discovery and Access service and other data resources, and possible additional computing services. The demonstrators are being worked out in a cooperation between WP3 and WP4, analysing their scientific workflows and technical set-up, and considering the present D4Science VRE infrastructure and services that will provide the basis platform. However, at this time of development, there is no insight yet whether the demonstrators will require upgrading of existing functionality or development of additional services. Therefore, the described architecture of the VRE concerns the current D4Science e-infrastructure. In addition, consideration is given to integration aspects, such as two-way linking between the 2 Blue Cloud components, and expanding the VRE with additional platforms for computing and D2.6Blue Cloud Architecture (Release 1) algorithms, and where needed for specific demonstrators, direct access to data infrastructures. Moreover, aspects of authentication and monitoring are considered on full Blue Cloud scale. The initial Blue Cloud architecture as described in this report, is designed to be scalable and sustainable for near-future expansions, such as connecting additional blue data infrastructures, implementing more and advanced blue analytical services, configuring more dedicated Virtual Labs, and targeting more (groups of) users.

The Blue Cloud data discovery and access service will be one of the components of the Blue-Cloud technical framework. It will serve federated discovery and access to blue data infrastructures and interact with the Blue-Cloud Virtual Research Environment(the component federating computing platforms and analytical services). The pilot Blue-Cloud project aims at federating initially in total 10blue data infrastructures. Each of these existing infrastructures have been described in this deliverable D2.1, in particular with a focus on their current data discovery and access mechanisms. From the descriptions and discussions, among others at the first TCom meeting in January 2020, it appears that a number of blue data infrastructures do not have to be federated with direct interfacing to the Blue-Cloud data discovery and access service, but indirectly as some are or will be coupled to another of the blue data infrastructures and that way it will be arranged that their contents will also be present in the Blue-Cloud data discovery and access service. In addition, activities are needed in several cases for providing API's which are suited for serving the planned Blue-Cloud data discovery and access service. In the following months a deeper analysis will be carried out, in order to detail technical specifications of the Blue-Cloud data discovery and access service, and developments required at and by each of the blue data infrastructures. This analysis, technical specification and workplan for the implementation and deployment of the Blue Cloud data discovery and access service will be documented in Deliverable D2.2 at M8. Thereafter, in the months till M17, the actual developments and deployment will take place for establishing an operational Blue-Cloud data discovery and access service by M17.