Time-critical data management in clouds: Challenges and a Dynamic Real-Time Infrastructure Planner (DRIP) solution

Type Article
Date 2020-08
Language English
Author(s) Koulouzis Spiros1, Martin Paul1, Zhou Huan1, Hu Yang1, Wang Junchao1, Carval ThierryORCID2, Grenier Baptiste3, Heikkinen Jani4, de Laat Cees1, Zhao ZhimingORCID1
Affiliation(s) 1 : Univ Amsterdam, Inst Informat, Sci Pk 904, NL-1098 XH Amsterdam, Netherlands.
2 : IFREMER, Ctr Brest France, Brest, France.
3 : EGI Fdn, Amsterdam, Netherlands.
4 : CSC IT Ctr Sci Ltd, Espoo, Finland.
Source Concurrency And Computation-practice & Experience (1532-0626) (Wiley), 2020-08 , Vol. 32 , N. 16 , P. e5269 (17p.)
DOI 10.1002/cpe.5269
WOS© Times Cited 12
Note Special Issue: Special Issue on SDN‐based Wireless Network Virtualization (SDNWNV2018). Special Issue of the Workshop on Data‐Intensive Computing in the Clouds (DataCloud2017)
Keyword(s) Dynamic Real-Time Infrastructure Planner, e-infrastructure, research infrastructure, time-critical services, virtual research environment
Abstract

The increasing volume of data being produced, curated, and made available by research infrastructures in the environmental science domain require services that are able to optimize the delivery and staging of data for researchers and other users of scientific data. Specialized data services for managing data life cycle, for creating and delivering data products, and for customized data processing and analysis all play a crucial role in how these research infrastructures serve their communities, and many of these activities are time-critical-needing to be carried out frequently within specific time windows. We describe our experiences identifying the time-critical requirements of environmental scientists making use of computational research support environments. We present a microservice-based infrastructure optimization suite, the Dynamic Real-Time Infrastructure Planner, used for constructing virtual infrastructures for research applications on demand. We provide a case study whereby our suite is used to optimize runtime service quality for a data subscription service provided by the Euro-Argo using EGI Federated Cloud and EUDAT's B2SAFE services, and to consider how such a case study relates to other application scenarios.

Full Text
File Pages Size Access
Publisher's official version 17 4 MB Open access
Top of the page

How to cite 

Koulouzis Spiros, Martin Paul, Zhou Huan, Hu Yang, Wang Junchao, Carval Thierry, Grenier Baptiste, Heikkinen Jani, de Laat Cees, Zhao Zhiming (2020). Time-critical data management in clouds: Challenges and a Dynamic Real-Time Infrastructure Planner (DRIP) solution. Concurrency And Computation-practice & Experience, 32(16), e5269 (17p.). Publisher's official version : https://doi.org/10.1002/cpe.5269 , Open Access version : https://archimer.ifremer.fr/doc/00643/75557/