Time-critical data management in clouds: Challenges and a Dynamic Real-Time Infrastructure Planner (DRIP) solution
|Author(s)||Koulouzis Spiros1, Martin Paul1, Zhou Huan1, Hu Yang1, Wang Junchao1, Carval Thierry2, Grenier Baptiste3, Heikkinen Jani4, de Laat Cees1, Zhao Zhiming1|
|Affiliation(s)||1 : Univ Amsterdam, Inst Informat, Sci Pk 904, NL-1098 XH Amsterdam, Netherlands.
2 : IFREMER, Ctr Brest France, Brest, France.
3 : EGI Fdn, Amsterdam, Netherlands.
4 : CSC IT Ctr Sci Ltd, Espoo, Finland.
|Source||Concurrency And Computation-practice & Experience (1532-0626) (Wiley), 2020-08 , Vol. 32 , N. 16 , P. e5269 (17p.)|
|WOS© Times Cited||23|
|Note||Special Issue: Special Issue on SDN‐based Wireless Network Virtualization (SDNWNV2018). Special Issue of the Workshop on Data‐Intensive Computing in the Clouds (DataCloud2017)|
|Keyword(s)||Dynamic Real-Time Infrastructure Planner, e-infrastructure, research infrastructure, time-critical services, virtual research environment|
The increasing volume of data being produced, curated, and made available by research infrastructures in the environmental science domain require services that are able to optimize the delivery and staging of data for researchers and other users of scientific data. Specialized data services for managing data life cycle, for creating and delivering data products, and for customized data processing and analysis all play a crucial role in how these research infrastructures serve their communities, and many of these activities are time-critical-needing to be carried out frequently within specific time windows. We describe our experiences identifying the time-critical requirements of environmental scientists making use of computational research support environments. We present a microservice-based infrastructure optimization suite, the Dynamic Real-Time Infrastructure Planner, used for constructing virtual infrastructures for research applications on demand. We provide a case study whereby our suite is used to optimize runtime service quality for a data subscription service provided by the Euro-Argo using EGI Federated Cloud and EUDAT's B2SAFE services, and to consider how such a case study relates to other application scenarios.