Adaptive caching for data-intensive scientific workflows in the cloud

Heidsieck Gaëtan, De Oliveira Daniel, Pacitti Esther, Pradal Christophe, Tardieu François, Valduriez Patrick. 2019. Adaptive caching for data-intensive scientific workflows in the cloud. In : Database and expert systems applications. DEXA 2019. Hartmann Sven (ed.), Küng Josef (ed.), Chakravarthy Sharma (ed.), Anderst-Kotsis Gabriele (ed.), Tjoa A. Min (ed.), Khalil Ismail (ed.). Cham : Springer, 452-466. (Lecture Notes in Computer Science, 11707, 11707) ISBN 978-3-030-27617-1 International Conference on Database and Expert Systems Applications (DEXA 2019). 30, Linz, Autriche, 26 Août 2019/29 Août 2019.

https://doi.org/10.1007/978-3-030-27618-8_33

Communication avec actes

	Version post-print - Anglais Accès réservé aux personnels Cirad Utilisation soumise à autorisation de l'auteur ou du Cirad. DEXA_2019.pdf Télécharger (2MB) \| Demander une copie
	Version publiée - Anglais Accès réservé aux personnels Cirad Utilisation soumise à autorisation de l'auteur ou du Cirad. ID593357.pdf Télécharger (5MB) \| Demander une copie

Résumé : Many scientific experiments are now carried on using scientific workflows, which are becoming more and more data-intensive and complex. We consider the efficient execution of such workflows in the cloud. Since it is common for workflow users to reuse other workflows or data generated by other workflows, a promising approach for efficient workflow execution is to cache intermediate data and exploit it to avoid task re-execution. In this paper, we propose an adaptive caching solution for data-intensive workflows in the cloud. Our solution is based on a new scientific workflow management architecture that automatically manages the storage and reuse of intermediate data and adapts to the variations in task execution times and output data size. We evaluated our solution by implementing it in the OpenAlea system and performing extensive experiments on real data with a data-intensive application in plant phenotyping. The results show that adaptive caching can yield major performance gains, e.g., up to 120.16% with 6 workflow re-executions.

Mots-clés Agrovoc : système d'information, données, recherche, phénotype, informatique

Classification Agris : C30 - Documentation et information
U10 - Informatique, mathématiques et statistiques
F70 - Taxonomie végétale et phytogéographie

Auteurs et affiliations