Agritrop
Home

Adaptive caching for data-intensive scientific workflows in the cloud

Heidsieck Gaetan, De Oliveira Daniel, Pacitti Esther, Pradal Christophe, Tardieu François, Valduriez Patrick. 2019. Adaptive caching for data-intensive scientific workflows in the cloud. In : Database and expert systems applications. DEXA 2019. Hartmann Sven (ed.), Küng Josef (ed.), Chakravarthy Sharma (ed.), Anderst-Kotsis Gabriele (ed.), Tjoa A. Min (ed.), Khalil Ismail (ed.). Cham : Springer, pp. 452-466. (Lecture Notes in Computer Science, 11707, 11707) ISBN 978-3-030-27617-1 International Conference on Database and Expert Systems Applications (DEXA 2019). 30, Linz, Autriche, 26 August 2019/29 August 2019.

Paper with proceedings
[img] Post-print version - Anglais
Access restricted to CIRAD agents
Use under authorization by the author or CIRAD.
DEXA_2019.pdf

Télécharger (2MB) | Request a copy
[img] Published version - Anglais
Access restricted to CIRAD agents
Use under authorization by the author or CIRAD.
ID593357.pdf

Télécharger (5MB) | Request a copy

Abstract : Many scientific experiments are now carried on using scientific workflows, which are becoming more and more data-intensive and complex. We consider the efficient execution of such workflows in the cloud. Since it is common for workflow users to reuse other workflows or data generated by other workflows, a promising approach for efficient workflow execution is to cache intermediate data and exploit it to avoid task re-execution. In this paper, we propose an adaptive caching solution for data-intensive workflows in the cloud. Our solution is based on a new scientific workflow management architecture that automatically manages the storage and reuse of intermediate data and adapts to the variations in task execution times and output data size. We evaluated our solution by implementing it in the OpenAlea system and performing extensive experiments on real data with a data-intensive application in plant phenotyping. The results show that adaptive caching can yield major performance gains, e.g., up to 120.16% with 6 workflow re-executions.

Mots-clés Agrovoc : Système d'information, Données, Recherche, Phénotype, Informatique

Classification Agris : C30 - Documentation and information
U10 - Computer science, mathematics and statistics
F70 - Plant taxonomy and geography

Auteurs et affiliations

  • Heidsieck Gaetan, INRIA (FRA)
  • De Oliveira Daniel, UFF (BRA)
  • Pacitti Esther, Université de Montpellier (FRA)
  • Pradal Christophe, CIRAD-BIOS-UMR AGAP (FRA) ORCID: 0000-0002-2555-761X
  • Tardieu François, INRA (FRA)
  • Valduriez Patrick, INRIA (FRA)

Autres liens de la publication

Source : Cirad-Agritrop (https://agritrop.cirad.fr/593357/)

View Item (staff only) View Item (staff only)

[ Page générée et mise en cache le 2021-02-05 ]