Parallelization of Scientific Workflows in the Cloud

Ji Liu (1, 2, 3) , Esther Pacitti (3, 4) , Patrick Valduriez (1, 3, 4) , Marta Mattoso (5)

1 MSR - INRIA - Microsoft Research - Inria Joint Centre
2 LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier
3 ZENITH - Scientific Data Management
4 IBC - Institut de Biologie Computationnelle
5 COPPE-UFRJ - Instituto Alberto Luiz Coimbra de Pós-Graduação e Pesquisa de Engenharia

Nowadays, more and more scientific experiments need to handle massive amounts of data. Their data processing consists of multiple computational steps and dependencies within them. A data-intensive scientific workflow is an appropriate tool for modeling such process. Since the execution of data-intensive scientific workflows requires large-scale computing and storage resources, a cloud environment, which provides virtually infinite resources is appealing. However, because of the general geographical distribution of scientific groups collaborating in the experiments, multisite management of data-intensive scientific workflows in the cloud is becoming an important problem. This paper presents a general study of the current state of the art of data-intensive scientific workflow execution in the cloud and corresponding multisite management techniques.

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Ji Liu, Esther Pacitti, Patrick Valduriez, Marta Mattoso. Parallelization of Scientific Workflows in the Cloud. [Research Report] RR-8565, INRIA. 2014. ⟨hal-01024101v2⟩