Scientific Workflow Scheduling with Provenance Support in Multisite Cloud

Ji Liu (1, 2, 3) , Esther Pacitti (3, 1) , Patrick Valduriez (3, 1) , Marta Mattoso (4)

1 IBC - Institut de Biologie Computationnelle
2 MSR - INRIA - Microsoft Research - Inria Joint Centre
3 ZENITH - Scientific Data Management
4 COPPE-UFRJ - Instituto Alberto Luiz Coimbra de Pós-Graduação e Pesquisa de Engenharia

Recently, some Scientific Workflow Management Systems (SWfMSs) with provenance support (e.g. Chiron) have been deployed in the cloud. However, they typically use a single cloud site. In this paper, we consider a multisite cloud, where the data and computing resources are distributed at different sites (possibly in different regions). Based on a multisite architecture of SWfMS, i.e. multisite Chiron, we propose a multisite task scheduling algorithm that considers the time to generate provenance data. We performed an experimental evaluation of our algorithm using Microsoft Azure multisite cloud and two real-life scientific workflows, i.e. Buzz and Montage. The results show that our scheduling algorithm is up to 49.6% better than baseline algorithms in terms of execution time.

Mots clés: Scheduling; Multisite cloud; Parallel execution; Scientific workflow management system; Scientific workflow

Domaines: Base de données [cs.DB]

Ji Liu, Esther Pacitti, Patrick Valduriez, Marta Mattoso. Scientific Workflow Scheduling with Provenance Support in Multisite Cloud. VECPAR: Vector and Parallel Processing, Faculty of Engineering of the University of Porto, Portugal, Jun 2016, Porto, Portugal. pp.206-219, ⟨10.1007/978-3-319-61982-8_19⟩. ⟨lirmm-01342190⟩