Budget-aware scheduling algorithms for scientific workflows with stochastic task weights on heterogeneous IaaS Cloud platforms

Yves Caniou (1) , Eddy Caron (2, 1) , Aurélie Kong Win Chang (3) , Yves Robert (3)

1 AVALON - Algorithms and Software Architectures for Distributed and HPC Platforms
2 PaLaDiN - Parallel Large Data set in Network
3 ROMA - Optimisation des ressources : modèles, algorithmes et ordonnancement

This paper introduces several budget-aware algorithms to deploy scientific workflows on IaaS Cloud platforms, where users can request Virtual Machines (VMs) of different types, each with specific cost and speed parameters. We use a realistic application/platform model with stochastic task weights, and VMs communicating through a datacenter. We extend two well-known algorithms, MIN-MIN and HEFT, and make scheduling decisions based upon machine availability and available budget. During the mapping process, the budget-aware algorithms make conservative assumptions to avoid exceeding the initial budget; we further improve our results with refined versions that aim at rescheduling some tasks onto faster VMs, thereby spending any budget fraction leftover by the first allocation. These refined variants are much more time-consuming than the former algorithms, so there is a trade-off to find in terms of scalability. We report an extensive set of simulations with workflows from the Pegasus benchmark suite. Most of the time our budget-aware algorithms succeed in achieving efficient makespans while enforcing the given budget, despite (i) the uncertainty in task weights and (ii) the heterogeneity of VMs in both cost and speed values.

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Yves Caniou, Eddy Caron, Aurélie Kong Win Chang, Yves Robert. Budget-aware scheduling algorithms for scientific workflows with stochastic task weights on heterogeneous IaaS Cloud platforms. IPDPSW 2018 - IEEE International Parallel and Distributed Processing Symposium Workshops, May 2018, Vancouver, Canada. pp.15-26, ⟨10.1109/IPDPSW.2018.00014⟩. ⟨hal-01808831⟩