Design and analysis of scheduling strategies for multi-CPU and multi-GPU architectures

Joao Vicente Ferreira Lima (1) , Thierry Gautier (1) , Vincent Danjean (1) , Bruno Raffin (1) , Nicolas Maillard (2)

1 MOAIS - PrograMming and scheduling design fOr Applications in Interactive Simulation
2 UFRGS - Instituto de Informática da UFRGS

In this paper, we present a comparison of scheduling strategies for heterogeneous multi-CPU and multi-GPU architectures. We designed and evaluated four scheduling strategies on top of XKaapi runtime: work stealing, data-aware work stealing, locality-aware work stealing, and Heterogeneous Earliest-Finish-Time (HEFT). On a heterogeneous architecture with 12 CPUs and 8 GPUs, we analysed our scheduling strategies with four benchmarks: a BLAS-1 AXPY vector operation, a Jacobi 2D iterative computation, and two linear algebra algorithms Cholesky and LU. We conclude that the use of work stealing may be efficient if task annotations are given along with a data locality strategy. Furthermore, our experimental results suggests that HEFT scheduling performs better on applications with very regular computations and low data locality.

Mots clés: Work stealing; Parallel programming; Accelerators; Task parallelism; Data-flow dependencies

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Joao Vicente Ferreira Lima, Thierry Gautier, Vincent Danjean, Bruno Raffin, Nicolas Maillard. Design and analysis of scheduling strategies for multi-CPU and multi-GPU architectures. Parallel Computing, 2015, 44, pp.37-52. ⟨10.1016/j.parco.2015.03.001⟩. ⟨hal-01132037⟩