ExBLAS: Reproducible and Accurate BLAS Library

Roman Iakymchuk (1, 2) , Stef Graillat (2) , Caroline Collange (3) , David Defour (4)

1 ISCD - Institut des Sciences du Calcul et des Données
2 PEQUAN - Performance et Qualité des Algorithmes Numériques
3 ALF - Amdahl's Law is Forever
4 DALI - Digits, Architectures et Logiciels Informatiques

Due to non-associativity of floating-point operations and dynamic scheduling on parallel architectures, getting a bitwise reproducible floating-point result for multiple executions of the same code on different or even similar parallel architectures is challenging. We address the problem of reproducibility in the context of fundamental linear algebra operations — like the ones included in the Basic Linear Algebra Subprograms (BLAS) library — and propose algorithms that yields both reproducible and accurate (rounding to the nearest) results. We present implementations of these reproducible and accurate algorithms for the BLAS routines in parallel environments such as Intel server CPUs, Intel Xeon Phi, and both NVIDIA and AMD GPUs. We show that the performance of our implementations is comparable to the standard ones.

Domaines: Calcul parallèle, distribué et partagé [cs.DC]; Arithmétique des ordinateurs

Roman Iakymchuk, Stef Graillat, Caroline Collange, David Defour. ExBLAS: Reproducible and Accurate BLAS Library. RAIM: Rencontres Arithmétiques de l’Informatique Mathématique, Apr 2015, Rennes, France. , 7ème Rencontre Arithmétique de l'Informatique Mathématique, 2015. ⟨hal-01140280⟩