Semi-two-dimensional partitioning for parallel sparse matrix-vector multiplication

Enver Kayaaslan (1, 2) , Bora Uçar (2, 1) , Cevdet Aykanat (3)

1 LIP - Laboratoire de l'Informatique du Parallélisme
2 ROMA - Optimisation des ressources : modèles, algorithmes et ordonnancement
3 Bilkent-CS - Department of Computer Engineering

We propose a novel sparse matrix partitioning scheme, called semi-two-dimensional (s2D), for efficient paral-lelization of sparse matrix-vector multiply (SpMV) operations on distributed memory systems. In s2D, matrix nonzeros are more flexibly distributed among processors than one dimensional (rowwise or columnwise) partitioning schemes. Yet, there is a constraint which renders s2D less flexible than two-dimensional (nonzero based) partitioning schemes. The constraint is enforced to confine all communication operations in a single phase, as in 1D partition, in a parallel SpMV operation. In a positive view, s2D thus can be seen as being close to 2D partitions in terms of flexibility, and being close 1D partitions in terms of computation/communication organization. We describe two methods that take partitions on the input and output vectors of SpMV and produce s2D partitions while reducing the total communication volume. The first method obtains optimal total communication volume, while the second one heuristically reduces this quantity and takes computational load balance into account. We demonstrate that the proposed partitioning method improves the performance of parallel SpMV operations both in theory and practice with respect to 1D and 2D partitionings.

Domaines: Calcul parallèle, distribué et partagé [cs.DC]; Mathématique discrète [cs.DM]; Algorithme et structure de données [cs.DS]

Enver Kayaaslan, Bora Uçar, Cevdet Aykanat. Semi-two-dimensional partitioning for parallel sparse matrix-vector multiplication. PCO2015 (IPDPSW), May 2015, Hyderabad, India. pp.1125--1134. ⟨hal-01159692⟩