Using Formal Grammars to Predict I/O Behaviors in HPC: the Omnisc'IO Approach

Matthieu Dorier (1, 2) , Shadi Ibrahim (3) , Gabriel Antoniu (3) , Robert Ross (2)

1 ENS Rennes - École normale supérieure - Rennes
2 MCS - Mathematics and Computer Science Division [ANL]
3 KerData - Scalable Storage for Clouds and Beyond

The increasing gap between the computation performance of post-petascale machines and the performance of their I/O subsystem has motivated many I/O optimizations including prefetching, caching, and scheduling. In order to further improve these techniques, modeling and predicting spatial and temporal I/O patterns of HPC applications as they run has become crucial. In this paper we present Omnisc'IO, an approach that builds a grammar-based model of the I/O behavior of HPC applications and uses it to predict when future I/O operations will occur, and where and how much data will be accessed. To infer grammars, Omnisc'IO is based on StarSequitur, a novel algorithm extending Nevill-Manning's Sequitur algorithm. Omnisc'IO is transparently integrated into the POSIX and MPI I/O stacks and does not require any modification in applications or higher-level I/O libraries. It works without any prior knowledge of the application and converges to accurate predictions of any N future I/O operations within a couple of iterations. Its implementation is efficient in both computation time and memory footprint.

Mots clés: HPC; I/O; Storage; Prediction; Grammar; Omnisc'IO

Domaines: Informatique; Calcul parallèle, distribué et partagé [cs.DC]; Réseaux et télécommunications [cs.NI]

Matthieu Dorier, Shadi Ibrahim, Gabriel Antoniu, Robert Ross. Using Formal Grammars to Predict I/O Behaviors in HPC: the Omnisc'IO Approach. IEEE Transactions on Parallel and Distributed Systems, 2016, ⟨10.1109/TPDS.2015.2485980⟩. ⟨hal-01238103⟩