An experimental survey on big data frameworks (Highlight paper)

Wissem Inoubli (1) , Sabeur Aridhi (2) , Haithem Mezni (3) , Mondher Maddouri (4) , Engelbert Mephu Nguifo (5)

1 UTM - Université de Tunis El Manar
2 CAPSID - Computational Algorithms for Protein Structures and Interactions
3 UJ - Université de Jendouba
4 Taibah University
5 LIMOS - Laboratoire d'Informatique, de Modélisation et d'optimisation des Systèmes

Recently, increasingly large amounts of data are generated from a variety of sources. Existing data processing technologies are not suitable to cope with the huge amounts of generated data. Yet, many research works focus on Big Data, a buzzword referring to the processing of massive volumes of (unstructured) data. Recently proposed frameworks for Big Data applications help to store, analyze and process the data. In this paper, we discuss the challenges of Big Data and we survey existing Big Data frameworks. We also present an experimental evaluation and a comparative study of the most popular Big Data frameworks with several representative batch.

Mots clés: Big data; MapReduce; Hadoop; HDFS; Spark; Flink; Storm; Samza; Batch/stream processing

Domaines: Base de données [cs.DB]; Calcul parallèle, distribué et partagé [cs.DC]

Wissem Inoubli, Sabeur Aridhi, Haithem Mezni, Mondher Maddouri, Engelbert Mephu Nguifo. An experimental survey on big data frameworks (Highlight paper). BDA 2018 - 34ème Conférence sur la Gestion de Données – Principes, Technologies et Applications, Oct 2018, Bucarest, Romania. ⟨hal-02014797⟩