Skip to Main content Skip to Navigation
Conference papers

Towards Control of MapReduce Performance and Availability

Abstract : MapReduce is a popular programming model for distributed data processing and Big Data applications. Extensive research has been conducted either to improve the dependability or to increase performance of MapReduce, ranging from adaptive and on-demand fault-tolerance solutions, adaptive task scheduling techniques to optimized job execution mechanisms. This paper investigates a novel solution that controls MapReduce systems and provides guarantees in terms of both performance and availability, while reducing utilization costs. We follow a control theoretic approach for MapReduce cluster scaling and admission control. Preliminary results based on a simulation environment, previously validated on a real MapReduce cluster, show the effectiveness of the proposed control solutions for a Hadoop MapReduce cluster.
Complete list of metadata

Cited literature [7 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01316521
Contributor : Matthieu Roy Connect in order to contact the contributor
Submitted on : Tuesday, May 17, 2016 - 11:42:16 AM
Last modification on : Tuesday, October 19, 2021 - 11:26:18 AM
Long-term archiving on: : Thursday, August 18, 2016 - 10:10:14 AM

File

Towards Control of MapReduce P...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01316521, version 1

Citation

Sophie Cerf, Mihaly Berekmeri, Bogdan Robu, Nicolas Marchand, Sara Bouchenak. Towards Control of MapReduce Performance and Availability. DSN 2016 - 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, Jun 2016, Toulouse, France. ⟨hal-01316521⟩

Share

Metrics

Record views

792

Files downloads

383