A workflow-inspired, modular and robust approach to experiments in distributed systems

Tomasz Buchert (1) , Lucas Nussbaum (1) , Jens Gustedt (1, 2)

1 ALGORILLE - Algorithms for the Grid
2 ICube - Laboratoire des sciences de l'ingénieur, de l'informatique et de l'imagerie

Experimentation in large-scale distributed systems research is very challenging due to the size and complexity of modern systems and applications spanning domains of high performance computing, P2P networks, cloud computing, etc. Some obstacles that each researcher must face are: the difficulty of properly structuring experiments due to their complexity, the inflexibility of existing methodologies and tools and the scalability problems resulting from the size of studied systems. In this paper, we propose a novel method of representing and executing experiments that solves these problems. To this end, we present an interdisciplinary approach to the control of large-scale experiments in distributed systems research that draws its foundations from workflow management and scientific workflows. This workflow-inspired approach distinguishes itself by its representation of experiments, modular architecture and robust error handling. We show how the aforementioned problems are solved by our approach in an exemplary performance study of an HTTP server.

Mots clés: experimentation; control of experiments; workflows; large-scale distributed systems; performance analysis

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Tomasz Buchert, Lucas Nussbaum, Jens Gustedt. A workflow-inspired, modular and robust approach to experiments in distributed systems. CCGRID - 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, May 2014, Chicago, United States. ⟨hal-00909347v5⟩