Powering Monitoring Analytics with ELK stack

Abdelkader Lahmadi (1) , Frédéric Beck (2)

1 MADYNES - Management of dynamic networks and services
2 Inria Nancy - Grand Est

Machine-generated data, including logs and network flows, are considerably growing and their collection, searching, and visualization is a challenging task for (a) daily administrator activities and (b) researchers aiming to better find out analytics and insights from monitoring data regarding their research goals, including amongst others security or modeling of network and systems. This tutorial introduces the open source ELK stack and its components, including Elasticsearch for deep search and data analytics, Logstash for centralized logging, log enrichment, and parsing, and Kibana for powerful and beautiful data visualizations. ELK enables the analysis and visualization of monitoring data, such as logs and netflows. The first part of the tutorial details these individual components. The second part provides guidelines for the deployment and configuration of ELK components. In the third part participants will perform hands-on practical work for collecting, processing, and enriching logs and netflows, combined with the creation of associated visualization and dashboards aspects.

Mots clés: Data analytics; Elasticsearch; Logstash; Kibana; Monitoring data

Domaines: Réseaux et télécommunications [cs.NI]; Technologies Émergeantes [cs.ET]

Abdelkader Lahmadi, Frédéric Beck. Powering Monitoring Analytics with ELK stack. 9th International Conference on Autonomous Infrastructure, Management and Security (AIMS 2015), Jun 2015, Ghent, Belgium. 9th International Conference on Autonomous Infrastructure, Management and Security (AIMS 2015), 2015. ⟨hal-01212015⟩