Layer Decomposition: An Effective Structure-based Approach for Scientific Workflow Similarity

Johannes Starlinger (1) , Sarah Cohen-Boulakia (2, 3, 4, 5, 6) , Sanjeev Khanna (7) , Susan Davidson (7) , Ulf Leser (1)

1 Department of Computer Science [Berlin]
2 LRI - Laboratoire de Recherche en Informatique
3 AMIB - Algorithms and Models for Integrative Biology
4 IBC - Institut de Biologie Computationnelle
5 ZENITH - Scientific Data Management
6 VIRTUAL PLANTS - Modeling plant morphogenesis at different scales, from genes to phenotype
7 University of Pennsylvania

Scientific workflows have become a valuable tool for large-scale data processing and analysis. This has led to the creation of specialized online repositories to facilitate workflow sharing and reuse. Over time, these repositories have grown to sizes that call for advanced methods to support workflow discovery, in particular for effective similarity search. Here, we present a novel and intuitive workflow similarity measure that is based on layer decomposition. Layer decomposition accounts for the directed dataflow underlying scientific workflows, a property which has not been adequately considered in previous methods. We comparatively evaluate our algorithm using a gold standard for 24 query workflows from a repository of almost 1500 scientific workflows, and show that it a) delivers the best results for similarity search, b) has a much lower runtime than other, often highly complex competitors in structure-aware workflow comparison, and c) can be stacked easily with even faster, structure-agnostic approaches to further reduce runtime while retaining result quality.

Domaines: Bio-informatique [q-bio.QM]; Bio-Informatique, Biologie Systémique [q-bio.QM]

Johannes Starlinger, Sarah Cohen-Boulakia, Sanjeev Khanna, Susan Davidson, Ulf Leser. Layer Decomposition: An Effective Structure-based Approach for Scientific Workflow Similarity. International Conference on e-Science, Oct 2014, Guarujá, Brazil. pp.169-176, ⟨10.1109/eScience.2014.19⟩. ⟨hal-01066076⟩