An Optimal Model for Optimizing the Placement and Parallelism of Data Stream Processing Applications on Cloud-Edge Computing

Felipe Rodrigo de Souza (1) , Marcos Dias de Assuncao (2) , Eddy Caron (1) , Alexandre da Silva Veith (3)

1 LIP - Laboratoire de l'Informatique du Parallélisme
2 AVALON - Algorithms and Software Architectures for Distributed and HPC Platforms
3 University of Toronto at Scarborough

The Internet of Things has enabled many application scenarios where a large number of connected devices generate unbounded streams of data, often processed by data stream processing frameworks deployed in the cloud. Edge computing enables offloading processing from the cloud and placing it close to where the data is generated, thereby reducing the time to process data events and deployment costs. However, edge resources are more computationally constrained than their cloud counterparts, raising two interrelated issues, namely deciding on the parallelism of processing tasks (a.k.a. operators) and their mapping onto available resources. In this work, we formulate the scenario of operator placement and parallelism as an optimal mixed-integer linear programming problem. The proposed model is termed as Cloud-Edge data Stream Placement (CESP). Experimental results using discrete-event simulation demonstrate that CESP can achieve an end-to-end latency at least 80% and monetary costs at least 30% better than traditional cloud deployment.

Mots clés: Data Stream Processing; Operator Placement; Operator Parallelism; End-to-end Latency; Edge Computing

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Felipe Rodrigo de Souza, Marcos Dias de Assuncao, Eddy Caron, Alexandre da Silva Veith. An Optimal Model for Optimizing the Placement and Parallelism of Data Stream Processing Applications on Cloud-Edge Computing. SBAC-PAD 2020 - IEEE 32nd International Symposium on Computer Architecture and High Performance Computing, Sep 2020, Porto, Portugal. ⟨hal-02926459⟩