What Can Be Learnt from Engineering Safety Critical Partly-Autonomous Systems when Engineering Recommender Systems

Camille Fayollas (1) , Célia Martinie (1) , Philippe Palanque (1) , Eric Barboni (1) , Yannick Deleris (2)

1 IRIT-ICS - Interactive Critical Systems
2 AIRBUS Operations Ltd.

Human-Automation Design main target is to design systems in such a way that the couple system operator performs as efficiently as possible. Means for such designs include identifying functions (on the system side) and tasks(on the operator’s side) and balancing the allocation of tasks and functions between operators and the systems being operated. Allocating functions to the most suitable actor has been the early driver of function allocation [18]. The philosophy of recommender systems is that the system will provide a set of options for the users to select from. Such behavior can be easily connected to previous work on levels of automation as defined by Sheridan [34] and lessons can be drawn from putting together these two views. When these automations (including the one of recommender systems) are not adequately designed (or correctly understood by the operator), they may result in so called automation surprises [25, 32] that degrade, instead of enhance, the overall performance of the operations. This position paper identifies issues related to bringing recommender systems in the domain of safety critical interactive systems. While their advantages are clearly pointed out by their advocates, limitations are usually hidden or overlooked. We present this argumentation in the case of the ECAM (Electronic Centralised Aircraft Monitor) of which some behavior could be considered as similar to the one of a recommender system. We also highlight some engineering aspects of deploying recommender systems in the safety critical domain.

Mots clés: Performance; Operator tasks; Transparent automation; Recommender systems; Automation

Domaines: Génie logiciel [cs.SE]; Systèmes embarqués; Cryptographie et sécurité [cs.CR]; Interface homme-machine [cs.HC]; Architectures Matérielles [cs.AR]; Modélisation et simulation

Camille Fayollas, Célia Martinie, Philippe Palanque, Eric Barboni, Yannick Deleris. What Can Be Learnt from Engineering Safety Critical Partly-Autonomous Systems when Engineering Recommender Systems. Workshop on Engineering Computer-Human Interaction in Recommender Systems (EnCHIReS 2016), Jun 2016, Bruxelles, Belgium. pp.14-25. ⟨hal-02603671⟩