Principles of user-centered online reinforcement learning for the emergence of service compositions

Walid Younes (1) , Sylvie Trouilhet (1) , Françoise Adreit (1, 2) , Jean-Paul Arcangeli (1, 3)

1 IRIT-SMAC - Systèmes Multi-Agents Coopératifs
2 UT2J - Université Toulouse - Jean Jaurès
3 UT3 - Université Toulouse III - Paul Sabatier

Cyber-physical and ambient systems surround the human user with services at her/his disposal. These services, which are more or less complex, must be as tailored as possible to her/his preferences and the current situation. We propose to build them automatically and on the fly by composition of more elementary services present at the time in the environment, without prior expression of the user's needs nor specification of a process or a composition model. In a context of high dynamic variability of both the ambient environment and the needs, the user must be involved at the minimum. In order to produce the knowledge necessary for automatic composition in the absence of an initial guideline, we have developed a generic solution based on online reinforcement learning. % at infinite horizon. It is decentralized within a multi-agent system in charge of the administration and composition of the services, which learns incrementally from and for the user. Thus, our architecture puts the user in the loop. It relies on an interaction protocol between agents that supports service discovery and selection in an open and unstable environment.

Mots clés: online learning; reinforcement learning; user feedback; service discovery; selection and composition; multi-agent system; ambient intelligence

Domaines: Système multi-agents [cs.MA]; Intelligence artificielle [cs.AI]

Walid Younes, Sylvie Trouilhet, Françoise Adreit, Jean-Paul Arcangeli. Principles of user-centered online reinforcement learning for the emergence of service compositions. [Research Report] IRIT/RR–2019–05–FR, IRIT : Institut de Recherche Informatique de Toulouse. 2019. ⟨hal-02976638⟩