Recognizing Internal States of Other Agents to Anticipate and Coordinate Interactions

Filipo Studzinski Perotto (1) , Filipo Studzinski Perotto

1 IRIT-SMAC - Systèmes Multi-Agents Coopératifs

In multi-agent systems, anticipating the behavior of other agents constitutes a difficult problem. In this paper we present the case where a cognitive agent is inserted into an unknown environment composed of different kinds of other objects and agents; our cognitive agent needs to incrementally learn a model of the environment dynamics, doing it only from its interaction experience; the learned model can then be used to define a policy of actions. It is relatively easy to do so when the agent interacts with static objects, with simple mobile objects, or with trivial reactive agents; however, when the agent deals with other complex agents that may change their behaviors according to some non-directly observable internal properties (like emotional or intentional states), the construction of a model becomes significantly harder. The complete system can be described as a Factored and Partially Observable Markov Decision Process (FPOMDP); our agent implements the Constructivist Anticipatory Learning Mechanism (CALM) algorithm, and the experiment (called mept) shows that the induction of non-observable variables enable the agent to learn a deterministic model of most of the system events (if it represents a well-structured universe), allowing it to anticipate other agents actions and to adapt to them, even if some interactions appear as non-deterministic in a first sight.

Mots clés: Factored Partially Observable Markov Decision Process (FPOMDP); Constructivist Learning Mechanisms; Anticipatory Learning; Model-Based RL

Domaines: Intelligence artificielle [cs.AI]

Filipo Studzinski Perotto, Filipo Studzinski Perotto. Recognizing Internal States of Other Agents to Anticipate and Coordinate Interactions. Multi-Agent Systems - 9th European Workshop, EUMAS 2011, Maastricht, The Netherlands, November 14-15, 2011. Revised Selected Papers, 2012, Berlin/Heidelberg, Georgia. pp.238--258, ⟨10.1007/978-3-642-34799-3_16⟩. ⟨hal-01762254⟩