IntPhys 2019: A Benchmark for Visual Intuitive Physics Understanding

Ronan Riochet (1, 2) , Mario Ynocente Castro (3) , Mathieu Bernard (2, 1) , Adam Lerer (4) , Rob Fergus (5) , Véronique Izard (6) , Emmanuel Dupoux (2, 1)

1 CoML - Apprentissage machine et développement cognitif
2 LSCP - Laboratoire de sciences cognitives et psycholinguistique
3 Preferred Networks
4 FAIR - Facebook AI Research [New York]
5 School of Computer and Information Security [University in South Australia]
6 INCC - UMR 8002 - Centre Neurosciences intégratives et Cognition / Integrative Neuroscience and Cognition Center

In order to reach human performance on complex visual tasks, artificial systems need to incorporate a significant amount of understanding of the world in terms of macroscopic objects, movements, forces, etc. Inspired by work on intuitive physics in infants, we propose an evaluation framework which diagnoses how much a given system understands about physics by testing whether it can tell apart well matched videos of possible versus impossible events. The test requires systems to compute a physical plausibility score over an entire video. It is free of bias and can test a range of specific physical reasoning skills. We then describe the first release of a benchmark dataset aimed at learning intuitive physics in an unsupervised way, using videos constructed with a game engine. We describe two Deep Neural Network baseline systems trained with a future frame prediction objective and tested on the possible versus impossible discrimination task. The analysis of their results compared to human data gives novel insights in the potentials and limitations of next frame prediction architectures.

Domaines: Intelligence artificielle [cs.AI]; Vision par ordinateur et reconnaissance de formes [cs.CV]

Ronan Riochet, Mario Ynocente Castro, Mathieu Bernard, Adam Lerer, Rob Fergus, et al.. IntPhys 2019: A Benchmark for Visual Intuitive Physics Understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, ⟨10.1109/TPAMI.2021.3083839⟩. ⟨hal-02274273v2⟩