What Makes Paris Look Like Paris?

Carl Doersch (1) , Saurabh Singh (1) , Abhinav Gupta (1) , Josef Sivic (2, 3) , Alexei Efros (4)

1 CMU - Carnegie Mellon University [Pittsburgh]
2 LIENS - Laboratoire d'informatique de l'école normale supérieure
3 WILLOW - Models of visual object recognition and scene understanding
4 LBNL - Lawrence Berkeley National Laboratory [Berkeley]

Given a large repository of geo-tagged imagery, we seek to automatically find visual elements, for example windows, balconies, and street signs, that are most distinctive for a certain geo-spatial area, for example the city of Paris. This is a tremendously difficult task as the visual features distinguishing architectural elements of different places can be very subtle. In addition, we face a hard search problem: given all possible patches in all images, which of them are both frequently occurring and geographically informative? To address these issues, we propose to use a discriminative clustering approach able to take into account the weak geographic supervision. We show that geographically representative image elements can be discovered automatically from Google Street View imagery in a discriminative manner. We demonstrate that these elements are visually interpretable and perceptually geo-informative. The discovered visual elements can also support a variety of computational geography tasks, such as mapping architectural correspondences and influences within and across cities, finding representative elements at different geo-spatial scales, and geographically informed image retrieval.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, Alexei Efros. What Makes Paris Look Like Paris?. Communications of the ACM, 2015, 58 (12), pp.103-110. ⟨10.1145/2830541⟩. ⟨hal-01248528⟩