Sparse weakly supervised models for object localization in road environment

Valentina Zadrija (1) , Josip Krapac (1) , Siniša Šegvić (1) , Jakob Verbeek (2)

1 FER - Faculty of Electrical Engineering and Computing [Zagreb]
2 Thoth - Apprentissage de modèles à partir de données massives

We propose a novel weakly supervised localization method based on Fisher-embedding of low-level features (CNN, SIFT), and model sparsity at the component level. Fisher-embedding provides an interesting alternative to raw low-level features, since it allows fast and accurate scoring of image subwindows with a model trained on entire images. Model sparsity reduces overfitting and enables fast evaluation. We also propose two new techniques for improving performance when our method is combined with nonlinear normalizations of the aggregated Fisher representation of the image. These techniques are i) intra-component metric normalization and ii) first-order approximation to the score of a normalized image representation. We evaluate our weakly supervised localization method on real traffic scenes acquired from driver's perspective. The method dramatically improves the localization AP over the dense non-normalized Fisher vector baseline (16 percentage points for zebra crossings, 21 percentage points for traffic signs) and leads to a huge gain in execution speed (91× for zebra crossings, 74× for traffic signs).

Mots clés: Geographic information system (GIS); Convolutional features; Sparse models; OpenStreetMap; Object localization; Weak supervision; Fisher Vectors

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Valentina Zadrija, Josip Krapac, Siniša Šegvić, Jakob Verbeek. Sparse weakly supervised models for object localization in road environment. Computer Vision and Image Understanding, 2018, 176-177, pp.9-21. ⟨10.1016/j.cviu.2018.10.004⟩. ⟨hal-01900418⟩