Energy-based Models in Earth Observation: from Generation to Semi-supervised Learning

Javiera Castillo-Navarro (1, 2) , Bertrand Le Saux (3) , Alexandre Boulch (4) , Sébastien Lefèvre (2)

1 DTIS, ONERA [Salon-de-Provence]
2 OBELIX - Observation de l’environnement par imagerie complexe
3 ESA - Agence Spatiale Européenne = European Space Agency
4 Valeo.ai

Deep learning, together with the availability of large amounts of data, has transformed the way we process Earth observation (EO) tasks, such as land cover mapping or image registration. Yet, today, new models are needed to push further the revolution and enable new possibilities. This work focuses on a recent framework for generative modeling and explores its applicability to the EO images. The framework learns an energy-based model (EBM) to estimate the underlying joint distribution of the data and the categories, obtaining a neural network that is able to classify and synthesize images. On these two tasks, we show that EBMs reach comparable or better performances than convolutional networks on various public EO datasets and that they are naturally adapted to semisupervised settings, with very few labeled data. Moreover, models of this kind allow us to address high-potential applications, such as out-of-distribution analysis and land cover mapping with confidence estimation.

Mots clés: Deep learning; Energy-based models (EBMs); Generative models; Semisupervised learning

Domaines: Traitement des images [eess.IV]

Javiera Castillo-Navarro, Bertrand Le Saux, Alexandre Boulch, Sébastien Lefèvre. Energy-based Models in Earth Observation: from Generation to Semi-supervised Learning. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60, pp.5613211. ⟨10.1109/TGRS.2021.3126428⟩. ⟨hal-03379500⟩