Physics-informed Guided Disentanglement in Generative Networks

Fabio Pizzati (1) , Pietro Cerri (2) , Raoul de Charette (1)

1 ASTRA - Systèmes de transport automatisés et sécurisés
2 VisLab

Image-to-image translation (i2i) networks suffer from entanglement effects in presence of physics-related phenomena in target domain (such as occlusions, fog, etc), thus lowering the translation quality and variability. In this paper, we present a comprehensive method for disentangling physics-based traits in the translation, guiding the learning process with neural or physical models. For the latter, we integrate adversarial estimation and genetic algorithms to correctly achieve disentanglement. The results show our approach dramatically increase performances in many challenging scenarios for image translation.

Mots clés: Adversarial learning; Image to image translation; Feature disentanglement; Adverse weather; Vision and rain; GAN

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Fabio Pizzati, Pietro Cerri, Raoul de Charette. Physics-informed Guided Disentanglement in Generative Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, ⟨10.1109/tpami.2023.3257486⟩. ⟨hal-03498130⟩