Probeless and Realistic Mixed Reality Application in Presence of Dynamic Light Sources

Salma Jiddi (1, 2) , Philippe Robert (1) , Eric Marchand (2) , Anthony Laurent (1) , Matthieu Fradet (1) , Pierrick Jouet (1) , Caroline Baillard (1)

1 Technicolor [Cesson Sévigné]
2 RAINBOW - Sensor-based and interactive robotics

In this work, we consider the challenge of achieving a coherent blending between real and virtual worlds in the context of a Mixed Reality (MR) scenario. Specifically, we have designed and implemented an interactive demonstrator that shows a realistic MR application without using any light probe. The proposed system takes as input the RGB stream of the real scene, and uses these data to recover both the position and intensity of light sources. The lighting can be static and/or dynamic and the geometry of the scene can be partially altered. Our system is robust in presence of specular effects and handles both uniform and/or textured surfaces.

Mots clés: Scene Analysis; Photometry; Modeling; Shadows; Textures; Lighting; Reflectance; Mixed Reality

Domaines: Informatique [cs]; Vision par ordinateur et reconnaissance de formes [cs.CV]

Salma Jiddi, Philippe Robert, Eric Marchand, Anthony Laurent, Matthieu Fradet, et al.. Probeless and Realistic Mixed Reality Application in Presence of Dynamic Light Sources. ISMAR 2018 - IEEE International Symposium on Mixed and Augmented Reality (demo session), Oct 2018, Munich, Germany. IEEE, pp.1-2. ⟨hal-01877124⟩