Pyramid Scene Parsing Network in 3D: improving semantic segmentation of point clouds with multi-scale contextual information

Hao Fang (1) , Florent Lafarge (1)

1 TITANE - Geometric Modeling of 3D Environments

Analyzing and extracting geometric features from 3D data is a fundamental step in 3D scene understanding. Recent works demonstrated that deep learning archi-tectures can operate directly on raw point clouds, i.e. without the use of intermediate grid-like structures. These architectures are however not designed to encode contextual information in-between objects efficiently. Inspired by a global feature aggregation algorithm designed for images, we propose a 3D pyramid module to enrich pointwise features with multi-scale contextual information. Our module can be easily coupled with 3D semantic segmantation methods operating on 3D point clouds. We evaluated our method on three large scale datasets with four baseline models. Experimental results show that the use of enriched features brings significant improvements to the semantic segmentation of indoor and outdoor scenes.

Mots clés: Point Cloud; Semantic Segmentation; Deep Learning; Multi-scale Contextual Information

Domaines: Mathématique discrète [cs.DM]

Hao Fang, Florent Lafarge. Pyramid Scene Parsing Network in 3D: improving semantic segmentation of point clouds with multi-scale contextual information. ISPRS Journal of Photogrammetry and Remote Sensing, In press. ⟨hal-02159279⟩