Multi-scale superpixel neighborhoods for contextual features for land cover mapping with high resolution satellite image time series
Résumé
Contextual features are known to be key descriptors for distinguishing certain land cover classes, for instance, different levels of urban density. Contextual information can be extracted from a fixed-size neighborhood, but this often comes at the cost of an undesired smoothing of high spatial frequency areas, inducing the loss of linear objects and the rounding of sharp corners. In standard Object Based Image Analysis, the use of objects instead of pixels as the basic unit for classification offers higher-level features while maintaining the high-frequency geometry, but also faces certain issues. Segmentation is aimed at extracting homogeneous objects, but the relevant contextual information can come from the combination of very diverse pixels in the neighborhood. These pixels will rarely be included in the same segment, meaning that the objects in a classical OBIA framework are limited in terms of the contextual features they can offer. The other issue is the difficulty of these methods to simultaneously include both short and long range information, which can be important because the discriminative context for some classes can be at several different scales. To face these issues, superpixels place themselves in between the pixel based and the object based methods. They offer neighborhoods that are adaptive to the image content, while maintaining the possibility of including diverse pixels in the same segment by imposing size and compacity constraints on the segments. This method is suited for the extraction of multi-scale information, because the size of a superpixel neighborhood is directly controllable. In other words, a feature at a given scale represents a context within a certain range, which makes these features comparable with each other. Feature comparability is an important property for the classifier. In this paper, a novel method will be presented, in which each pixel is described by a series of features calculated in the superpixels containing it, at several different scales. The focus of the paper will be the comparison of different contextual features in the frame of high resolution image time series as the ones provided by Sentinel-2, where a pixel is described by 10 spectral bands with acquisitions every 5 days. These features range from the most basic, such as mean and variance, to more complex ones, based on spatial autocorrelation, covariance matrices, and semantic contextual information. The different features will be compared on their ability to improve classification, while maintaining spatial accuracy, meaning no loss of high spatial frequencies. The impact of the the number of scales, their sizes, and the presence of pixel information will also be evaluated. Experiments on a Sentinel-2 time series on a land cover classification problem with 14 classes including urban, agricultural, and natural cover classes are used to show the differences in behavior and performance of these different features. Overall, it is shown that including contextual features increases classification performance for both superpixel and fixed-size neighborhoods, when compared to traditional OBIA methods. It is also shown that superpixel neighborhoods don't smooth high frequency areas, compared to fixed size neighborhoods.