Describing Textures in the Wild

Mircea Cimpoi (1) , Subhransu Maji (2) , Iasonas Kokkinos (3) , Sammy Mohamed (4) , Andrea Vedaldi (1)

1 Department of Engineering Science
2 Toyota Technological Institute
3 GALEN - Organ Modeling through Extraction, Representation and Understanding of Medical Image Content
4 SBU - Stony Brook University [SUNY]

Patterns and textures are key characteristics of many natural objects: a shirt can be striped, the wings of a butterfly can be veined, and the skin of an animal can be scaly. Aiming at supporting this dimension in image understanding, we address the problem of describing textures with semantic attributes. We identify a vocabulary of forty-seven texture terms and use them to describe a large dataset of patterns collected “in the wild”. The resulting Describable Textures Dataset (DTD) is a basis to seek the best representation for recognizing describable texture attributes in images. We port from object recognition to texture recognition the Improved Fisher Vector (IFV) and Deep Convolutional network Activation Features (DeCAF), and show that surprisingly, they both outperform specialized texture descriptors not only on our problem, but also in established material recognition datasets. We also show that our describable attributes are excellent texture descriptors, transferring between datasets and tasks; in particular, combined with IFV and DeCAF, they significantly outperform the state-of-the-art by more than 10% on both FMD and KTH-TIPS-2b benchmarks. We also demonstrate that they produce intuitive descriptions of materials and Internet images.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, Andrea Vedaldi. Describing Textures in the Wild. IEEE Conference in Computer Vision and Pattern Recognition, Jun 2014, Greater Columbus Convention Center in Columbus, Ohio., United States. pp.3606 - 3613, ⟨10.1109/CVPR.2014.461⟩. ⟨hal-01109284⟩