An O(n log n) Cutting Plane Algorithm for Structured Output Ranking

Matthew Blaschko (1, 2) , Arpit Mittal (3) , Esa Rahtu (4)

1 GALEN - Organ Modeling through Extraction, Representation and Understanding of Medical Image Content
2 CVN - Centre de vision numérique
3 Department of Engineering Science
4 MVG - Machine Vision Group

In this work, we consider ranking as a training strategy for structured output prediction. Recent work has begun to explore structured output prediction in the ranking setting, but has mostly focused on the special case of bipartite preference graphs. The bipartite special case is computationally efficient as there exists a linear time cutting plane training strategy for hinge loss bounded regularized risk, but it is unclear how to feasibly extend the approach to complete preference graphs. We develop here a highly parallelizable O(n log n) algorithm for cutting plane training with complete preference graphs that is scalable to millions of samples on a single core. We explore theoretically and empirically the relationship between slack rescaling and margin rescaling variants of the hinge loss bound to structured losses, showing that the slack rescaling variant has better stability properties and empirical performance with no additional computational cost per cutting plane iteration. We further show generalization bounds based on uniform convergence. Finally, we demonstrate the effectiveness of the proposed family of approaches on the problem of object detection in computer vision.

Domaines: Apprentissage [cs.LG]; Machine Learning [stat.ML]

Matthew Blaschko, Arpit Mittal, Esa Rahtu. An O(n log n) Cutting Plane Algorithm for Structured Output Ranking. German Conference on Pattern Recognition, Sep 2014, Münster, Germany. ⟨10.1007/978-3-319-11752-2_11⟩. ⟨hal-01020943⟩