Sarvadevabhatla, Ravi Kiran and Venkatraman, Shanthakumar and Venkatesh Babu, R (2017) ‘Part’ly First Among Equals: Semantic Part-Based Benchmarking for State-of-the-Art Object Recognition Systems. In: 13th Asian Conference on Computer Vision, ACCV 2016, November 20-24, 2016, Taipei, Taiwan, pp. 181-197.
Full text not available from this repository.Abstract
An examination of object recognition challenge leaderboards (ILSVRC, PASCAL-VOC) reveals that the top-performing classifiers typically exhibit small differences amongst themselves in terms of error rate/mAP. To better differentiate the top performers, additional criteria are required. Moreover, the (test) images, on which the performance scores are based, predominantly contain fully visible objects. Therefore, ‘harder’ test images, mimicking the challenging conditions (e.g. occlusion) in which humans routinely recognize objects, need to be utilized for benchmarking. To address the concerns mentioned above, we make two contributions. First, we systematically vary the level of local objectpart content, global detail and spatial context in images from PASCAL VOC 2010 to create a new benchmarking dataset dubbed PPSS-12. Second, we propose an object-part based benchmarking procedure which quantifies classifiers’ robustness to a range of visibility and contextual settings. The benchmarking procedure relies on a semantic similarity measure that naturally addresses potential semantic granularity differences between the category labels in training and test datasets, thus eliminating manual mapping. We use our procedure on the PPSS-12 dataset to benchmark top-performing classifiers trained on the ILSVRC-2012 dataset. Our results show that the proposed benchmarking procedure enables additional differentiation among state-of-the-art object classifiers in terms of their ability to handle missing content and insufficient object detail. Given this capability for additional differentiation, our approach can potentially supplement existing benchmarking procedures used in object recognition challenge leaderboards.
| Item Type: | Conference Paper |
|---|---|
| Publisher: | Springer Verlag |
| Additional Information: | The Copyright of this article belongs to the Springer |
| Keywords: | Classification (of information); Computer vision; Object recognition; Semantics; Error rate; Manual mapping; Object recognition systems; Semantic parts; Semantic similarity measures; Spatial context; State of the art; Test images; Benchmarking |
| Department/Centre: | Division of Interdisciplinary Sciences > Computational and Data Sciences |
| Date Deposited: | 29 May 2022 06:18 |
| Last Modified: | 31 May 2022 04:54 |
| URI: | https://eprints.iisc.ac.in/id/eprint/72590 |
Actions (login required)
 |
View Item |
