Katti, Harish and Arun, S P (2019) Are you from North or South India? A hard face-classification task reveals systematic representational differences between humans and machines. In: JOURNAL OF VISION, 19 (7).
|
PDF
jou_vis_19-7_2019.pdf - Published Version Download (2MB) | Preview |
Abstract
We make a rich variety of judgments on faces, but the underlying features are poorly understood. Here we describe a challenging geographical-origin classification problem that elucidates feature representations in both humans and machine algorithms. In Experiment 1, we collected a diverse set of 1,647 faces from India labeled with their fine-grained geographical origin (North vs. South India), characterized the categorization performance of 129 human subjects on these faces, and compared this with the performance of machine vision algorithms. Our main finding is that while many machine algorithms achieved an overall performance comparable to that of humans (64%), their error patterns across faces were qualitatively different despite training. To elucidate the face parts used by humans for classification, we trained linear classifiers on overcomplete sets of features derived from each face part. This revealed mouth shape to be the most discriminative part compared to eyes, nose, or external contour. In Experiment 2, we confirmed that humans relied the most on mouth shape for classification using an additional experiment in which subjects classified faces with occluded parts. In Experiment 3, we compared human performance for briefly viewed faces and for inverted faces. Interestingly, human performance on inverted faces was predicted better by computational models compared to upright faces, suggesting that humans use relatively more generic features on inverted faces. Taken together, our results show that studying hard classification tasks can lead to useful insights into both machine and human vision.
| Item Type: | Journal Article |
|---|---|
| Publication: | JOURNAL OF VISION |
| Publisher: | ASSOC RESEARCH VISION OPHTHALMOLOGY INC |
| Additional Information: | copyright for this article belongs to ASSOC RESEARCH VISION OPHTHALMOLOGY INC |
| Keywords: | face categorization; ethnicity; computational models; deep convolutional networks; feature representation |
| Department/Centre: | Division of Biological Sciences > Centre for Neuroscience |
| Date Deposited: | 05 Aug 2019 10:11 |
| Last Modified: | 05 Aug 2019 10:11 |
| URI: | http://eprints.iisc.ac.in/id/eprint/63380 |
Actions (login required)
 |
View Item |
