ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

Are you from North or South India? A hard face-classification task reveals systematic representational differences between humans and machines

Katti, Harish and Arun, S P (2019) Are you from North or South India? A hard face-classification task reveals systematic representational differences between humans and machines. In: JOURNAL OF VISION, 19 (7).

jou_vis_19-7_2019.pdf - Published Version

Download (2MB) | Preview
Official URL: https://dx.doi.org/10.1167/19.7.1


We make a rich variety of judgments on faces, but the underlying features are poorly understood. Here we describe a challenging geographical-origin classification problem that elucidates feature representations in both humans and machine algorithms. In Experiment 1, we collected a diverse set of 1,647 faces from India labeled with their fine-grained geographical origin (North vs. South India), characterized the categorization performance of 129 human subjects on these faces, and compared this with the performance of machine vision algorithms. Our main finding is that while many machine algorithms achieved an overall performance comparable to that of humans (64%), their error patterns across faces were qualitatively different despite training. To elucidate the face parts used by humans for classification, we trained linear classifiers on overcomplete sets of features derived from each face part. This revealed mouth shape to be the most discriminative part compared to eyes, nose, or external contour. In Experiment 2, we confirmed that humans relied the most on mouth shape for classification using an additional experiment in which subjects classified faces with occluded parts. In Experiment 3, we compared human performance for briefly viewed faces and for inverted faces. Interestingly, human performance on inverted faces was predicted better by computational models compared to upright faces, suggesting that humans use relatively more generic features on inverted faces. Taken together, our results show that studying hard classification tasks can lead to useful insights into both machine and human vision.

Item Type: Journal Article
Additional Information: copyright for this article belongs to ASSOC RESEARCH VISION OPHTHALMOLOGY INC
Keywords: face categorization; ethnicity; computational models; deep convolutional networks; feature representation
Department/Centre: Division of Biological Sciences > Centre for Neuroscience
Date Deposited: 05 Aug 2019 10:11
Last Modified: 05 Aug 2019 10:11
URI: http://eprints.iisc.ac.in/id/eprint/63380

Actions (login required)

View Item View Item