Using compositionality to understand parts in whole objects

Arun, SP (2022) Using compositionality to understand parts in whole objects. In: European Journal of Neuroscience .

PDF eur_jou_neu_2022.pdf - Published Version Restricted to Registered users only Download (2MB) | Request a copy

Abstract

A fundamental question for any visual system is whether its image representation can be understood in terms of its components. Decomposing any image into components is challenging because there are many possible decompositions with no common dictionary, and enumerating the components leads to a combinatorial explosion. Even in perception, many objects are readily seen as containing parts, but there are many exceptions. These exceptions include objects that are not perceived as containing parts, properties like symmetry that cannot be localized to any single part and special categories like words and faces whose perception is widely believed to be holistic. Here, I describe a novel approach we have used to address these issues and evaluate compositionality at the behavioural and neural levels. The key design principle is to create a large number of objects by combining a small number of pre-defined components in all possible ways. This allows for building component-based models that explain neural and behavioural responses to whole objects using a combination of these components. Importantly, any systematic error in model fits can be used to detect the presence of emergent or holistic properties. Using this approach, we have found that whole object representations are surprisingly predictable from their components, that some components are preferred to others in perception and that emergent properties can be discovered or explained using compositional models. Thus, compositionality is a powerful approach for understanding how whole objects relate to their parts.

Item Type:	Journal Article
Publication:	European Journal of Neuroscience
Publisher:	John Wiley and Sons Inc
Additional Information:	The copyright for this article belongs to the John Wiley and Sons Inc.
Keywords:	perception; recognition; review; systematic error; vision
Department/Centre:	Division of Biological Sciences > Centre for Neuroscience
Date Deposited:	04 Aug 2022 10:59
Last Modified:	04 Aug 2022 10:59
URI:	https://eprints.iisc.ac.in/id/eprint/75334

Actions (login required)

View Item