Learning representations with strong supervision for image search

Mopuri, KR and Athreya, VB and Babu, RV (2019) Learning representations with strong supervision for image search. In: 12th International Conference on Signal Processing and Communications, SPCOM 2018, 16 - 19 July 2018, Bangalore, pp. 192-196.

PDF SPCOM_2018.pdf - Published Version Restricted to Registered users only Download (676kB) | Request a copy

Abstract

Deep learning exploits large volumes of labeled data to learn powerful models. When the target dataset is small, it is a common practice to perform transfer learning using pretrained models to learn new task specific representations. However, pretrained CNNs for image recognition are provided with limited information about the image during training, which is label alone. Tasks such as scene retrieval suffer from features learned from this weak supervision and require stronger supervision to better understand the contents of the image. In this paper, we exploit the features learned from caption generating models to learn novel task specific image representations. In particular, we consider the state-of-the-art captioning system Show and Tell 1 and the dense region description model DenseCap 2. We demonstrate that, owing to richer supervision provided during the process of training, the features learned by the captioning system perform better than those of CNNs. Further, we train a siamese network with a modified pair-wise loss to fuse the features learned by 1 and 2 and learn image representations suitable for retrieval. Experiments show that the proposed fusion exploits the complementary nature of the individual features and yields state-of-the art retrieval results on benchmark datasets.

Item Type:	Conference Paper
Publication:	SPCOM 2018 - 12th International Conference on Signal Processing and Communications
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	Deep learning; Image recognition, Benchmark datasets; Generating models; Image representations; Individual features; Limited information; Scene retrieval; State of the art; Transfer learning, Signal processing
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	08 Aug 2022 05:19
Last Modified:	08 Aug 2022 05:19
URI:	https://eprints.iisc.ac.in/id/eprint/75482

Actions (login required)

View Item