StyleGuide: Zero-Shot Sketch-Based Image Retrieval Using Style-Guided Image Generation

Dutta, T and Singh, A and Biswas, S (2021) StyleGuide: Zero-Shot Sketch-Based Image Retrieval Using Style-Guided Image Generation. In: IEEE Transactions on Multimedia, 23 . pp. 2833-2842.

PDF iee_tra_mul_23_2833-2842_2021.pdf - Published Version Restricted to Registered users only Download (3MB) | Request a copy

Abstract

The goal of zero-shot sketch-based image retrieval is to retrieve relevant images from a search set against a hand-drawn sketch query, which belongs to a class, previously unseen by the model. The knowledge gap between such unseen and seen classes along with the domain-gap between the query and search-set makes the problem extremely challenging. In this work, we address this problem by proposing a novel retrieval methodology, StyleGuide using style-guided fake-image generation. In addition, we further study the scenario of generalized zero-shot sketch-based image retrieval, where the search set contains images from both seen and unseen categories. Specifically, we propose a detection approach for unseen class samples in the search-set, based on pre-computed seen class-prototypes, to obtain a refined search-set for a particular unseen-class query. Thus, the query sketch needs to be compared only to those image data which are more likely to belong to the unseen classes, resulting in improved retrieval performance. Extensive experiments on two large-scale sketch-image datasets, Sketchy extended and TU-Berlin show that the proposed approach performs better or comparable to the state-of-the-art for ZS-SBIR and gives significant improvements over the state-of-the-art for generalized ZS-SBIR. © 1999-2012 IEEE.

Item Type:	Journal Article
Publication:	IEEE Transactions on Multimedia
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	Drawing (graphics); Image enhancement; Large dataset, Detection approach; Hand-drawn sketches; Image generations; Knowledge gaps; Query and search; Retrieval performance; Sketch-based image retrievals; State of the art, Image retrieval
Department/Centre:	Division of Electrical Sciences > Electrical Engineering
Date Deposited:	03 Dec 2021 06:51
Last Modified:	03 Dec 2021 06:51
URI:	http://eprints.iisc.ac.in/id/eprint/70103

Actions (login required)

View Item