S-SBIR: Style augmented sketch based image retrieval

Dutta, T and Biswas, S (2020) S-SBIR: Style augmented sketch based image retrieval. In: 2020 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 1-5 March 2020, Snowmass Village, CO, USA, USA, pp. 3250-3259.

PDF IEEE_WIN_CON_APP_COM_VIS_3250-3259_2020.pdf - Published Version Restricted to Registered users only Download (1MB) | Request a copy

Abstract

Sketch-based image retrieval (SBIR) is gaining increasing popularity because of its flexibility to search natural images using unrestricted hand-drawn sketch query. Here, we address a related, but relatively unexplored problem, where the users can also specify their preferred styles of the images they want to retrieve, e.g., color, shape, etc., as keywords, whose information is not present in the sketch. The contribution of this work is three-fold. First, we propose a deep network for the problem of style-augmented SBIR (or s-SBIR) having three main components - category module, style module and mixer module, which are trained in an end-to-end manner. Second, we propose a quintuplet loss, which takes into consideration both the category and style, while giving appropriate importance to the two components. Third, we propose a normalized composite evaluation metric or ncMAP which can quantitatively evaluate s-SBIR approaches. Extensive experiments on subsets of two benchmark image-sketch datasets, Sketchy and TU-Berlin show the effectiveness of the proposed approach. © 2020 IEEE.

Item Type:	Conference Paper
Publication:	Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	cited By 0; Conference of 2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020 ; Conference Date: 1 March 2020 Through 5 March 2020; Conference Code:159803
Keywords:	Computer vision; Drawing (graphics), End to end; Evaluation metrics; Hand-drawn sketches; Natural images; Sketch-based image retrievals; Three folds; Two-component, Image retrieval
Department/Centre:	Division of Electrical Sciences > Electrical Engineering
Date Deposited:	16 Nov 2020 09:17
Last Modified:	16 Nov 2020 09:17
URI:	http://eprints.iisc.ac.in/id/eprint/65618

Actions (login required)

View Item