Few-shot classification without forgetting of event-camera data

Goyal, A and Biswas, S (2021) Few-shot classification without forgetting of event-camera data. In: 12th Indian Conference on Computer Vision, Graphics and Image Processing, ICVGIP 2021, 20-22 Dec 2021, Virtual, Online.

PDF ICVGIP_2021.pdf - Published Version Restricted to Registered users only Download (951kB) | Request a copy

Abstract

Event-based cameras can capture changes in brightness in the form of asynchronous events, unlike traditional cameras, which has sparked tremendous interest due to their wide range of applications. In this work, we address for the first time in literature, the task of few-shot classification of event data without forgetting the base classes on which it has been initially trained. This not only relaxes the constraint of data availability from all possible classes before the initial model is trained, but also the constraint of capturing large amounts of training data for each of the classes we want to classify. The proposed framework has three main stages: First, we train the base classifier by augmenting the original event data using a data mixing technique, so that the feature extractor can better generalize to unseen classes. We also utilize an adaptive semantic similarity between the classifier weights. This guarantees that the margin between similar classes is greater than that between dissimilar classes which in turn reduces confusion between similar classes. Second, weight imprinting is employed to learn the initial classifier weights for the new classes with few examples. Finally, we finetune the entire framework using a class-imbalance aware loss in an end-to-end manner. This is accomplished by converting the event data via a series of differentiable operations, which are then fed into our network. Extensive experiments on few-shot versions of two standard event-camera datasets justify the effectiveness of the proposed framework. We believe that this study will serve as a solid foundation for future work in this critical field. © 2021 ACM.

Item Type:	Conference Paper
Publication:	ACM International Conference Proceeding Series
Publisher:	Association for Computing Machinery
Additional Information:	The copyright for this article belongs to Association for Computing Machinery
Keywords:	Classification (of information); Semantics, Adaptive semantic similarity; Asynchronous event; Class imbalance; Class-imbalance aware loss; Event-based; Event-based camera; Mixup; Semantic similarity; Shot classification; Weight imprinting, Cameras
Department/Centre:	Division of Electrical Sciences > Electrical Communication Engineering
Date Deposited:	20 Jan 2022 06:54
Last Modified:	20 Jan 2022 06:54
URI:	http://eprints.iisc.ac.in/id/eprint/70997

Actions (login required)

View Item