N-HAR: A neuromorphic event-based human activity recognition system using memory surfaces

Pradhan, BR and Bethi, Y and Narayanan, S and Chakraborty, A and Thakur, CS (2019) N-HAR: A neuromorphic event-based human activity recognition system using memory surfaces. In: 2019 IEEE International Symposium on Circuits and Systems, ISCAS 2019, 26 - 29 May 2019, Sapporo.

PDF ISCAS_pra_2019.pdf - Published Version Restricted to Registered users only Download (706kB) | Request a copy

Abstract

In recent years, a new generation of low-power, neuromorphic, event-based vision sensors has been gaining popularity for their very low latency and data sparsity. Though the conventional frame-based cameras have advanced in a lot of ways, they suffer from data redundancy and temporal latency. The bio-inspired artificial retinas eliminate the data redundancy by capturing only the change in illumination at each pixel and asynchronously communicating in binary spikes. In this work, we propose a system to achieve the task of human activity recognition based on the event-based camera data. We show that such tasks, which generally need high frame rate sensors for accurate predictions, can be achieved by adapting existing computer vision techniques to the spiking domain. We used event memory surfaces to make the sparse event data compatible with deep convolutional neural networks (CNNs). We leverage upon the recent advances in deep convolutional networks based video analysis and adapt such frameworks onto the neuromorphic domain. We also provide the community with a new dataset consisting of five categories of human activities captured in real world without any simulations. We achieved an accuracy of 94.3 using event memory surfaces on our activity recognition dataset.

Item Type:	Conference Paper
Publication:	Proceedings - IEEE International Symposium on Circuits and Systems
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	Cameras; Convolution; Deep neural networks; Neural networks; Redundancy, Accurate prediction; Activity recognition; Computer vision techniques; Convolutional networks; Convolutional neural network; Event based vision sensors; Human activity recognition; Human activity recognition systems, Pattern recognition
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	04 Dec 2022 05:18
Last Modified:	04 Dec 2022 05:18
URI:	https://eprints.iisc.ac.in/id/eprint/77948

Actions (login required)

View Item