Hardware Accelerator for Capsule Network based Reinforcement Learning

Ram, D and Panwar, S and Varghese, K (2022) Hardware Accelerator for Capsule Network based Reinforcement Learning. In: 35th International Conference on VLSI Design, VLSID 2022 - held concurrently with 2022 21st International Conference on Embedded Systems, ES 2022, 26 February - 2 March 2022, Virtual, Online, pp. 162-167.

PDF VLSID_2022.pdf - Published Version Restricted to Registered users only Download (1MB) | Request a copy

Abstract

Convolutional neural networks are widely used in reinforcement learning. Capsule networks are gaining popularity over the traditional convolutional neural networks in many classification tasks. A capsule is a multidimensional activity vector consisting of neurons that represent the features of a specific type of entity such as an object or part of an object. In this paper, we explore the capability of a Capsule Network for deep reinforcement learning-based applications. Our proposed capsule network architecture with the same number of parameters as that of a convolutional neural network for reinforcement learning takes on average nine times lesser number of network update iterations than that of a convolutional neural network. We also propose a hardware accelerator for deep Q-learning that uses the capsule network as a deep Q-network instead of a convolutional neural network. We have implemented a capsule network-based deep Q-learning architecture for inference on the Xilinx Kintex UltraScale field-programmable gate array. We have tested the network on pygame based environments. Our hardware implementation achieves an overall speedup of 77.45x as compared to the software implementation of the capsule network for deep reinforcement learning on Intel Xeon CPU E5-1607, 4 core, @3.1GHz and 10.86x as compared to implementation on Nvidia Ge-Force GTX1080 GPU.

Item Type:	Conference Paper
Publication:	Proceedings - 2022 35th International Conference on VLSI Design, VLSID 2022 - held concurrently with 2022 21st International Conference on Embedded Systems, ES 2022
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to the Institute of Electrical and Electronics Engineers Inc.
Keywords:	Convolution; Convolutional neural networks; Deep learning; Field programmable gate arrays (FPGA); Learning systems; Network architecture, Classification tasks; Convolutional neural network; Field programmables; Hardware accelerators; Learning architectures; N/a; Network-based; Programmable gate array; Q-learning; Reinforcement learnings, Reinforcement learning
Department/Centre:	Division of Electrical Sciences > Electronic Systems Engineering (Formerly Centre for Electronic Design & Technology)
Date Deposited:	04 Nov 2022 07:13
Last Modified:	04 Nov 2022 07:13
URI:	https://eprints.iisc.ac.in/id/eprint/77678

Actions (login required)

View Item