Dukkipati, A and Banerjee, R and Ayyagari, RS and Udaybhai, DP (2022) Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm. In: IEEE International Conference on Intelligent Robots and Systems, 23 - 27 October 2022, Kyoto, pp. 2483-2489.
|
PDF
IROS_2022.pdf - Published Version Download (3MB) | Preview |
Abstract
Solving complex problems using reinforcement learning necessitates breaking down the problem into manageable tasks, and learning policies to solve these tasks. These policies, in turn, have to be controlled by a master policy that takes high-level decisions. Hence learning policies involves hierarchical decision structures. However, training such methods in practice may lead to poor generalization, with either sub-policies executing actions for too few time steps or devolving into a single policy altogether. In our work, we introduce an alternative approach to learn such skills sequentially without using an overarching hierarchical policy. We propose this method in the context of environments where a major component of the objective of a learning agent is to prolong the episode for as long as possible. We refer to our proposed method as Sequential Soft Option Critic. We demonstrate the utility of our approach on navigation and goal-based tasks in a flexible simulated 3D navigation environment that we have developed. We also show that our method outperforms prior methods such as Soft Actor-Critic and Soft Option Critic on various environments, including the Atari River Raid environment and the Gym-Duckietown self-driving car simulator. © 2022 IEEE.
| Item Type: | Conference Paper |
|---|---|
| Publication: | IEEE International Conference on Intelligent Robots and Systems |
| Publisher: | Institute of Electrical and Electronics Engineers Inc. |
| Additional Information: | The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc. |
| Keywords: | Air navigation; Learning algorithms; Learning systems, Breakings; Complex problems; Generalisation; Hierarchical decisions; Learning policy; Learning skills; Multi policies; Reinforcement learning algorithms; Reinforcement learnings; Time step, Reinforcement learning |
| Department/Centre: | Division of Electrical Sciences > Computer Science & Automation |
| Date Deposited: | 09 Feb 2023 11:21 |
| Last Modified: | 09 Feb 2023 11:21 |
| URI: | https://eprints.iisc.ac.in/id/eprint/80125 |
Actions (login required)
 |
View Item |
