Bhatnagar, Shalabh and Panigrahi, Ranjan J (2006) Actor-critic algorithms for hierarchical Markov decision processes. In: Automatica, 42 (4). pp. 637-644.