Borkar, Vivek S and Konda, Vijaymohan R (1997) The actor-critic algorithm as multi-time-scale stochastic approximation. In: Sadhana : Academy Proceedings in Engineering Sciences, 22 (part 4). pp. 525-543.