Back Propagation Through Adjoints for the Identification of Nonlinear Dynamic Systems Using Recurrent Neural Models

Srinivasan, B and Prasad, UR and Rao, NJ (1994) Back Propagation Through Adjoints for the Identification of Nonlinear Dynamic Systems Using Recurrent Neural Models. In: IEEE Transactions on Neural Networks, 5 (2). pp. 213-228.

Preview

PDF back.pdf Download (1MB)

Abstract

In this paper, back propagation is reinvestigated for an efficient evaluation of the gradient in arbitrary interconnections of recurrent subsystems. It is shown that the error has to be back-propagated through the adjoint model of the system and that the gradient can only be obtained after a delay. A faster version, accelerated back propagation, that eliminates this delay, is also developed. Various schemes including the sensitivity method are studied to update the weights of the network using these gradients. Motivated by the Lyapunov approach and the adjoint model, the predictive back propagation and its variant, targeted back propagation, are proposed. A further refinement, predictive back propagation with filtering is then developed, where the states of the model are also updated. The convergence of this scheme is assured. It is shown that it is sufficient to back propagate as many time steps as the order of the system for convergence. As a preamble, convergence of online batch and sample-wise updates in feedforward models is analyzed using the Lyapunov approach.

Item Type:	Journal Article
Publication:	IEEE Transactions on Neural Networks
Publisher:	IEEE
Additional Information:	Copyright 1990 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	25 Aug 2008
Last Modified:	19 Sep 2010 04:27
URI:	http://eprints.iisc.ac.in/id/eprint/6934

Actions (login required)

View Item