RNN for Parameter Estimation

The idea is to estimate the parameters of a dynamic system

5c = Ax + Bu; x(0) = x0

Error

elements of A and B, and we obtain expressions dE/dA and dE/dB for A and B vectors, respectively, with ^ (•) = ^^=1 (•), after simplification:

(9.73)

(9.74)

dE dA dE dB

— = ^xxT + ^ uxT — ^2 xxT – A У^ xu + B Eu2 — E x u In terms of individual elements of the matrices (for A(2,2) and B(2,1)), we obtain

(9.75)

dE d я11 dE d Я12 dE d Я21 dE d Я22

Я11 y^x^ + —-b1 E x1u x 1×1 ЯЦ УУ x1x2 + ••• + b^ УУ ux2 У^ x 1×2 Я21 Ex?+ ^+b2E ux1 — E x 2×1 Я21 E X1X2 ————————– b2 УУ ux2 — УУ. r2x2

(9.76)

dE db1 dE db2

Я11 УУ x1u ——————————— b1 Eu2 — E x 1 u Я21 УУ x1u ——————————- b2 Eu2 — E x2u

(9.77)

j=1

+ hi, j=i

and since bi = f (Xi), and Xi = (f-1)’b,• dPi

we have, (f 1)’/3,• = —; hence

Comparing expressions from Equations 9.76 and 9.77 to Equation 9.80, the expres­sions for the weight matrix W and the bias vector b are obtained as

(9.83)

(9.84)

The algorithm for parameter estimation of a dynamic system is given as follows: (1) as the measurements of X, X, and u are available for a certain time interval T, compute W matrix and bias vector b; (2) choose initial values of b randomly; and (3) solve the following differential equation.

Since bi = f (X;) and the sigmoid nonlinearities are known, by differentiating and simplifying, we get

Integration of Equation 9.85 would yield the solution to the parameter-estimation problem posed in equation error/RNN structure. Proper tuning of A and p is

essential. Often l is chosen as a small number, i. e., less than 1.0. The value of p is chosen such that when xt (of RNN) approaches ±1, f approaches ± p.