Least mean square algorithms with switched Markov ODE limit

TitleLeast mean square algorithms with switched Markov ODE limit
Publication TypeConference Paper
Year of Publication2004
AuthorsKrishnamurthy, V., and G. Yin
Conference NameDecision and Control, 2004. CDC. 43rd IEEE Conference on
Pagination4134 - 4139 Vol.4
Date Publisheddec.
Keywordscontinuous time interpolation, continuous-time interpolation, continuous-time Markov chain, differential equations, error sequence, interpolation, least mean square algorithm, least mean squares methods, Markov processes, ordinary differential equations, parameter tracking, regime switching, regression vectors, stochastic approximation, switched Markov ODE limit, time-varying systems, tracking errors

We analyze the tracking performance of a least mean square (LMS) algorithm for tracking a parameter that evolves according to a Markov chain with infrequent jumps. By allowing the Markov chain to evolve as the same rate of change as the LMS algorithm, we use a combined approach of two-time-scale Markov chains and stochastic approximation method to derive the limit dynamics satisfied by continuous-time interpolation of the estimates. Unlike most previous analyses of stochastic approximation algorithms, the limit we obtain is a system of ordinary differential equations with regime switching controlled by a continuous-time Markov chain. To further analyze the tracking errors, we take a continuous-time interpolation of a scaled sequence of the error sequence and derive its diffusion limit. Somewhat remarkably, for correlated regression vectors we obtain a system of switching diffusions.


a place of mind, The University of British Columbia

Electrical and Computer Engineering
2332 Main Mall
Vancouver, BC Canada V6T 1Z4
Tel +1.604.822.2872
Fax +1.604.822.5949

Emergency Procedures | Accessibility | Contact UBC | © Copyright 2020 The University of British Columbia