Hidden Markov model multiarm bandits: a methodology for beam scheduling in multitarget tracking

TitleHidden Markov model multiarm bandits: a methodology for beam scheduling in multitarget tracking
Publication TypeJournal Article
Year of Publication2001
AuthorsKrishnamurthy, V., and R. J. Evans
JournalSignal Processing, IEEE Transactions on
Pagination2893 -2908
Date Publisheddec.
Keywordsarray signal processing, electronically scanned array, electronically scanned array tracking systems, finite-dimensional algorithm, finite-dimensional optimal solution, Gittins index, hidden Markov model multiarm bandits, hidden Markov models, HMM multiarm bandits, intelligent sensor, intelligent sensors, multitarget tracking, optimal beam scheduling algorithms, optimisation, suboptimal algorithms, suboptimal beam scheduling algorithms, target tracking

We derive optimal and suboptimal beam scheduling algorithms for electronically scanned array tracking systems. We formulate the scheduling problem as a multiarm bandit problem involving hidden Markov models (HMMs). A finite-dimensional optimal solution to this multiarm bandit problem is presented. The key to solving any multiarm bandit problem is to compute the Gittins (1989) index. We present a finite-dimensional algorithm that computes the Gittins index. Suboptimal algorithms for computing the Gittins index are also presented. Numerical examples are presented to illustrate the algorithms


a place of mind, The University of British Columbia

Electrical and Computer Engineering
2332 Main Mall
Vancouver, BC Canada V6T 1Z4
Tel +1.604.822.2872
Fax +1.604.822.5949

Emergency Procedures | Accessibility | Contact UBC | © Copyright 2021 The University of British Columbia