Minimizing the learning loss in adaptive control of Markov chains under the weak accessibility condition

The paper considers the adaptive control of Markov chains under the weak accessibility condition with a view to minimizing the learning loss. A certainty equivalence control with a forcing scheme is constructed. The paper uses a stationary randomized control scheme for forcing and computes a maximum likelihood estimate of the unknown parameter from the resulting observations. It obtains an exponential upper bound on the rate of decay of the probability of error. This allows the choice of the rate of forcing appropriately, whereby a o(f(n)logn) learning loss for any function f(n)•• as n•• is achieved.