Adaptive policies for time-varying stochastic systems under discounted criterion

We consider a class of time-varying stochastic control systems, with Borel state and action spaces, and possibly unbounded costs. The processes evolve according to a discrete-time equation xn+1 = Gn(xn,an,ζn), n = 0,1,..., where the ζn are i.i.d. ℝ^k-valued random vectors whose common density is unknown, and the Gn are given functions converging, in a restricted way, to some function G∞ as n → ∞. Assuming observability of ζn, we construct an adaptive policy which is asymptotically discounted cost optimal for the limiting control system xn+1 = G∞(xn,an,ζn).