This paper studies discrete-time nonlinear controlled stochastic systems, modeled by controlled Markov chains (CMC) with denumerable state space and compact action space, and with an infinite planning horizon. Recently, there has been a renewed interest in CMC with a long-run, expected average cost (AC) optimality criterion. A classical approach to study average optimality consists in formulating the AC case as a limit of the discounted cost case, as the discount factor increases to 1, i.e., as the discounting effect vanishes. This approach has been rekindled in recent years, with the introduction by Sennott and others of conditions under which AC optimal stationary policies are shown to exist. However, AC optimality is a rather underselective criterion, which completely neglects the finite-time evolution of the controlled process. The present main interest in this paper is to study the relation between the notions of AC optimality and strong average cost (SAC) optimality. The latter criterion is introduced to assess the performance of a policy over long but finite horizons, as well as in the long-run average sense. The authors show that for bounded one-stage cost functions, Sennott’s conditions are sufficient to guarantee that every AC optimal policy is also SAC optimal. On the other hand, a detailed counterexample is given that shows that the latter result does not extend to the case of unbounded cost functions. In this counterexample, Sennott’s conditions are verified and a policy is exhibited that is both average and Blackwell optimal and satisfies the average cost inequality.
