Sample-path optimality and variance-minimization of average cost Markov control processes

This paper studies several average-cost criteria for Markov control processes on Borel spaces with possibly unbounded costs. Under suitable hypotheses we show (i) the existence of a sample-path average cost (SPAC-)optimal stationary policy; (ii) a stationary policy is SPAC-optimal if and only if it is expected average cost (EAC-)optimal; and (iii) within the class of stationary SPAC-optimal (equivalently, EAC-optimal) policies there exists one with a minimal limiting average variance.