Constrained Markov decision processes with total cost criteria: Occupation measures and primal LP

This paper is the third in a series on constrained Markov decision processes with a countable state space and unbounded cost. The previous paper studied the expected average and the discounted cost. It analyzes in this paper the total cost criterion. The paper studies the properties of the set of occupation measures achieved by differnt classes of policies; it then focuses on stationary policies and on mixed deterministic policies and present conditions under which optimal policies exist within these classes. The paper concludes by introducing an equivalent infinite Linear Program.