The compactness of a policy space in dynamic programming via an extension theorem for Caratheodory functions

The background for this paper is a dynamic programming model with a Borel state space and compact action sets. A new simple proof of the compactness of a space of measures corresponding to randomized stationary policies is given, in a topology induced by Caratheodory functions on the state-action space. For this purpose an extension theorem for Caratheodory functions is proved, which is a new version of the Tietze's extension theorem for continuous functions.