Zero-sum games for discrete-time multi-armed bandit processes with a generalized discount

The present paper deals with zero-sum games for multi-armed bandid processes and solves them as control problems of multi-parameter Markov processes. This paper extends the results of Lawler-Vanderbei to zero-sum games with discounts which depend on transition of states. The aim of this paper is to give unique optimal values and the optimal Markov strategies which are constructively provided by Bellman’s equation derived from a value iteration.