Evaluating strategies for Markov decision processes in parallel

The authors’ work on the use of Gittins indices in the evaluation of strategies for families of alternative bandit processes has found many applications. Among these are procedures for sensitivity analysis in stochastic scheduling. This theoretical paper aims at developing results which will form the basis of an approach to strategy evaluation for a class of processes of greater complexity. These are Markov decision processes in parallel satisfying a condition first enunciated by Whittle. The theory of Gittins indices forms the basis of the analysis.