Structured policies in the sequential design of experiments

A general control model under uncertainty is considered. Using a Bayesian approach and dynamic programming, the authors investigate structural properties of optimal decision rules. In particular, they show the monotonicity of the total expected reward and of the so-called Gittins-Index. The authors extend the stopping rule and the stay-on-a-winner rule, which are well-known in bandit problems. The present approach is based on the multivariate likelihood ratio order and TP2 functions.