Criteria for selecting the relaxation factor of the value iteration algorithm for undiscounted Markov and semi-Markov decision processes

The authors present two criteria for selecting the adaptive relaxation factor being used in speeding-up the value iteration algorithm for undiscounted Markov decision processes. The criteria are: 1, Minimum Ratio, 2. Minimum Variance. For the problems tested it was found that the criterion of minimum variance is most effective for Markov models while an hybrid use of both criteria yields best results when solving semi-Markov decision problems. The advantage of using these criteria increases with the dimension of the models.