Convergence of simulation-based policy iteration

Convergence of simulation-based policy iteration

0.00 Avg rating0 Votes
Article ID: iaor2005378
Country: United Kingdom
Volume: 17
Issue: 2
Start Page Number: 213
End Page Number: 234
Publication Date: Apr 2003
Journal: Probability in the Engineering and Informational Sciences
Authors: , ,
Keywords: markov processes
Abstract:

Simulation-based policy iteration (SBPI) is a modification of the policy iteration algorithm for computing optimal policies for Markov decision processes. At each iteration, rather than solving the average evaluation equations, SBPI employs simulation to estimate a solution to these equations. For recurrent average-reward Markov decision processes with finite state and action spaces, we provide easily verifiable conditions that ensure that simulation-based policy iteration almost-surely eventually never leaves the set of optimal decision rules. We analyze three simulation estimators for solutions to the average evaluation equations. Using our general results, we derive simple conditions on the simulation run lengths that guarantee the almost-sure convergence of the algorithm.

Reviews

Required fields are marked *. Your email address will not be published.