Article ID: | iaor200935012 |
Country: | United States |
Volume: | 31 |
Issue: | 1 |
Start Page Number: | 199 |
End Page Number: | 216 |
Publication Date: | Feb 2006 |
Journal: | Mathematics of Operations Research |
Authors: | Hannig Jan, Chong Edwin K P, Kulkarni Sanjeev R |
Keywords: | markov processes |
We consider a class of nonhomogeneous Markov chains arising in simulated annealing and related stochastic search algorithms. Using only elementary first principles, we analyze the convergence and rate of convergence of the relative frequencies of visits to states in the Markov chain. We describe in detail three examples, including the standard simulated annealing algorithm, to show how our framework applies to specific stochastic search algorithms—these examples have not previously been recognized to be sufficiently similar to share common analytical grounds. Our analysis, though elementary, provides the strongest sample path convergence results to date for simulated annealing–type Markov chains. Our results serve to illustrate that by taking a purely sample path view, surprisingly strong statements can be made using only relatively elementary tools.