Stochastic optimization by simulation: Numerical experiments with the M/M/1 queue in steady-state

This paper gives numerical illustrations of the behavior of stochastic approximation, combined with different derivative estimation techniques, to optimize a steady-state system. It is a companion paper to L’Ecuyer and Glynn, which gives convergence proofs for most of the variants experimented here. The numerical experiments are made with a simple M/M/1 queue, which while simple, serves to illustrate the basic convergence properties and possible pitfalls of the various techniques.