Stochastic optimization of discrete-event systems simulation

The Score Function method has been proposed to estimate the gradient of a performance measure with respect to some continuous parameters in a stochastic system. In this paper we experiment with the use of this estimate in a stochastic approximation algorithm to perform a single-run optimization. The experiment is done on a simple M/M/1 queue. The performance measure involves the average system time per customer at steady-state, and the decision variable is the service rate. The optimal solution is easy to compute analytically, which facilitates the evaluation of the algorithm. Combined with appropriate variance reduction techniques, the method has been shown to be effective in the test problem. We study the algorithm's properties and examine the validity of the estimates based on this single run procedure by performing some experimental studies. Implementation ‘details’ necessary for packaging this method with existing simulation software are provided. Finally, there is a set of recommended directions for future research.