Optimization of the transient and steady-state behavior of discrete event systems

The paper presents a general framework for applying simulation to optimize the behavior of discrete event systems. The present approach involves modeling the discrete event system under study as a general state space Markov chain whose distribution depends on the decision parameters. The paper then shows how simulation and the likelihood ratio method can be used to evaluate the performance measure of interest and its gradient, and it presents conditions that guarantee that the Robbins-Monro stochastic approximation algorithm will converge almost surely to the optimal values of the decision parameters. Both transient and steady-state performance measures are considered. For steady-state performance measures, the paper considers both the case when the Markov chain of interest is regenerative in the standard sense, as well as the case when this Markov chain is Harris recurrent, and thereby regenerative in a wider sense.