Discrete search methods for optimizing stochastic systems

In simulation practice, although estimating the performance of a complex stochastic system is of great value to the decision maker, it is not always enough. For example, a warehouse manager may be interested in finding out the probability that all demands are met from on-hand inventory under a certain system configuration of a fixed safety stock and a fixed order quantity. But he might be more interested in finding out what values of safety stock and order quantity will maximize this probability. In this paper we develop three strategies of a new iterative search procedure for finding the optimal parameters of a stochastic system, where the objective function cannot be evaluated exactly but must be estimated through Monte Carlo simulation. In each iteration, two neighboring configurations are compared and the one that appears to be the better one is passed on to the next iteration. The first strategy of the proposed method uses a single observation of each configuration in every iteration, while the second strategy uses a fixed number of observations of each configuration. In every iteration. The third strategy uses sequential sampling with fixed boundaries. We show that, for all of these three strategies, the search process satisfies local balance equations and its equilibrium distribution gives most weight to the optimal point (when suitably normalized by the size of the neighborhoods). We also show that the configuration that has been visited most often in the first m iterations converges almost surely to an optimum solution.