On performance potentials and conditional Monte Carlo for gradient estimation for Markov chains

We consider the problem of sample path-based gradient estimation for long-run (steady-state) performance measures defined on discrete-time Markov chains. We show how two estimators – one derived using the likelihood ratio method with conditional Monte Carlo and splitting, and the other derived using performance potentials and perturbation analysis – are related. In particular, one can be expressed as the conditional expectation of a suitably weighted average of the other. This demonstrates yet another connection between the two gradient estimation techniques of perturbation analysis and the likelihood ratio method.