Let {Zn} be a Markovian process, the transition of which depends on a control parameter x. Let μx be its invariant law. It is shown that the solution of the optimization problem F(x):¸=∫H(x,z)dμx(z)=min!, x∈S can be found with a recursive estimation procedure of the stochastic approximation-type. The method consists in finding a stochastic quasigradient of F(x) and in adapting the parameter x in the direction of descent. An a.s. convergence is proved and a practical example is given.
