We first derive from abstract results on Feller transition kernels that, under some mild assumptions, a Markov stochastic algorithm with constant step size ε usually has a tight family of invariant distributions ν(ε), ε is an element of (0;ε0], whose weak limiting distributions as ε ↓ 0 are all flow-invariant for its ordinary differential equation (ODE). Then the main part of the paper deals with a kind of converse: what are the possible limiting distributions among all flow-invariant distributions of the ODE? We first show that no repulsive invariant (thin) set can belong to their supports. When the ODE is a stochastic pseudogradient descent, these supports cannot contain saddle or spurious equilibrium points either, so that they are eventually supported by the set of local minima of their potential. Such results require only the random perturbation to lie in L2. Various examples are treated, showing that these results yield some weak convergence results for the ν(ε)s, sometimes toward a saddle point when the algorithm is not a pseudogradient.
