Analysis of practical step size selection in stochastic approximation algorithms

For many popular stochastic approximation algorithms, such as the stochastic gradient method and the simultaneous perturbation stochastic approximation method, the practical gain sequence selection is different from the optimal selection, that is theoretically derived from asymptotical performance. We provide formal justification for the reasons why we choose such gain sequence in practice.