Since the gradients of V(\cdot, \cdot) in the incremental gradient descent iterations are also stochastic estimates of the gradient of I_n[w], this interpretation is also related to the stochastic gradient descent method, but applied to minimize the empirical risk as opposed to the expected risk. Since this interpretation concerns the empirical risk and not the expected risk, multiple passes through the data are readily allowed and actually lead to tighter bounds on the deviations I_n[w_t] - I_n[w^\ast_n], where w^\ast_n is the minimizer of I_n[w].
