  The problem of minimizing an objective that can be written as the sum of a
set of $n$ smooth and strongly convex functions is considered. The Incremental
Quasi-Newton (IQN) method proposed here belongs to the family of stochastic and
incremental methods that have a cost per iteration independent of $n$. IQN
iterations are a stochastic version of BFGS iterations that use memory to
reduce the variance of stochastic approximations. The convergence properties of
IQN bridge a gap between deterministic and stochastic quasi-Newton methods.
Deterministic quasi-Newton methods exploit the possibility of approximating the
Newton step using objective gradient differences. They are appealing because
they have a smaller computational cost per iteration relative to Newton's
method and achieve a superlinear convergence rate under customary regularity
assumptions. Stochastic quasi-Newton methods utilize stochastic gradient
differences in lieu of actual gradient differences. This makes their
computational cost per iteration independent of the number of objective
functions $n$. However, existing stochastic quasi-Newton methods have sublinear
or linear convergence at best. IQN is the first stochastic quasi-Newton method
proven to converge superlinearly in a local neighborhood of the optimal
solution. IQN differs from state-of-the-art incremental quasi-Newton methods in
three aspects: (i) The use of aggregated information of variables, gradients,
and quasi-Newton Hessian approximation matrices to reduce the noise of gradient
and Hessian approximations. (ii) The approximation of each individual function
by its Taylor's expansion in which the linear and quadratic terms are evaluated
with respect to the same iterate. (iii) The use of a cyclic scheme to update
the functions in lieu of a random selection routine. We use these fundamental
properties of IQN to establish its local superlinear convergence rate.
