ptive-stepsize-sequence" class="level3 unnumbered">
<h3 class="unnumbered anchored" data-anchor-id="adaptive-stepsize-sequence">Adaptive stepsize sequence</h3>
<p>ADVI uses a finite-memory version of adaGrad <span class="citation" data-cites="Duchi:2011">Duchi, Hazan, and Singer (<a href="#ref-Duchi:2011" role="doc-biblioref">2011</a>)</span>. This has a single parameter that we expose, denoted <code>eta</code>. We now have a warmup adaptation phase that selects a good value for <code>eta</code>. The proced