Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A Unified View of Regularized Dual Averaging and Mirror Descent with
  Implicit Updates   We study three families of online convex optimization algorithms:
follow-the-proximally-regularized-leader (FTRL-Proximal), regularized dual
averaging (RDA), and composite-objective mirror descent. We first prove
equivalence theorems that show all of these algorithms are instantiations of a
general FTRL update. This provides theoretical insight on previous experimental
observations. In particular, even though the FOBOS composite mirror descent
algorithm handles L1 regularization explicitly, it has been observed that RDA
is even more effective at producing sparsity. Our results demonstrate that
FOBOS uses subgradient approximations to the L1 penalty from previous rounds,
leading to less sparsity than RDA, which handles the cumulative penalty in
closed form. The FTRL-Proximal algorithm can be seen as a hybrid of these two,
and outperforms both on a large, real-world dataset.
  Our second contribution is a unified analysis which produces regret bounds
that match (up to logarithmic terms) or improve the best previously known
bounds. This analysis also extends these algorithms in two important ways: we
support a more general type of composite objective and we analyze implicit
updates, which replace the subgradient approximation of the current loss
function with an exact optimization.

30
The categories of this paper are: q-alg
29
The categories of this paper are: funct-an
13
The categories of this paper are: quant-ph
21
The categories of this paper are: adap-org
22
The categories of this paper are: chao-dyn
10
The categories of this paper are: q-bio
34
The categories of this paper are: chem-ph
19
The categories of this paper are: eess
33
The categories of this paper are: plasm-ph
23
The categories of this paper are: solv-int
17
The categories of this paper are: q-fin
9
The categories of this paper are: nlin
0
The categories of this paper are: hep-ph
12
The categories of this paper are: nucl-th
6
The categories of this paper are: astro-ph
28
The categories of this paper are: alg-geom