Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Lipschitz Adaptivity with Multiple Learning Rates in Online Learning   We aim to design adaptive online learning algorithms that take advantage of
any special structure that might be present in the learning task at hand, with
as little manual tuning by the user as possible. A fundamental obstacle that
comes up in the design of such adaptive algorithms is to calibrate a so-called
step-size or learning rate hyperparameter depending on variance, gradient
norms, etc. A recent technique promises to overcome this difficulty by
maintaining multiple learning rates in parallel. This technique has been
applied in the MetaGrad algorithm for online convex optimization and the Squint
algorithm for prediction with expert advice. However, in both cases the user
still has to provide in advance a Lipschitz hyperparameter that bounds the norm
of the gradients. Although this hyperparameter is typically not available in
advance, tuning it correctly is crucial: if it is set too small, the methods
may fail completely; but if it is taken too large, performance deteriorates
significantly. In the present work we remove this Lipschitz hyperparameter by
designing new versions of MetaGrad and Squint that adapt to its optimal value
automatically. We achieve this by dynamically updating the set of active
learning rates. For MetaGrad, we further improve the computational efficiency
of handling constraints on the domain of prediction, and we remove the need to
specify the number of rounds in advance.

18
The categories of this paper are: econ
17
The categories of this paper are: q-fin
21
The categories of this paper are: adap-org
31
The categories of this paper are: ao-sci
32
The categories of this paper are: atom-ph
6
The categories of this paper are: astro-ph
30
The categories of this paper are: q-alg
34
The categories of this paper are: chem-ph
14
The categories of this paper are: math-ph
2
The categories of this paper are: math
10
The categories of this paper are: q-bio
7
The categories of this paper are: hep-th
26
The categories of this paper are: bayes-an
23
The categories of this paper are: solv-int
27
The categories of this paper are: comp-gas
3
The categories of this paper are: physics