Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Backtracking Gradient Descent allowing unbounded learning rates   In unconstrained optimisation on an Euclidean space, to prove convergence in
Gradient Descent processes (GD) $x_{n+1}=x_n-\delta _n \nabla f(x_n)$ it
usually is required that the learning rates $\delta _n$'s are bounded: $\delta
_n\leq \delta $ for some positive $\delta $. Under this assumption, if the
sequence $x_n$ converges to a critical point $z$, then with large values of $n$
the update will be small because $||x_{n+1}-x_n||\lesssim ||\nabla f(x_n)||$.
This may also force the sequence to converge to a bad minimum. If we can allow,
at least theoretically, that the learning rates $\delta _n$'s are not bounded,
then we may have better convergence to better minima.
  A previous joint paper by the author showed convergence for the usual version
of Backtracking GD under very general assumptions on the cost function $f$. In
this paper, we allow the learning rates $\delta _n$ to be unbounded, in the
sense that there is a function $h:(0,\infty)\rightarrow (0,\infty )$ such that
$\lim _{t\rightarrow 0}th(t)=0$ and $\delta _n\lesssim \max \{h(x_n),\delta \}$
satisfies Armijo's condition for all $n$, and prove convergence under the same
assumptions as in the mentioned paper. It will be shown that this growth rate
of $h$ is best possible if one wants convergence of the sequence $\{x_n\}$.
  A specific way for choosing $\delta _n$ in a discrete way connects to Two-way
Backtracking GD defined in the mentioned paper. We provide some results which
either improve or are implicitly contained in those in the mentioned paper and
another recent paper on avoidance of saddle points.

27
The categories of this paper are: comp-gas
11
The categories of this paper are: hep-lat
24
The categories of this paper are: dg-ga
20
The categories of this paper are: acc-phys
17
The categories of this paper are: q-fin
19
The categories of this paper are: eess
3
The categories of this paper are: physics
32
The categories of this paper are: atom-ph
23
The categories of this paper are: solv-int
31
The categories of this paper are: ao-sci
21
The categories of this paper are: adap-org
4
The categories of this paper are: cond-mat
29
The categories of this paper are: funct-an
12
The categories of this paper are: nucl-th
15
The categories of this paper are: nucl-ex
34
The categories of this paper are: chem-ph