Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Concavifiability and convergence: necessary and sufficient conditions
  for gradient descent analysis   Convergence of the gradient descent algorithm has been attracting renewed
interest due to its utility in deep learning applications. Even as multiple
variants of gradient descent were proposed, the assumption that the gradient of
the objective is Lipschitz continuous remained an integral part of the analysis
until recently. In this work, we look at convergence analysis by focusing on a
property that we term as concavifiability, instead of Lipschitz continuity of
gradients. We show that concavifiability is a necessary and sufficient
condition to satisfy the upper quadratic approximation which is key in proving
that the objective function decreases after every gradient descent update. We
also show that any gradient Lipschitz function satisfies concavifiability. A
constant known as the concavifier analogous to the gradient Lipschitz constant
is derived which is indicative of the optimal step size. As an application, we
demonstrate the utility of finding the concavifier the in convergence of
gradient descent through an example inspired by neural networks. We derive
bounds on the concavifier to obtain a fixed step size for a single hidden layer
ReLU network.

24
The categories of this paper are: dg-ga
16
The categories of this paper are: stat
35
The categories of this paper are: mtrl-th
37
The categories of this paper are: supr-con
29
The categories of this paper are: funct-an
30
The categories of this paper are: q-alg
5
The categories of this paper are: gr-qc
32
The categories of this paper are: atom-ph
0
The categories of this paper are: hep-ph
12
The categories of this paper are: nucl-th
19
The categories of this paper are: eess
22
The categories of this paper are: chao-dyn
8
The categories of this paper are: hep-ex
27
The categories of this paper are: comp-gas
11
The categories of this paper are: hep-lat
14
The categories of this paper are: math-ph