Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Optimal Rates for Averaged Stochastic Gradient Descent under Neural
  Tangent Kernel Regime   We analyze the convergence of the averaged stochastic gradient descent for
overparameterized two-layer neural networks for regression problems. It was
recently found that a neural tangent kernel (NTK) plays an important role in
showing the global convergence of gradient-based methods under the NTK regime,
where the learning dynamics for overparameterized neural networks can be almost
characterized by that for the associated reproducing kernel Hilbert space
(RKHS). However, there is still room for a convergence rate analysis in the NTK
regime. In this study, we show that the averaged stochastic gradient descent
can achieve the minimax optimal convergence rate, with the global convergence
guarantee, by exploiting the complexities of the target function and the RKHS
associated with the NTK. Moreover, we show that the target function specified
by the NTK of a ReLU network can be learned at the optimal convergence rate
through a smooth approximation of a ReLU network under certain conditions.

0
The categories of this paper are: hep-ph
31
The categories of this paper are: ao-sci
34
The categories of this paper are: chem-ph
26
The categories of this paper are: bayes-an
15
The categories of this paper are: nucl-ex
24
The categories of this paper are: dg-ga
37
The categories of this paper are: supr-con
28
The categories of this paper are: alg-geom
12
The categories of this paper are: nucl-th
14
The categories of this paper are: math-ph
32
The categories of this paper are: atom-ph
2
The categories of this paper are: math
29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
25
The categories of this paper are: patt-sol
33
The categories of this paper are: plasm-ph