Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: A priori generalization error for two-layer ReLU neural network through
  minimum norm solution   We focus on estimating \emph{a priori} generalization error of two-layer ReLU
neural networks (NNs) trained by mean squared error, which only depends on
initial parameters and the target function, through the following research
line. We first estimate \emph{a priori} generalization error of finite-width
two-layer ReLU NN with constraint of minimal norm solution, which is proved by
\cite{zhang2019type} to be an equivalent solution of a linearized (w.r.t.
parameter) finite-width two-layer NN. As the width goes to infinity, the
linearized NN converges to the NN in Neural Tangent Kernel (NTK) regime
\citep{jacot2018neural}. Thus, we can derive the \emph{a priori} generalization
error of two-layer ReLU NN in NTK regime. The distance between NN in a NTK
regime and a finite-width NN with gradient training is estimated by
\cite{arora2019exact}. Based on the results in \cite{arora2019exact}, our work
proves an \emph{a priori} generalization error bound of two-layer ReLU NNs.
This estimate uses the intrinsic implicit bias of the minimum norm solution
without requiring extra regularity in the loss function. This \emph{a priori}
estimate also implies that NN does not suffer from curse of dimensionality, and
a small generalization error can be achieved without requiring exponentially
large number of neurons. In addition the research line proposed in this paper
can also be used to study other properties of the finite-width network, such as
the posterior generalization error.

0
The categories of this paper are: hep-ph
11
The categories of this paper are: hep-lat
27
The categories of this paper are: comp-gas
17
The categories of this paper are: q-fin
15
The categories of this paper are: nucl-ex
37
The categories of this paper are: supr-con
13
The categories of this paper are: quant-ph
26
The categories of this paper are: bayes-an
29
The categories of this paper are: funct-an
30
The categories of this paper are: q-alg
31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
34
The categories of this paper are: chem-ph
28
The categories of this paper are: alg-geom
2
The categories of this paper are: math
6
The categories of this paper are: astro-ph