Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Universal mean field upper bound for the generalisation gap of deep
  neural networks   Modern deep neural networks (DNNs) represent a formidable challenge for
theorists: according to the commonly accepted probabilistic framework that
describes their performance, these architectures should overfit due to the huge
number of parameters to train, but in practice they do not. Here we employ
results from replica mean field theory to compute the generalisation gap of
machine learning models with quenched features, in the teacher-student scenario
and for regression problems with quadratic loss function. Notably, this
framework includes the case of DNNs where the last layer is optimised given a
specific realisation of the remaining weights. We show how these results --
combined with ideas from statistical learning theory -- provide a stringent
asymptotic upper bound on the generalisation gap of fully trained DNN as a
function of the size of the dataset $P$. In particular, in the limit of large
$P$ and $N_{\textrm{out}} $ (where $N_\textrm{out}$ is the size of the last
layer) and $N_\textrm{out} \ll P$, the generalisation gap approaches zero
faster than $2 N_\textrm{out}/P$, for any choice of both architecture and
teacher function. Notably, this result greatly improves existing bounds from
statistical learning theory. We test our predictions on a broad range of
architectures, from toy fully-connected neural networks with few hidden layers
to state-of-the-art deep convolutional neural networks.

15
The categories of this paper are: nucl-ex
7
The categories of this paper are: hep-th
10
The categories of this paper are: q-bio
31
The categories of this paper are: ao-sci
32
The categories of this paper are: atom-ph
37
The categories of this paper are: supr-con
2
The categories of this paper are: math
12
The categories of this paper are: nucl-th
30
The categories of this paper are: q-alg
14
The categories of this paper are: math-ph
0
The categories of this paper are: hep-ph
17
The categories of this paper are: q-fin
22
The categories of this paper are: chao-dyn
16
The categories of this paper are: stat
29
The categories of this paper are: funct-an
13
The categories of this paper are: quant-ph