Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Deep Neural Networks as Gaussian Processes   It has long been known that a single-layer fully-connected neural network
with an i.i.d. prior over its parameters is equivalent to a Gaussian process
(GP), in the limit of infinite network width. This correspondence enables exact
Bayesian inference for infinite width neural networks on regression tasks by
means of evaluating the corresponding GP. Recently, kernel functions which
mimic multi-layer random neural networks have been developed, but only outside
of a Bayesian framework. As such, previous work has not identified that these
kernels can be used as covariance functions for GPs and allow fully Bayesian
prediction with a deep neural network.
  In this work, we derive the exact equivalence between infinitely wide deep
networks and GPs. We further develop a computationally efficient pipeline to
compute the covariance function for these GPs. We then use the resulting GPs to
perform Bayesian inference for wide deep neural networks on MNIST and CIFAR-10.
We observe that trained neural network accuracy approaches that of the
corresponding GP with increasing layer width, and that the GP uncertainty is
strongly correlated with trained network prediction error. We further find that
test performance increases as finite-width trained networks are made wider and
more similar to a GP, and thus that GP predictions typically outperform those
of finite-width networks. Finally we connect the performance of these GPs to
the recent theory of signal propagation in random neural networks.

31
The categories of this paper are: ao-sci
18
The categories of this paper are: econ
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol
29
The categories of this paper are: funct-an
0
The categories of this paper are: hep-ph
5
The categories of this paper are: gr-qc
4
The categories of this paper are: cond-mat
7
The categories of this paper are: hep-th
28
The categories of this paper are: alg-geom
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
24
The categories of this paper are: dg-ga
14
The categories of this paper are: math-ph
2
The categories of this paper are: math
6
The categories of this paper are: astro-ph