Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: On the expected behaviour of noise regularised deep neural networks as
  Gaussian processes   Recent work has established the equivalence between deep neural networks and
Gaussian processes (GPs), resulting in so-called neural network Gaussian
processes (NNGPs). The behaviour of these models depends on the initialisation
of the corresponding network. In this work, we consider the impact of noise
regularisation (e.g. dropout) on NNGPs, and relate their behaviour to signal
propagation theory in noise regularised deep neural networks. For ReLU
activations, we find that the best performing NNGPs have kernel parameters that
correspond to a recently proposed initialisation scheme for noise regularised
ReLU networks. In addition, we show how the noise influences the covariance
matrix of the NNGP, producing a stronger prior towards simple functions away
from the training points. We verify our theoretical findings with experiments
on MNIST and CIFAR-10 as well as on synthetic data.

13
The categories of this paper are: quant-ph
20
The categories of this paper are: acc-phys
26
The categories of this paper are: bayes-an
35
The categories of this paper are: mtrl-th
19
The categories of this paper are: eess
24
The categories of this paper are: dg-ga
34
The categories of this paper are: chem-ph
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
12
The categories of this paper are: nucl-th
6
The categories of this paper are: astro-ph
25
The categories of this paper are: patt-sol
3
The categories of this paper are: physics
32
The categories of this paper are: atom-ph
7
The categories of this paper are: hep-th
18
The categories of this paper are: econ