Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Correlation Functions in Random Fully Connected Neural Networks at
  Finite Width   This article considers fully connected neural networks with Gaussian random
weights and biases and $L$ hidden layers, each of width proportional to a large
parameter $n$. For polynomially bounded non-linearities we give sharp estimates
in powers of $1/n$ for the joint correlation functions of the network output
and its derivatives. Moreover, we obtain exact layerwise recursions for these
correlation functions and solve a number of special cases for classes of
non-linearities including $\mathrm{ReLU}$ and $\tanh$. We find in both cases
that the depth-to-width ratio $L/n$ plays the role of an effective network
depth, controlling both the scale of fluctuations at individual neurons and the
size of inter-neuron correlations. We use this to study a somewhat simplified
version of the so-called exploding and vanishing gradient problem, proving that
this particular variant occurs if and only if $L/n$ is large. Several of the
key ideas in this article were first developed at a physics level of rigor in a
recent monograph with Roberts and Yaida.

17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
10
The categories of this paper are: q-bio
26
The categories of this paper are: bayes-an
13
The categories of this paper are: quant-ph
28
The categories of this paper are: alg-geom
37
The categories of this paper are: supr-con
30
The categories of this paper are: q-alg
15
The categories of this paper are: nucl-ex
6
The categories of this paper are: astro-ph
5
The categories of this paper are: gr-qc
7
The categories of this paper are: hep-th
36
The categories of this paper are: cmp-lg
11
The categories of this paper are: hep-lat
25
The categories of this paper are: patt-sol
3
The categories of this paper are: physics