Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The capacity of feedforward neural networks   A long standing open problem in the theory of neural networks is the
development of quantitative methods to estimate and compare the capabilities of
different architectures. Here we define the capacity of an architecture by the
binary logarithm of the number of functions it can compute, as the synaptic
weights are varied. The capacity provides an upper bound on the number of bits
that can be extracted from the training data and stored in the architecture
during learning. We study the capacity of layered, fully-connected,
architectures of linear threshold neurons with $L$ layers of size $n_1,n_2,
\ldots, n_L$ and show that in essence the capacity is given by a cubic
polynomial in the layer sizes: $C(n_1,\ldots, n_L)=\sum_{k=1}^{L-1}
\min(n_1,\ldots,n_k)n_kn_{k+1}$, where layers that are smaller than all
previous layers act as bottlenecks. In proving the main result, we also develop
new techniques (multiplexing, enrichment, and stacking) as well as new bounds
on the capacity of finite sets. We use the main result to identify
architectures with maximal or minimal capacity under a number of natural
constraints. This leads to the notion of structural regularization for deep
architectures. While in general, everything else being equal, shallow networks
compute more functions than deep networks, the functions computed by deep
networks are more regular and "interesting".

21
The categories of this paper are: adap-org
25
The categories of this paper are: patt-sol
20
The categories of this paper are: acc-phys
10
The categories of this paper are: q-bio
22
The categories of this paper are: chao-dyn
13
The categories of this paper are: quant-ph
14
The categories of this paper are: math-ph
36
The categories of this paper are: cmp-lg
18
The categories of this paper are: econ
31
The categories of this paper are: ao-sci
34
The categories of this paper are: chem-ph
35
The categories of this paper are: mtrl-th
0
The categories of this paper are: hep-ph
24
The categories of this paper are: dg-ga
19
The categories of this paper are: eess
11
The categories of this paper are: hep-lat