Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Universal Approximation with Deep Narrow Networks   The classical Universal Approximation Theorem holds for neural networks of
arbitrary width and bounded depth. Here we consider the natural `dual' scenario
for networks of bounded width and arbitrary depth. Precisely, let $n$ be the
number of inputs neurons, $m$ be the number of output neurons, and let $\rho$
be any nonaffine continuous function, with a continuous nonzero derivative at
some point. Then we show that the class of neural networks of arbitrary depth,
width $n + m + 2$, and activation function $\rho$, is dense in $C(K;
\mathbb{R}^m)$ for $K \subseteq \mathbb{R}^n$ with $K$ compact. This covers
every activation function possible to use in practice, and also includes
polynomial activation functions, which is unlike the classical version of the
theorem, and provides a qualitative difference between deep narrow networks and
shallow wide networks. We then consider several extensions of this result. In
particular we consider nowhere differentiable activation functions, density in
noncompact domains with respect to the $L^p$-norm, and how the width may be
reduced to just $n + m + 1$ for `most' activation functions.

15
The categories of this paper are: nucl-ex
18
The categories of this paper are: econ
30
The categories of this paper are: q-alg
4
The categories of this paper are: cond-mat
14
The categories of this paper are: math-ph
6
The categories of this paper are: astro-ph
3
The categories of this paper are: physics
24
The categories of this paper are: dg-ga
11
The categories of this paper are: hep-lat
20
The categories of this paper are: acc-phys
13
The categories of this paper are: quant-ph
27
The categories of this paper are: comp-gas
23
The categories of this paper are: solv-int
12
The categories of this paper are: nucl-th
8
The categories of this paper are: hep-ex
37
The categories of this paper are: supr-con