Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Neural Network with Unbounded Activation Functions is Universal
  Approximator   This paper presents an investigation of the approximation property of neural
networks with unbounded activation functions, such as the rectified linear unit
(ReLU), which is the new de-facto standard of deep learning. The ReLU network
can be analyzed by the ridgelet transform with respect to Lizorkin
distributions. By showing three reconstruction formulas by using the Fourier
slice theorem, the Radon transform, and Parseval's relation, it is shown that a
neural network with unbounded activation functions still satisfies the
universal approximation property. As an additional consequence, the ridgelet
transform, or the backprojection filter in the Radon domain, is what the
network learns after backpropagation. Subject to a constructive admissibility
condition, the trained network can be obtained by simply discretizing the
ridgelet transform, without backpropagation. Numerical examples not only
support the consistency of the admissibility condition but also imply that some
non-admissible cases result in low-pass filtering.

26
The categories of this paper are: bayes-an
13
The categories of this paper are: quant-ph
15
The categories of this paper are: nucl-ex
32
The categories of this paper are: atom-ph
25
The categories of this paper are: patt-sol
12
The categories of this paper are: nucl-th
3
The categories of this paper are: physics
34
The categories of this paper are: chem-ph
27
The categories of this paper are: comp-gas
22
The categories of this paper are: chao-dyn
18
The categories of this paper are: econ
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
6
The categories of this paper are: astro-ph
4
The categories of this paper are: cond-mat
31
The categories of this paper are: ao-sci